While taping his Thursday show, host Stephen Colbert made the surprising announcement that CBS is ending his late night show in May.

The live audience at New York’s Ed Sullivan Theater booed when he delivered the news that this would be the show’s final season.

“Yeah, I share your feelings,” he told the audience. “It’s not just the end our our show, but it’s the end of The Late Show on CBS. I’m not being replaced. This is all just going away.”