Raheem Sterling had broken the deadlock just before half-time when he slotted the ball in after Sergio Aguero’s heavy touch from a cutback by Gabriel Jesus.
Aguero then had the opportunity to double City’s lead moments later after Billy Gilmour bundled Jesus over in the area to concede a penalty.
However, the Argentinian forward saw his attempted Panenka easily saved by Edouard Mendy, who stayed on his feet to make a comfortable stop.
It proved a costly error as Chelsea equalised just after the hour when Rodri was caught in possession on the halfway line and and slick Chelsea moved finished with a fine Hakim Ziyech finish.
Timo Werner and Callum Hudson-Odoi both had goals ruled out for offside, while City saw a couple of penalty shouts turned down by referee Anthony Taylor before Alonso's injury time leveller.
Guardiola was not impressed by his side's second half display, but he was convinced his side were robbed of a late penalty when Sterling was fouled in the box by Kurt Zouma.
"It’s a penalty, that’s all," said Guardiola. "It's happens, it’s football. We will try next time to score our penalty. That's all I can say. I saw on the TV it was a penalty."
Sterling was also bemused by the penalty decision, as he questioned why it was not reviewed by the VAR officials: "I couldn't get a shot off because he put his knee on the back of my hamstring.
"I don't know how it's not a penalty or reviewed properly. I thought VAR was here to help. but it wasn't meant to be. The referee said it had been checked and it wasn't a penalty, but it was a sure penalty."
City will now be crowned as Premier League champions if Manchester United lose against Aston Villa on Sunday, with Guardiola keen to play down the significance of a second defeat against Chelsea in quick succession after last month's FA Cup semi-final loss.
"It was a tight game, but in general we played really good. We miss our chance in last minute of first half with the penalty," he added.
"They are a top side and we were playing better. We were in a good position but in the end they score."
"In 21 days we are going to face them again in the Champions League final. We will learn from that and get the points we need to be champions."
Chelsea boss Thomas Tuchel celebrated with his players on the pitch after the game, as his remarkable run of success continues.
"I think the second half was a fantastic response to the last five minutes of the first half where we almost lost it all," said the German.
"To me it was a clear 0-0 half but suddenly we did one decisive mistake and maybe the match becomes out of reach at half-time.
"With every minute we gained confidence and quality. We got better and better. It was a tough battle and once we got the equaliser we never stopped attacking. The winning goal was a huge reward.
"This tells us we can rely on our fitness which is a fantastic feeling at the end of a season. We can also rely on our team spirit, we keep going.
"This team never stops, they don't want to lose, they want it that bad."