Some Chelsea players seemed to take the defeat especially hard, including defender Wesley Fofana, who looked to be tearing up when he was snapped by the cameras at full-time.
Speaking on Sky Sports, Merson said: "I am numb, I am flabbergasted. I can't believe what I have just watched. I honestly can't believe what I have just watched.
"Arsenal fans and the players themselves...this is Chelsea Football Club. They have got international football players, a World Cup winner. They have got players all over...this is not a bottom four or five team, they have just come here...they should be crying because they never had a go.
"I am lost for words, I cannot believe what's just happened. They have gone out with an absolute whimper in the semi-final of a big competition.
"If it worked it was great but if it hasn't worked, you can't gamble like that. Have a go and go out in a blaze of glory. Don't go out like that. I am lost for words."