The Albion boss was unhappy with the yellow card shown by Peter Bankes to his midfielder inside two minutes.
TV replays also showed a foul by Baleba shortly afterwards.
The head coach quickly prepared James Milner and eventually sent him on midway through the first half.
Milner, equalling Gareth Barry’s record for Premier League appearances, went on to give a strong performance in the 1-0 defeat.
Baleba has previously been taken off at half-time for both club and country after being booked.
But this time Hurzeler had sympathy for his player as he spoke to media.
He said: “We've already spoken. In the end, it’s a bad decision from the referee to give him, in this situation, a yellow card.
“It's a wrong decision for me. I saw a lot of similar fouls afterwards without giving yellow cards so it doesn't make sense for me.
“And I think we have to take responsibility.
“But, in the end, I also think that they have to take responsibility for their decisions.
“I think we have to avoid the second yellow card.
“We have to protect the young player, and we all have to understand what makes these kind of things with a young player.
“We only see always the player, but not the person behind it.
“I saw these fouls so many times in the Premier League and 99% you don't give a yellow card and, therefore, I really don't get it.”