But he played down any fears about Jan Paul van Hecke after the defender limped off.
Albion contained Villa well and hit the bar through Ferdi Kadioglu as they started to pose a threat in the second half.
But they were sunk late on when Tyroine Mings’ near-post header at a corner flicked in off Jack Hinshelwood.
The Seagulls’ poor run has been extended to one league win in 13 matches.
Head coach Hurzeler said: “We know that we definitely deserve more, especially after the second half.
"I think the first half was tight, was not a really good football game.
“A lot of personal duels. Second half, we dominated. We controlled the game. We could have scored if you are more ruthless.
“I think in games like this, the team who scores the first goal will win it and unfortunately Aston Villa did it out of nowhere with a set-piece and therefore I feel sorry for the supporters because they were great today, they supported us for 90 minutes.
“I feel sorry for the team but, in the end, we have to take responsibility and we have to keep pushing, we have to keep doing these small margins right.
“And then we get the results and then we will sit here at one point hopefully being a winner.”
The impressive van Hecke went off after an hour, having retuned from a minor hamstring injury.
Hurzeler said: “We know that he could not play for 90 minutes, so that was the normal procedure.
“We had to take him out after 60, 65. That was the maximum of his playing time, what he was capable of doing.
“So therefore we knew that we have to take him off.”