A defeat at Sunderland condemned Wolves to just two points from their opening eight matches as fans vented their anger at the club sitting bottom of the table.
The defender, who has spent the vast majority of his career at the club over two spells, is finding the team's plight painful but he believes they can 'prosper' if they find that elusive first win.
"You have to have accountability in every team and we've got that," Doherty said.
"People might think that we don't, but we definitely have that in the squad.
"We're serious people, we want to do well for Wolves.
"Wolves is my club, I've been there basically my whole career.