Keenan Appiah-Forson's late header was just adjudged to have crossed the line after initially hitting the inside of the right post.
READ MORE: Blues fans have their say on win against Yeovil
And Rowley believes the Blues fans may have played a part in the goal being given.
"I've got a friend behind the goal who is actually a Southend supporter," Rowley told the club's Twitter account.
"I just caught up with him and he said it probably wasn't a goal as well.
"I haven't seen it.
"The angle we're seeing it from isn't great so I can't really comment.
"This is what 8,000 fans do to referees and officials, they influence them.
" I'm not saying it wasn't over the line or it was, I haven't seen it.
"But as a referee in front of 8,000 people it's probably easier to side with the home team at times."
Yeovil's defeat kept them 16th in the standings and Rowley was frustrated to be heading home empty handed.
"We're disappointed," said the Glovers chief.
"Whenever you lose a game of football you don't walk away happy.
"That's my overall feeling at the moment.
"The first half we were really bad and it reminded me a little bit of Woking away.
"Emotionally this place does something different to you than Woking.
"You're playing in front of 8,000 people.
"It's fantastic support and a really, really good team.
"I felt like the occasion got to us a little bit. "
But Rowley was pleased with the improvements made after the break.
"We came out in the second half with nothing to lose," he said.
"We started to find more bravery and composure to get on the ball.
"We looked good and then scored a great goal.
"The game changes again then.
"They're back to being on the front foot with a high line with more pressure on the ball.
"They make four changes and all four players are top level.
"It brings a lot more energy into their press and we were on the tilt at that point.
"We couldn't really get out of our half."
And Rowley was disappointed with how his side reacted at that stage.
"You have two choices when you're under the cosh like that," said Rowley.
"You can go ultra brave and try to keep on playing or you can try to get the ball over the opposition and in their half
"We kind of did nether.
"We were in the middle of that and we found ourselves in an arena which was difficult on the night.
"They're a good team with a good management team and over the course of the 90 minutes they probably earned the win but in the second half I thought we were the better team."