Fabian Hurzeler's men looked like they would be heading to a third league game in a row without a win after Erling Haaland's cool finish 11 minutes before the break.
A wasteful City were made to pay when 39-year-old James Milner scored from the spot in the 67th minute after Matheus Nunes inexplicably handled Lewis Dunk's volley in the box.
And the comeback was complete late on when Gruda brilliantly rounded goalkeeper James Trafford to slot home and give Albion their first Premier League win of the season.
After picking up one point from their opening two Premier League matches, Albion needed a fast start to allay fans' fears. Instead, they lacked precision and cohesion, despite all their huff and puff.
They were fortunate to stay level early on when Norwegian international Haaland was put through on goal, but the striker's shot was uncharacteristically tame and straight at goalkeeper Bart Verbruggen from 12 yards.
At the other end, Danny Welbeck fed the onrushing Kaoru Mitoma, who brought a terrific fingertip save from Trafford in an energetic start to the contest.
A backpedalling Albion breathed a sigh of relief when Haaland fired an effort just wide of the post, before Verbruggen did well to keep out his goal-bound header following a teasing byline cross from Rayan Ait-Nouri.
But City's pressure finally told in the 34th minute when Omar Marmoush nutmegged Dunk, and Haaland then lofted the loose ball past the helpless Verbruggen.
Following a bad-tempered half, where Albion fans vented their fury at referee Darren England's officiating and their side frequently gave the ball away, Hurzeler's players began the second period with a bit more gusto.
The German head coach brought on Gruda, Georginio Rutter, Milner, and Yasin Ayari after the hour mark, and instantly, Albion had more zip about them.
Minteh nearly made it 1-1 after Mitoma threaded an inch-perfect pass into his path, but Trafford stretched his 6ft 6in frame to keep out his strike.
Moments later, referee England was pointing to the spot after Nunes foolishly handled Dunk's fierce volley. Many fans were dismayed when Milner, who turns 40 in January, stepped up to take the penalty but they were singing his name soon after when he sent Trafford the wrong way from 12 yards in the 67th minute.
Albion, who earned four points against City last season, looked the more likely to grab a winner in the game's dying embers, as Rutter cracked a well-hit shot straight down the throat of Trafford.
City were indebted to the former Burnley stopper when he made a remarkable acrobatic save to keep out Jan Paul van Hecke's close-range effort.
But the 22-year-old had no answers to Gruda's brilliance as his quick feet bamboozled the City defence, before he slotted into an empty net as Hurzeler raced down the touchline in delight.
Following a frustrating draw with Fulham and an exasperating defeat to Everton, Albion bagged their first three points of the season to send the fans home happy.
In truth, City, who have now lost two league games in a row, should have been out of sight in the first half, but Haaland's profligacy in front of goal came back to haunt them.
When Pep Guardiola was content to sit on City's narrow lead, Hurzeler rolled the dice and it paid dividends. Now, Albion can go into the international break satisfied after saving their best 45 minutes of the season for the Premier League heavyweights.
Hurzeler will hope his players can bring that same intensity and quality to Bournemouth on September 13.