Liverpool opened the scoring in the 35th minute through Ryan Gravenberch before the Magpies were reduced to 10 men with Anthony Gordon sent off for a challenge on Virgil van Dijk.
The Reds doubled their tally shortly after half-time through Hugo Ekitike but despite their man advantage conceded twice to Bruno Guimaraes and substitute Will Osula.
And just as the game looked to be heading for a draw, substitute Ngumoha popped up in the box right at the death to inflict a devastating defeat on Eddie Howe’s side.
Howe kept faith with the same starting XI that drew 0-0 at Aston Villa on the opening day of the campaign, with Gordon continuing to lead the line in Alexander Isak’s absence.
Gordon missed a hat-trick of chances in the space of a few minutes before the Magpies fell behind against the run of play when Gravenberch’s low shot found its way past goalkeeper Nick Pope.
Things then turned from bad to worse for the home side when Gordon was shown a straight red card in first-half stoppage time following a VAR check on a foul on van Dijk.
Ekitike, the man Newcastle tried to sign three times, doubled the Reds’ lead just 30 seconds after half-time.
But Newcastle managed to show a response. Tino Livramento’s cross found its way to the back post and Guimaraes headed home in the 57th minute for 2-1.
The Magpies piled the pressure on and deservedly level through substitute Osula. But their efforts did not get them their just awards as Ngumoha scored in the last minute.
Our writer Jordan Cronin was at St James’ Park - and here are his Newcastle United player ratings:
Rooted to the spot as Ryan Gravenberch opened the scoring. Conceded three goals but tellingly didn’t have a save to make.
1. Nick Pope - 6
Rooted to the spot as Ryan Gravenberch opened the scoring. Conceded three goals but tellingly didn’t have a save to make. | Getty Images
Defended well in the main but was undone by some balls over the top that he misjudged. Put in one dangerous corner but couldn’t really threaten from his set pieces.
2. Kieran Trippier - 7
Defended well in the main but was undone by some balls over the top that he misjudged. Put in one dangerous corner but couldn’t really threaten from his set pieces. | Liverpool FC via Getty Images
Gifted Florian Wirtz a chance with a misplaced pass but looked settled with the ball after that. Stood up well to Hugo Ekitike.
3. Fabian Schar - 7.5
Gifted Florian Wirtz a chance with a misplaced pass but looked settled with the ball after that. Stood up well to Hugo Ekitike. | Getty Images
Typically dominant in the air but couldn’t score this time. Committed as ever. Put in some big challenges.
4. Dan Burn - 8
Typically dominant in the air but couldn’t score this time. Committed as ever. Put in some big challenges. | Getty Images