Manager Dave McNabb named an unchanged starting XI from the side that won 6-0 in the FA Cup last week against South Liverpool. Against a higher-level league opponent, however, the home side found chances harder to come by.
Bury held most of the possession in the opening stages and earned a slew of early corners but found a tall, strong and well-organised Newcastle Town difficult to break down.
The best opportunity Bury had came from a Kai Evans free-kick from a promising position but the Bury number 10 could only hit the wall and the Castle were able to clear their lines.
While the Shakers struggled to get the upper hand from set-pieces they found more success from open play. Scott Duxbury turned defence into attack with numerous mazy runs that put the away team on their heels.
What the hosts lacked, however, was the finishing touch. Connor Pye, DJ Pedro and Aaron Chalmers all managed to put headers on target but none with enough power to trouble Newcastle goalkeeper Adam Whitehouse.
At the other end, Newcastle Town were kept at arm's length. They rarely gave Bury stopper Mitch Allen much to do besides a Sean Lawton free-kick that, like Evans' earlier effort, went no further than the wall.
Just before half time the Shakers had a penalty appeal. A Rustam Stepan shot from the right-hand side struck the hand of Luca Whitney before bouncing out of play. The appeals were waved away and the subsequent corner was defended well.
After the break Bury came out of the blocks quickly and with renewed attacking intensity.
A Stepans cross found a heavily-marked Tyler James, he was just able to lay the ball off to Cameron Fogerty but the Bury midfielder blazed his effort over the bar from close range.
Then Pedro managed to carve out a chance of his own, dispossessing Daniel Matthews in his own half he charged one-on-one with the keeper only to see Whitehouse save his effort and Tyler James’ rebound as well.
The missed chances almost came back to haunt the Shakers as a Newcastle counterattack yielded their best chances of the match. Harlan Rodgers-Morris the Castle winger burst into the box exchanging a quick passing move with Joel Jones only for Aaron Chalmers to put in a vital last second challenge. Noel Brindle fired the loose ball just wide of Mitch Allen’s post.
The home side continued to push hard for a goal. Bury captain Chalmers had a great chance putting a header onto goal from a Pye cross but it was palmed away well by Whitehouse.
Another Stepans cross fell perfectly to Evans but his overpowered shot just cleared the bar. Evans would later try a lobbed effort that beat Whitehouse but landed beyond the far post and wide.
As the match entered the closing minutes substitute Gabriel Johnson found the back of the net with a close-range finish, but the goal was disallowed for a foul in the build up.
With less than three minutes of time remaining the dam finally burst. With much of the Bury XI still up in attack from an earlier set-piece, a deep Stepans cross from the left reached Pedro. His header back across goal found Walker who, from a yard out, couldn’t miss.