The U’s ran out 4-0 winners, with Emma Thompson scoring her second hat-trick in just seven days.
Riva Casley also got her name on the scoresheet to put United into the draw for the next round.
Oxford started brightly but were given an early warning when Chelsea had the ball in the net from a free-kick in the 10th minute, only to be denied by the offside flag.
The visitors were always a threat on the counter attack with their pace, but the U’s soon took control of the game.
However, it wasn’t until just after the half-hour mark that Oxford took the lead as Thompson ghosted in at the back post to tap home Nicole Barrett’s cross from the right.
She doubled the advantage 10 minutes into the second half, reacting quickest after Chelsea keeper Isla Ball parried her initial effort.
The game was put to bed in the 67th minute when Casley added a third from a set piece.
Holly Manders delivered a free-kick from the left, and Casley met it on the volley to side-foot past Ball.
With 15 minutes remaining, Thompson completed her hat-trick in style, once again getting on the end of a Barrett cross.
Despite being well beaten, Chelsea put Oxford under some pressure for the first time during the last 10 minutes.
Goalkeeper Maddison Millington-Stanley replaced Beth Wookey for her United debut in the 88th minute, making a fine save with her first touch to deny a close-range Chelsea attempt.
After the game, manager Sam Rose said: “We wanted to back up the performance and result from last week’s game, and we did that, and we want to put a cup run together, so I was pleased to keep a clean sheet and pleased we’re going through.”
[Oxford United](http://www.thisisunited.com/?ref=au) Women are in cup action again this weekend, when they travel to Hashtag United in the Women’s FA Cup.
That will be followed by two huge league games, with United travelling to second-placed Bournemouth on Wednesday, October 29, before hosting top-of-the-table Watford on Sunday, November 2.
The U's are enjoying a good season, epitomised by a 10-0 win against Billericay Town in the WNL Southern Premier League last time out.
United were 4-0 up by half-time thanks to goals from Thompson, Manders, Maisy Barker, and captain Naomi Bedeau.
Their dominance continued in the second half as they reached double figures in the league for the first time since March 2019.