Elliott slotted against the run of play after a layoff from fellow substitute Darwin Nunez for a scarcely-deserved victory in Paris.
PSG, who had won 18 of their last 20 competitive games, drawing the other two, were left to rue their missed chances from dozens of attempts.
Also on Wednesday night, Bayern Munich striker Harry Kane scored once in each half to guide his team to a 3-0 victory over 10-man Bayer Leverkusen in the all-German first-leg match.
The Bavarians, who earned their first victory in the last seven matches against Xabi Alonso's Leverkusen and will take a significant advantage into next week's return, established a ninth-minute lead through Kane's powerful header.
Keeper Matej Kovar gifted them a second in the 54th minute after he dropped the ball from a Joshua Kimmich cross to allow Jamal Musiala to tap home.
Things got worse for the visitors when Nordi Mukiele was sent off after a second booking and Kane sealed Bayern's comfortable win with a well-taken 75th-minute penalty.