A night of frustration for PSG forward Neymar looked set to ensure more continental disappointment for the French champions as unfancied Atalanta led through Mario Pasalic.
But the Brazil forward then laid on a 90th-minute equaliser for Marquinhos and substitute Kylian Mbappe teed up former Stoke striker Eric Maxim Choupo-Moting to slide in the winner.
On the 50th anniversary of the club's founding, it looked like being a familiar tale of failure for PSG on European club football's biggest stage.
Neymar, the world's most expensive player, was all too often the only bright spark for Thomas Tuchel's side, but was also guilty of two glaring misses.
But the introduction of Mbappe - the world's second-most expensive player - proved crucial as his pace and quality finally undid the stubborn Atalanta rearguard to provide an astonishing finale to the first one-legged quarter-final in the re-shaped competition.
PSG will now face either RB Leipzig or Atletico Madrid in the first semi-final next Tuesday night.
What do you think?