The visitors, managed by former Magpies manager Shaun Derry, had gone into the game without an away goal in 700 minutes of football.
However, following a first half in which both sides traded chances - and both managers traded words on the touchline - the U's took the lead five minutes before the break.
Ade Azeez linked well with his strike partner Uche Ikpeazu, who cut on to his left foot and curled the ball into the top corner of the net beyond Ross Fitzsimons.
Just four minutes into the second half, Jevani Brown doubled his side's lead, much to the chagrin of the home crowd, when County failed to clear a corner and he rifled home via the inside of the post.
And the Notts fans could have been forgiven for declaring it game over when, 68 minutes in, Brown got his second of the game, receiving the ball from Ikpeazu and finding the top corner from 12 yards.
When Lewis Alessandra headed home Dan Jones's cross in the 74th minute, many would have deemed it a consolation and little more.
However, things got even better when, with two minutes of normal time remaining, Shaun Brisley's heavily deflected effort sent the ball looping up over David Forde and into the net.
And deep into injury time came the moment of delirium when Jonathan Forte fired an overhead kick beyond Forde to salvage a point against all odds for Kevin Nolan's side.
Share your thoughts about this match report on Pride of Nottingham by signing up to the website, visiting the forum and joining in the chat with hundreds of fellow Notts County fans.
Fancy showcasing your brand or business on Pride of Nottingham? Click here to find out more about sponsorship and advertising opportunities with us.