An excellent first-half hat-trick from Jo Kuffour looked as if it was only going to be enough to earn the Gills a point after visitors Northampton hit back from 3-1 down, that was until Danny Kedwell smashed home an injury-time penalty.
In an end-to-end enthralling encounter, which both managers probably hated because their defences were opened up far too frequently, the Gills had looked reasonably comfortable at the break but were left shaken as a lively Town put the defence under immense pressure.
Having threatened to shake things up after two straight defeats, boss Andy Hessenthaler did just that by making three changes to his starting XI with skipper Andy Frampton, Joe Martin and Danny Kedwell all being recalled. Those to drop onto the bench were Matt Fish, Curtis Weston and Luke Rooney.
In a reshuffled defence Matt Lawrence moved to right-back while Frampton played in central defence, although that lasted until the 38th minute when Garry Richards limped off with Fish going on as his replacement.
From the start Northampton looked dangerous on the break and the first item of note came in the eighth minute when Frampton received a yellow card for a halfway challenge on the lively Chris Arthur.
Moments later the returning Adebayo Akinfenwa ran onto a ball over the Gills defence but his weak effort was saved by Ross Flitney.
In the tenth minute Flitney could only parry a low shot from Michael Jacobs but although West Brom loanee Saido Berahino reacted quickly he saw his rebound strike the post.
A minute later and the Gills went in front. The 'new look' Frank Nouble laid the ball back for Lawrence to cross from the right, the ball was deflected and dropped perfectly for Jo Kuffour to swivel and fire home brilliantly.
Then after Kedwell had fired high onto the Town End, Michael Jacobs was allowed to escape from Charlie Lee, who'd moments earlier picked up a booking, but Flitney parried the midfielders effort.
The end-to-end flow continued with Kedwell rising to meet a Danny Jackman corner but his header was inches off target.
Jackman then dragged a shot wide before the Gills went two-up in the 31st minute. Nouble, set up by Kedwell crossed and when Chris Whelpdale nodded the ball down to Kuffour, the striker, with his back to goal, back-heeled the ball past Sam Walker from around five yards.
Four minutes later the Cobblers pulled one back when Akinfenwa fed Berahino who turned Frampton and stroked the ball wide of the advancing Flitney.
Derek Niven then struck a low shot at Flitney before Richards departed.
Then in the 44th minute the Gills restored their two-goal lead from a disputed penalty. Kuffour went down under a challenge from Kelvin Langmead and although referee Carl Berry waved aside penalty appeals, his assistant Ian Crouch flagged and after a lengthy consultation the referee pointed to the spot.
Both Akinfenwa and Chris Arthur were booked for protesting before Kuffour, presumably given the chance to complete his hat-trick, stepped up to confidently send Walker the wrong way from the spot.
There was still time for further appeals for a penalty to be turned down after Whelpdale seemed to be nudged as he looked to get onto another Nouble cross.
Two minutes after half-time Kuffour was denied a fourth goal by a tremendous defensive block and Lee fired well over before Gary Johnson's side pulled one back in the 54th minute. A free-kick was pumped forward and as Akinfenwa took up much of the defences attention, Langmead found the space to tuck away a low shot inside Flitney's left-hand post.
Whelpdale, put through by Kedwell, had a great chance for the Gills only to see his low shot strike the trailing boot of Walker and almost immediately Northampton were back on level terms following a free-kick conceded by Martin on the edge of the area.
Jacobs played a low ball into the area and Berahino showed some class as he fired home on the run, the 219 travelling Cobblers fans were ecstatic.
Jackman then went close to scoring against his former club but his searing drive just cleared the bar.
Town were getting forward at ease with Berahino, Arthur and former Gills trialist Lewis Young causing problems with the latter firing an effort at Flitney.
Akinfenwa was then put clear by Berahino but as Martin looked to get back, the big man fired over the bar then substitute Jake Robinson saw his fitst touch, a header, saved.
It was still anyone's game and seven minutes from time a great one-two between substitute Luke Rooney, on for Lee, and Kuffour saw the former through on goal but the youngster was denied Walker's block.
Jacobs twice sent long-range efforts goalwards, the first went just wide before Flitney managed to beat away the second.
Then in the 91st minute came the Gills winner.
A great ball from Kedwell fed Nouble who was unceremoniously dumped on the ground by Langmead and referee Berry had no hesitation in pointing to the spot for a second time. This time Kedwell stepped up to smash the penalty almost straight down the middle as Walker chose wrongly once again.
The final whistle went shortly after the restart and the relief from three sides of the ground was immense as a large cheer went up.
Gillingham have moved up a place to ninth in the League Two table.
Attendance: 4,704 (219 Cobblers fans)
Gillingham: Flitney, Lawrence, Richards (Fish 39), Frampton, Martin, Whelpdale (Weston 83), Lee (Rooney 72), Jackman, Nouble, Kedwell, Kuffour.
Unused subs: S.Payne, Gazzaniga
Northampton Town: Walker, Young, Tozer, Berahino, Akinfenwa, Johnson, Jacobs, Westwood, Kelvin, Arthur (Robinson 79, Niven (Turnbull 86).
Unused subs: McKoy, Hall, Savage