Dave Brammer pushed the ball through the legs of Momo Sylla and delivered a sublime cross for Bangoura to bury an eight-yard header in the 78th minute.
It was Stoke's second goal in three minutes after Leicester had missed numerous opportunities to sew up what could and probably should have been their second away victory of the campaign.
The Potters suffered a blow before kick-off when Luke Chadwick pulled up in the warm-up and was replaced in the starting line-up by Peter Sweeney.
The home side had the better of the early exchanges with Paul Gallagher's free-kick blocked before Mamady Sidibe blazed wide from eight yards.
But it was Leicester that took the lead after Patrick Kisnorbo touched a short free-kick to Joey Gudjonsson four minutes later. Gudjonsson, whose father Gudjon Thoradson used to manage Stoke, cleverly spun on the ball to re-adjust his position before lashing a fierce 25-yard shot beyond Steve Simonsen.
The goal knocked Stoke out of their stride before they equalised from the impressive Gallagher's twice-taken penalty in the 35th minute. Gallagher's pass had released Bangoura who was pulled down by Leicester keeper Rab Douglas as he prepared to shoot.
On-loan Gallagher, from Blackburn, had to beat Douglas twice from the spot, drilling his second shot low to the keeper's right after his first effort was ruled out for encroachment.
Leicester regained the initiative for the rest of the half and took the lead again courtesy of some awful Stoke defending three minutes after the break.
Carl Hoefkens and then Marlon Broomes both slipped to the ground, allowing Iain Hume to find Elvis Hammond who drilled an unstoppable cross-shot inside Simonsen's right-hand post.
Gudjonsson was no more than an inch from putting the visitors into the comfort zone in the 63rd minute when his 30-yard shot came off the underside of the bar and bounced to safety.
Hume and Ryan Smith then failed to convert opportunities before Stoke launched their late salvo, which was inspired by Brammer.
He also set up the equaliser when his cross from the right was met with a splendid header from Sidibe, sending the ball beyond Rab Douglas' reach.