The Potters went in at half-time 2-1 up after Xherdan Shaqiri scored twice - the second a sublime effort - either side of a Romelu Lukaku strike.
The hosts subsequently fought back as Lukaku notched his second equaliser of the game in the 64th minute and Gerard Deulofeu put them 3-2 up in the 71st.
But Stoke substitute Joselu then brought things level once again with 10 minutes to go, before Arnautovic made the most of the opportunity to wrap up victory in stoppage time, converting from the spot after referee Mark Clattenburg judged the Austrian had been fouled in the box by John Stones.
The game was lively from the off, with Arnautovic having one early shot blocked and then firing another narrowly wide, while in between, Ross Barkley saw a deflected strike go behind at the other end.
The Potters - looking to build on their 2-0 Boxing Day victory over Manchester United - were once again showcasing the dynamic style they have developed under manager Mark Hughes.
And it was very much in evidence in the 16th minute as they grabbed the lead thanks to a flowing move involving all three of their exciting front trio, the ball going from Bojan Krkic to Arnautovic and on to Shaqiri, who tucked it into the corner from close range for his first Stoke goal.
The advantage was soon extinguished as Lukaku - Everton's own man in form and, like Shaqiri, his club's record signing - made it 1-1 on 22 minutes.
It was another goal resulting from some pleasing football as Barkley squared to James McCarthy and his pass was collected by Lukaku, who turned and smashed past Jack Butland.
With Barkley to the fore and displaying some eye-catching tricks, Everton started to get on top, Brendan Galloway shooting wide and Seamus Coleman having appeals for a penalty following a challenge from Erik Pieters waved away by Clattenburg.
A promising situation for the hosts went to waste when Deulofeu went to ground and was judged by Clattenburg, correctly it seemed, to have dived - and then Stoke were in front once more as Shaqiri produced a moment of magic just before the interval.
The impressive Bojan was involved once again, lofting a ball to the edge of the area, and Shaqiri met it first time with a wonderful touch that sent it floating over the stranded Tim Howard and in.
Howard did well to avert the danger early in the second half as he halted Glen Johnson's attempt to put the ball into the danger zone, and Everton then pulled again level through Lukaku.
Substitute Tom Cleverley swept the ball towards the box and Lukaku took it down on his chest in fine style before firing past Butland, taking him to 11 goals in his last 10 games and 18 for the season.
Cleverley had to come off almost immediately after the goal, and his replacement Steven Naismith then shot narrowly wide before Everton went ahead for the first time, Lukaku feeding Barkley and his cross being converted from close range by Deulofeu.
Stoke were far from finished, though, and after Joselu took advantage of sloppy Everton defending from an Arnautovic cross by rifling the ball in, the latter wrapped up victory for the away side in injury time, striking confidently from 12 yards after going down under Stones' challenge.
Source : PA
Source: PA