Plymouth started well with Peter Halmosi's shot drifting narrowly wide and City defender Clint Hill charging down a powerful effort from Lilian Nalis.
But Stoke took the lead against the run of play following a well-worked free-kick routine in the tenth minute.
The ball was worked to Liam Lawrence on the right and his cross was diverted into his own net by Marcel Seip.
Seip was caught flat-footed shortly afterwards as Fuller burst clear to shoot straight at Luke McCormick.
But Plymouth remained well in the hunt with David Norris and Akos Buzsaky testing Steve Simonsen.
The visitors came out bombing again at the start of the second half and got their reward in the 52nd minute.
Buzsaky swung over a left-wing corner and Seip timed his run at the far post to perfection to send a header high inside Simonsen's left-hand post.
Plymouth now really fancied their chances and Rory Fallon spurned a golden chance before they did take the lead in the 59th minute.
Halmosi headed Buzsaky's cross into the six-yard box and Fallon nipped between Gaby Zakuani and Simonson to score.
Stoke were on the rack, but they were back on terms in the 66th minute when MrCormick failed to deal with Rory Delap's long throw and Lawrence pounced at the far post.
And the winner arrived in the 73rd minute when Argyle were caught out by Stephen Wright's ball and Fuller raced clear to lob the on-rushing McCormick.