Bojan Krkic's driving shot gave Stoke an early lead and Spurs fell two behind shortly after the half-hour mark when Jonathan Walters tapped home from close range.
Tottenham scored twice late on to beat Aston Villa last week and the home side were give hope when Nacer Chadli smashed home a volley with 13 minutes to play.
Stoke were dogged and determined, however, and after Kyle Naughton was sent off with six minutes to play, the visitors held on for their first victory in four matches.
Tottenham have already lost at home to Newcastle, West Brom and Liverpool this season following Europa League fixtures but Pochettino made seven changes to the side that beat Asteras Tripolis in midweek while also handing Harry Kane his first league start of the campaign
Kane almost made an instant impact as a neat ball from Christian Eriksen set Chadli away and the Belgian drove forward before picking out Kane at the back post, but the striker's angled header was saved.
Instead, it was Stoke who opened the scoring in the sixth minute as Townsend gave the ball away cheaply in midfield and Bojan dribbled 20 yards unchallenged before shooting low past Hugo Lloris.
The goal turned a nervous atmosphere at White Hart Lane sour and there was a smattering of boos in the 18th minute when Younes Kaboul's chipped ball long ran out unchallenged for a goal kick.
Ryan Mason struck a low shot narrowly wide of Asmir Begovic's near post soon after but Tottenham struggled to find any rhythm or fluency in their opponents' half.
The home side were also wide open on the counter and in the 33rd minute Stoke doubled their lead as Mame Diouf broke free down the right before squaring for Walters, who applied a simple finish from close range.
Spurs continued to dominate possession but hardly a chance was created before half-time, which was greeted with a chorus of jeers by the home support.
The interval saw Townsend and Eriksen replaced with Erik Lamela and Mousa Dembele but there was only a slight improvement after the restart.
Lamela's free-kick shortly before the hour was met by the head of Kane but the ball flashed narrowly past the far post as Stoke continued to sit deep and frustrate the hosts.
Steven N'Zonzi and Steve Sidwell worked tirelessly in midfield and Tottenham's disjointed display turned to frustration with a number of desperate challenges flying in throughout the latter stages.
Lamela almost gave Spurs a lifeline with 15 minutes to play, bending a left-footed free-kick towards the top corner, but Begovic did well to palm the ball away.
Two minutes later, Tottenham finally gave their fans something to cheer about as Danny Rose chipped a cross to the back post and Chadli hammered the ball into the net with an emphatic volley from an astute angle.
Buoyed by their goal, Spurs poured forward and Mason almost pulled his side level with a dipping long-range shot that flew over.
Mason landed his side in trouble in the 84th minute, however, when he was caught on the ball and as Victor Moses ran through, Naughton brought the winger down and was duly shown a red card.
Kane had snatched a late winner with a free-kick against Aston Villa last weekend but when a similar chance presented itself in injury time, the young striker hit the set-piece straight into the wall as Tottenham sent the White Hart Lane faithful home feeling deflated again.
Source : PA
Source: PA