Portsmouth goalkeeper David James' poor clearance went straight to Delap, whose first-time ball put Kitson through on goal, but the striker shot wide.
It was a rare chance for Stoke during a first half in which right-back Andy Griffin was given a torrid time by Pompey's left-sided combination of Nadir Belhadj and Armand Traore.
Interview: Harry Redknapp
On-loan Arsenal winger Traore sent a 20-yard shot just wide and Fuller had a header saved by James, before Crouch put Harry Redknapp's side ahead.
The goal was Crouch's fifth of the season and his third in two matches.
Stoke hit back three minutes into the second half, when Delap's trademark throw-in caused havoc and former Pompey forward Fuller bundled the ball in from close range.
Defoe almost restored the lead immediately with a cracking effort that beat stand-in keeper Steve Simonsen and smashed against the woodwork.
But there was no reprieve for Stoke when Olofinjana invited Defoe to try his luck again.
Interview: Tony Pulis
This time the England striker let fly from just outside the penalty area and his effort beat Simonsen at the near post.
Simonsen, playing because first-choice stopper Thomas Sorensen has flu, produced good saves to keep out shots from Crouch and Traore before Crouch headed Glen Johnson's free-kick wide.
Stoke boss Tony Pulis sent on striker Mamady Sidibe and then recent signing Tom Soares for his Potters debut, but his team never looked like finding another equaliser.
City now have a two-week break because of the international fixtures and then face a crunch showdown against bottom club Tottenham.
Portsmouth manager Harry Redknpapp: "Peter Crouch can win the ball in the air but he also has great ability.
"He's more than just a big man. He's a terrific player and he showed that with great technique for the goal. It was an example for all young kids watching.
"We were full of energy again. Full of inventive play, with passing and movement. We had other great opportunities and hit the post."
Stoke manager Tony Pulis: "The difference was at both ends of the pitch. We had chances and didn't take them, but they had chances and did take them.
"Crouch and Defoe up front are good players and Crouch showed that with the goal he scored.
"In the first seven games of the season we've played teams who finished in the top five last season, so we knew it would be a tough start. We'll keep learning and keep believing."
Bookmark with:
What are these?