The visitors survived a major scare after just 47 seconds as Windass intercepted Alex Palmer's weak clearing header, although the goalkeeper recovered to keep out Sam Smith's subsequent shot.
Ipswich enjoyed numerous prolonged spells on the ball, although captain George Hirst and Chuba Akpom were unable to trouble Arthur Okonkwo with headers from close range as they lacked a threat in front of goal.
Having already been denied by Palmer with a shot from distance, Windass provided the vital touch on 34 minutes to put the hosts ahead.
Ryan Longman kept the ball alive from George Thomason's over-hit cross and duly whipped in a delivery towards Windass, who coolly guided the ball into the net from six yards out.
Having struck twice in quick succession before the break in their dramatic third-round win over Nottingham Forest, Wrexham very nearly achieved the same feat against the Tractor Boys - only for Palmer to produce a spectacular one-handed save to keep out Ollie Rathbone's curling strike.
Kieran McKenna's side improved after the restart and, after Wrexham had survived a goalmouth scramble, the visitors saw Elkan Baggott guide an effort wide at the back post from Ben Johnson's cross.
Palmer had to be alert to keep out another Rathbone effort as the Red Dragons regained a foothold in the contest - with Ipswich struggling to carve out any clear openings.
Wrexham held on for a first clean sheet in nine home matches across all competitions, moving into the fifth round for the first time since reaching the quarter-finals of the competition during the 1996-97 season.