The Giants looked to be sweeping to a big win after tries from Paul Reilly, Steven Wild (2) and Brad Drew put them into a 22-0 lead after 12 minutes.
Castleford showed real fortitude as tries from Luke Dyer, Willie Manu (2) and Deon Bird brought the scores level.
But the Giants' James Evans and Steve Snitch scored in the last 10 minutes.
Huddersfield coach Jon Sharp:
"We over-played at times and gave Cas the opportunity to get back into the game. � "It's difficult to turn the tap back on but fortunately we managed it. We weathered the storm, came back and finished strongly. �
"Castleford is going to be a tough place to come and I'm really pleased we came away with the two points."
Castleford: Platt, Dyer, Shenton, Viane, Bird, McGoldrick, Kain, Ward, Henderson, Nutley, Whitaker, Huby, Huby, Fletcher, Manu, Roarty, Handforth.
Replacements: Huby for Ward (51), Handforth for Henderson (17), Fletcher for Nutley (50), Edwards for Whitaker (74), Manu for Huby (12), Handforth for Huby (74), Roarty for Fletcher (17), Ward for Manu (66), Nutley for Roarty (56), Henderson for Handforth (55).
Huddersfield: Reilly, Gardner, Evans, Torrens, Donlan, Thorman, Paul, Crabtree, Crabtree, Drew, Jackson, Nero, Jones, Wild, Snitch, March, McDonald, Raleigh.
Replacements: Snitch for Torrens (24), Raleigh for Crabtree (24), McDonald for Crabtree (54), March for Drew (18), McDonald for Jackson (24), Jackson for Snitch (54), Drew for March (52), Crabtree for McDonald (32), Snitch for Raleigh (66).
Att: 6,881
Ref: S Ganson (St Helens).