But their bid took a turn for the worse when Max Gradel was shown a straight red card on 33 minutes and Darryl Duffy put Rovers ahead shortly after half-time.
Grayson responded by bringing Jonathan Howson on for Shane Lowry and the move paid off in spectacular fashion as Howson equalised with a stunning 20-yard strike.
With Millwall now ahead against Swindon, Leeds had to go in search of a winner and it came in the 63rd minute when Jermaine Beckford pounced on a loose clearance to convert from six yards.
The goal sparked wild celebrations and there was a pitch invasion at the final whistle as Leeds' promotion was confirmed.
"When we went down to 10 men it was obviously going to be a difficult ask for us," said Grayson, who succeeded Gary McAllister on 23 December 2008 following a run of five successive defeats.
"But we kept believing we could do it and knew we'd get opportunities. Even when we went 1-0 down we still believed we'd do it and full credit to the players."
Grayson, a Leeds fan who began his playing career at Elland Road, added: "My professional pride means I wanted to get a promotion on to my CV.
"But it's even better that it's the club I supported and started my career with. We're going to cherish tonight.
"We're back in the next level of football where we deserve to be, I think. We'll take stock and get ready for next year."
Bookmark with:
What are these?