Villa boss Martin O'Neill won this competition in 1997 and 2000 when he was in charge of the Foxes but his old club ensured he will have to wait to complete his hat-trick.
O'Neill decided to shuffle his squad, handing defender Curtis Davies his debut and bringing Taylor, Marlon Harewood, Stiliyan Petrov, Isaiah Osbourne and Maloney into the starting line-up.
But, apart from a piledriver from Maloney that forced Leicester keeper Martin Fulop into a fine save, the home team rarely threatened.
Leicester's best opportunity fell to Hume in a first half that saw few real chances.
Villa did improve after the restart and Fulop did well to keep out Harewood's header from an inswinging Gareth Barry free-kick.
But it was Leicester who took the lead when Fryatt latched on to a James Chambers' pass and slotted past Taylor.
O'Neill made three substitutions to try to shake up his team in response and Maloney almost levelled when Fulop tipped his dipping free-kick around the post.
Gabriel Agbonlahor hooked the ball over the bar from close range in injury time as Villa continued to press but Leicester hung on to reach the fourth round.
Leicester boss Gary Megson: "Everybody put their bodies on the line and worked their socks off and deserved a result that probably no-one expected.
"We could have done with more quality and patience in the last third but this club has been in a relegation battle for three years.
"We have the basis of a fantastic football club. We have the support, a great ground and we just need to get things right on the pitch."
E-mail this to a friend 
Printable version 
Interview: Leicester City boss Gary Megson
Bookmark with:
What are these?