Ed Joyce and Murray Goodwin combined to take the visitors into the lead, but when they fell in quick succession - to Gareth Batty and Stuart Meaker respectively - Sussex were in the mire.
But that was Surrey's last success for 32 overs as Yardy and Brown turned things around.
Meaker had Brown caught behind with the second new ball but Arafat arrived to see Yardy to three figures and Sussex to the draw.
Surrey cricket manager Chris Adams told BBC Surrey: "We were playing against a good team who were top of the league.
"I think they showed today why they are where they are; they dug in and were resilient and (Michael) Yardy's innings was exceptional.
"But over the three days we managed to play we dominated a good team and that's with seven players under the age of 24."
Bookmark with:
What are these?