The visitors had taken the lead on the half-hour mark when Jay Tabb slotted in after he pounced on a loose ball.
But Brentford suffered a setback when keeper Stuart Nelson was sent off.
And Ricketts struck after 87 minutes with a deflected shot just as it looked like Brentford were set to take a crucial advantage in the tie.
Brentford boss Martin Allen:
"I didn't think the referee gave us any protection for the amount of tackles from behind on our centre-forwards in the first half. "I thought there should have been numerous yellow cards. But there was no protection, and the referee got carried away with the atmosphere and gave us nothing.
"We passed the ball well and controlled the game. Up until the sending-off, we were in control."
Swansea: Gueret, Ricketts, Austin, Tate, Monk, Britton, Robinson, Tudur-Jones, MacDonald (Forbes 45), Knight (Trundle 68), Fallon (Akinfenwa 90).
Subs Not Used: Murphy, Williams.
Booked: Tate.
Goals: Ricketts 87.
Brentford: Nelson, O'Connor, Frampton, Turner, Tillen, Newman, Tabb (Brooker 84), Smith, Pratley, Rankin (Bankole 54), Gayle.
Subs Not Used: Hutchinson, Rhodes, Lewis.
Sent Off: Nelson (53).
Goals: Tabb 29.
Att: 19,060
Ref: K Stroud (Hampshire).