United left back Evra was given a hostile Anfield reception after the Frenchman's involvement in a spat with Liverpool's Luis Suarez in October, which led to the Uruguayan forward's current eight-match ban for racial abuse.
Racism allegations were the backdrop to Chelsea's 1-0 win at Queens Park Rangers, where John Terry, in court next week after being charged over comments allegedly made to QPR's Anton Ferdinand in October, was kept away from his fellow defender as the clubs decided not to shake hands before kickoff.
Like Evra, Terry was constantly heckled by the home fans whose wrath switched to the referee on 62 minutes when he awarded a soft penalty to Chelsea after Clint Hill was adjudged to have pushed Daniel Sturridge. Juan Mata converted the spot kick.
Despite the lucky break Chelsea deserved to win, but even Liverpool's captain, Steven Gerrard, reckoned his side were fortunate to get past United after Daniel Agger's opener was cancelled out by Park Ji-Sung before Kuyt's late strike.
Asked if Liverpool had merited victory, Gerrard told ITV Sport: "Not on possession, probably not on general play; we've got to give Manchester United credit, they controlled large periods of the game today, but the most important thing was the result and we got there in the end."
United boss Alex Ferguson, whose side beat Premier league title rivals Manchester City in the third round but are now out of the FA Cup as well as the Champions League, agreed.
"I don't know how we lost it, it was a really good performance by us," he said.
Liverpool were woeful in a league defeat at struggling Bolton Wanderers last weekend but eliminated Premier League leaders City on Wednesday to reach the League Cup final, and have now beaten bitter rivals and league champions United.
Kenny Dalglish's side, still missing the verve offered by Suarez, took the lead on 21 minutes when Agger headed in a corner that struggling United goalkeeper David de Gea failed to collect after he was crowded out by the hosts.
United, without the injured Wayne Rooney, hit back through a crisp strike inside the near post from Park on 39 minutes and the visitors had other good chances, with Antonio Valencia hitting the post.
However, Liverpool had the last laugh to send Anfield into raptures and prompt more hopes in the city that Dalglish can bring back the glory days from his previous spell in charge.
Striker Andy Carroll, widely criticised for his form since a £35m move from Newcastle United last year, flicked on the ball to leave Evra out of position and Dutchman Kuyt slammed it in when De Gea could again have done better.
The goal avoided a replay at Old Trafford next month, which Suarez would have been eligible to play in.
The only minor shock on the day was inflicted on former Premier League side Hull City, who now play in the Championship (second division) but lost 1-0 at home to fourth division Crawley Town.
In two all-Premier League ties, Bolton Wanderers overcame Swansea City 2-1 at home and Norwich City prevailed 2-1 at West Bromwich Albion to also make round five.
Last season's runners-up, Stoke City, progressed with a simple 2-0 victory at Derby County.
Other results included Blackpool 1-1 Sheffield Wednesday, Leicester City 2-0 Swindon Town, Millwall 1-1 Southampton, Sheffield United 0-4 Birmingham City, and Stevenage 1-0 Notts County.
Tottenham Hotspur are also through after beating second-tier Watford 1-0 on Friday, also when Everton overcame Fulham 2-1.
Newcastle visit Brighton and Hove Albion in Saturday's late game, while the two remaining fourth-round fixtures take place on Sunday with Sunderland entertaining north-east rivals Middlesbrough and Arsenal hosting Aston Villa.
