The 1987 winners, 17th in the Championship (second division) and in severe financial trouble, won with two goals by Michael Mifsud and one each from Elliott Ward and Dele Adebola against a strong Blackburn side.
Third division Huddersfield were also celebrating after a 2-1 home win over Birmingham City while Bolton Wanderers were another Premier League side to go out when they lost 1-0 at home to Sheffield United.
An 87th-minute equaliser by Rocky Baptiste earned Havant & Waterlooville a 1-1 draw at third division leaders Swansea City and ensured they will be the only minor league team in Monday's draw after Chasetown lost 3-1 at home to Cardiff City and Cambridge United went down 2-1 at Wolverhampton Wanderers, both outsiders having led.
Holders Chelsea beat Queens Park Rangers 1-0 with an own goal, while Tottenham Hotspur drew 2-2 with Reading and West Ham United were goalless against Manchester City in two all-Premier League ties.
Manchester United, Cup winners on a record 11 occasions, were away to Aston Villa later on Saturday, the teams' fourth meeting in the third round in seven years.