British police have freed six men without charge after they were arrested over an alleged plot to launch an attack during Pope Benedict XVI's state visit, Scotland Yard said.

"Six men who were arrested under the Terrorism Act 2000 on Friday, 17 September, were all released without charge late on Saturday night and early this morning (Sunday)," it said in a statement.

Counter-terrorism police raided a cleaning depot in London early Friday to arrest five men, aged between 26 and 50, "on suspicion of the commission, preparation or instigation of acts of terrorism", police had said.

A sixth man was detained later Friday.

The men were street cleaners employed in the Westminster district of London, where the pope spent much of Friday and Saturday, the local authority confirmed.

Reports said the men were all of North African origin but there was no confirmation from police.

The Vatican had played down the threat after the arrests of the men.

"We never attributed much importance to these arrests," Vatican spokesman Federico Lombardi told AFP earlier Saturday.

The 83-year-old Pope was "very calm", he said, and the four-day trip -- the first ever state visit by a Pope to Britain -- was "taking place smoothly".

Britain's Sunday Mirror tabloid quoted a police source as saying that the men were arrested after they were overheard joking in a staff canteen about blowing up the Pope with a rocket-propelled grenade.