Updated 9.40pm

Thousands of people took to the streets of Boston today to protest hate speech a week after a woman was killed at a Virginia white-supremacist demonstration, and their shouts drowned out the "Free Speech" rally that sparked their march.

Organizers of the rally had invited several far-right speakers who were confined to a small pen that police set up in the historic Boston Common park to keep the two sides separate. The city largely avoided a repeat of last weekend's violence in Charlottesville, Virginia, where there were bloody street battles and one woman was killed.

The rally never numbered more than a few dozen people, and its speakers could not be heard over the shouts of those protesting it and due to the wide security cordon between the two sides.

Counter-protesters gesture during clashes with Boston police. Photo: Reuters

Protesters surrounded people leaving the rally, shouting "shame" at them and occasionally throwing plastic water bottles. Police escorted several rally participants through the crowds, sometimes struggling against protesters who tried to stop them.

A Reuters photographer saw multiple marchers arrested.

WATCH: Horror show as car ploughs into crowd in Virginia

Two male rally participants wearing Trump's red "Make America Great Again" campaign hats attempted to enter penned-in protest area. They were swarmed by black-clad protesters, some with their faces covered, as the crowd screamed "go home" and "no hate" at them.

"They heard our message loud and clear: Boston will not tolerate hate," said Owen Toney, a 58-year-old community activist who attended the anti-racism protest. "I think they'll think again about coming here."

Some 500 police officers had placed barricades, including large white dump trucks, to prevent vehicles from entering the park, the nation's oldest.

Last weekend's clashes in Charlottesville, Virginia, where one woman was killed in a car rampage after bloody street battles, ratcheted up racial tensions already inflamed by white supremacist groups marching more openly in rallies across the United States.

Photo: Reuters

White nationalists had converged in the Southern university city to defend a statue of Robert E. Lee, who led the pro-slavery Confederacy's army during the Civil War, which ended in 1865.

A growing number of US political leaders have called for the removal of statues honoring the Confederacy, with civil rights activists charging that they promote racism. Advocates of the statues contend they are a reminder of their heritage.

Duke University removed a statue of Lee from the entrance of a chapel on its Durham, North Carolina campus, officials said on Saturday.

Organizers of Saturday's rally in Boston have denounced the white supremacist message and violence of Charlottesville and said their event would be peaceful.

READ: Trump under fire for his response to white supremacist violence

"The point of this is to have political speech from across the spectrum, conservative, libertarian, centrist," said Chris Hood, an 18-year-old Boston resident who stood among a crowd of a few dozen people who planned to join the Free Speech rally. "This is not about Nazis. If there were Nazis here, I'd be protesting against them."

Last weekend's violence sparked the biggest domestic crisis yet for US President Donald Trump, who provoked ire across the political spectrum for not immediately condemning white nationalists and for praising "very fine people" on both sides of the fight.

Two male rally participants wearing Trump's red "Make America Great Again" campaign hats attempted to enter the protest pen that police had set up to keep the two sides separated. They were swarmed by black-clad protesters, some with their faces covered, as the crowd screamed "go home" and "no hate" at them.

Beyond the Boston rally and march, protests are also expected on Saturday in Texas, with the Houston chapter of Black Lives Matter holding a rally to remove a "Spirit of the Confederacy" monument from a park and civil rights activists in Dallas planning a rally against white supremacy.

Boston authorities had roadblocks in place to avert car attacks like the deadly one carried out in Charlottesville by a man said to have neo-Nazi sympathies against counter-protesters and a similar spate of attacks by Islamist extremists in Europe, most recently Barcelona.

Protesters reject plea

Boston Mayor Marty Walsh had asked protesters to avoid Boston Common, saying their presence would draw more attention to the far-right activists. He joined the crowd of thousands assembling in Boston's historically black Roxbury neighborhood early on Saturday.

"These signs and the message so far this morning is all about love and peace," Walsh told reporters. "That's a good message."

Monica Cannon, an organizer of the "Fight White Supremacy" march, said it was a necessary move.

"Ignoring a problem has never solved it," Cannon said in a phone interview. "We cannot continue to ignore racism."

The Free Speech rally's scheduled speakers include Kyle Chapman, a California activist who was arrested at a Berkeley rally earlier this year that turned violent, and Joe Biggs, formerly of the right-wing conspiracy site Infowars.

Antonio Vargas, a 20-year-old student at Gordon College, joined the protest march.

"I believe in equality," Vargas said. "I believe race shouldn't define the pattern of your life or the result of your life.

"There also is a time to stand up and not be silent."