The two groups clashed before police broke it up, NBC29 in Charlottesville reported. An AFP journalist at the scene witnessed demonstrators, some clad in militia uniforms, throwing punches and hurling bottles even before the official 12 pm EST (1600 GMT) rally start time.
But some of the white nationalists cited Trump's victory as validation for their beliefs, and Trump's critics pointed to the president's racially tinged rhetoric as exploiting the nation's festering racial tension.
At about 11:40 a.m., police appeared and ordered everyone to vacate the park.
Right-wing blogger Jason Kessler, the organizer of the rally that is expected to draw "alt-right" activists and white nationalists, filed a lawsuit in federal court Thursday against the city of Charlottesville. A spokeswoman for the University of Virginia Hospital said it treated 20 patients from the incident, and one had died.
The fast-paced march was made up nearly exclusively of men in their 20s and 30s, though there were some who looked to be in their mid-teens.
A pro-Democratic town linked to the university founded in 1819 by Thomas Jefferson, people have voiced strong condemnation for the planned arrival of members of the Loyal White Knights of the Ku Klux Klan, a small white supremacist group based in North Carolina. On Saturday morning, protesters and counterprotesters faced off and kicked and punched, hurled water bottles and deployed chemical sprays against one another. It left them floundering and seeking medical assistance.
There were also fights on Friday night, when hundreds of white nationalists marched through the University of Virginia campus carrying torches.
It is unclear if anyone was injured or arrested.
An estimated several hundred white nationalists descended on the University of Virginia's campus in Charlottesville on Friday night for an impromptu torchlit rally.
Witnesses said a vehicle ploughed into a crowd of people who were protesting against a rally held by white nationalists who oppose the removal of a statue of Confederate General Robert E Lee by civic officials in Charlottesville.
The future of the statue is to be decided by a judge in a ruling expected in the next few weeks.
Many city leaders and residents have expressed concern about the prospect of violence at Saturday's event.
Ku Klux Klan supporters staged a march in Virginia last month, but were easily outnumbered by counter-protesters.
"I am heartbroken that a life has been lost here", Charlottesville Mayor Mike Singer said on his Twitter feed.