Golden Dawn MP Panagiotis Iliopoulos was ordered out by the acting speaker after he poked fun at the leader of the main leftist opposition Syriza party.
The acting speaker is also from Syriza.
As Iliopoulos was leaving the chamber with fellow Golden Dawn lawmakers, he called MPs applauding in approval "goats" and "pathetic" and amid the mayhem, someone shouted "Heil Hitler" three times.
State-run Athens News Agency said the shouts came from Golden Dawn parliament spokesperson Christos Pappas, who is considered a senior party ideologue. Golden Dawn denied this.
Earlier this week, Pappas admitted in parliament that he had written a tract several years ago praising Adolf Hitler as a "visionary".
Rise in social tensions
"Thank you for bringing up writings from our youth," Pappas told the chamber. "Nationalism is the youth of the world and its future belongs to nationalism."
Golden Dawn was elected to parliament in June, winning nearly 7% of the vote and 18 seats out of an overall 300.
The party has benefitted from a rise in social tensions as Greece experiences a fourth year of strict austerity policies.
Many Golden Dawn supporters have been implicated in violent assaults against immigrants and the party is known for its anti-Semitic and xenophobic discourse.
A number of its lawmakers have also been implicated in aggressive behaviour incidents - one of them recently attempting to punch the Athens mayor - and have had their parliamentary immunity lifted to enable their prosecution.