"Long live President Sidi Mohamed Ould Cheikh Abdallahi," protesters shouted before paramilitary gendarmes fired tear gas at them and beat some of them with batons. Gendarmes also broke up several smaller groups of 10 to 20 protesters.
The United Front for the Defence of Democracy (FUDD), a group of parties opposed to the coup, called Sunday's protest.
The demonstration came ahead of the African Union's (AU) Monday deadline for Sidi Mohamed Ould Cheikh Abdallahi to be freed and restored to power. Ould Cheikh Abdallahi, who was Mauritania's first democratically-elected president, has been held under house arrest since the coup in August.
The military leadership in Mauritania on Monday ignored an AU ultimatum for the junta to reinstate the ousted president, which is due to expire at midnight. "We don't care about these (AU) ultimatums, this doesn't scare anybody," Sidi Mohamed Ould Maham, a spokesperson for the majority of members of Parliament who support the new military leadership, said.
The president was ousted on August 6 just hours after he issued a decree firing the Mauritanian military's top brass, including Ould Abdel Aziz, who was the commander of the presidential guard.
The AU has already suspended Mauritania's membership over the coup and has warned of further sanctions.
But the army coup leader has dismissed the AU's demand as nonconstructive.
It is unclear what sanctions the AU would impose, especially as several of its members in the region have given their tacit approval to the coup. But overseas donors including the United States, France and the World Bank have acted quickly to cut aid.
Sunday's protest came in defiance of a ban on demonstrations imposed by the country's military rulers.
Local journalist Hamdi Ould Mohamed el Hacen told the BBC's Network Africa programme that police used tear gas and bludgeons to disperse the protesters, but no arrests were made.
According to France 24 News, after the demonstrators were dispersed, the organisers of the march announced they had changed tactics.
"We have decided to break up the protest movement in a dozen points where people will reassemble between 4 pm and 7 pm (1600 GMT and 1900 GMT). Leaders of the front (which organised the protests) will be at one of the meetings and will speak there," spokesman Ahmedou Ould Wedia said.
Mauritania's armed forces says it intervened to overthrow the elected government because it had allowed the social, economic and security conditions to deteriorate so badly that it threatened the country's foundations.
Abderrahim Ould Hadrami, Permanent Representative of Mauritania, delivered a statement last week to his country's Assembly's annual General Debate on behalf of the new Government in which he said the situation inside his country had become so dire that the armed forces had no other option.
"Economically and socially the situation has become disastrous," Mr. Ould Hadrami said, citing a rise in hunger, disease and misery after Sidi Mohamed Ould Cheikh Abdallah became president in March last year.
"And while the people suffer the former president, who only visited one city inside Mauritania, the city of Tainatan, in the aftermath of floods that hit the city, made 25 visits abroad, costing the Mauritanian taxpayer a lot of money, without achieving any tangible results in the field of cooperation between Mauritania and its development partners."
Hadrami said last month's coup was supported by two-thirds of parliamentary members, about 90 per cent of the country's mayors and two-thirds of the recognized political parties, as well as numerous civil society groups.
The United States and its allies support the ousted government leader.
The European Union has begun proceedings under its Cotonou Agreement with developing former colonies that could lead to further aid cuts. Prime Minister Laghdaf said last week Mauritania could turn to its Arab partners for help instead.
Even so, Abdel Aziz's military junta is isolated at a time Mauritanian forces are struggling to stem the spread of al-Qaeda, whose North African arm has launched a series of attacks in the past year, fuelling fears it is expanding south.
After the most recent attack, 11 army soldiers and their civilian guide were found with their heads cut off near the country's main iron ore mine in the remote northwestern Sahara.
The following is from Aljazeera.
Police tear gas Mauritania rally
Mauritanian police have beaten protesters who had taken to the streets of the capital against the ruling military.
Scores of residents loyal to Sidi Mohamed Ould Cheikh Abdallah, the president ousted in August, flocked to Nouakchott's main hospital on Sunday chanting: "Long live Cheikh Abdallahi."
Security forces using tear gas broke up the demonstration and small groups of protesters reassembled in other parts of the city.
Senior military personnel seized power at the beginning of August after Abdallahi had announced he was dismissing them.
All public demonstrations have since been outlawed, but Sunday's protest came a day before Abdallahi was due to be released from detention under an African Union ultimatum.
Mauritania risks sanctions if it fails to let him go.
One protester said: "I wish the soldiers were on the borders defending the country instead of assuming they have the right to ride roughshod over the fundamental right to demonstrate."
Other protesters chanted: "No to dictatorship, long live democracy."
Protesters came from the National Front for the Defence of Democracy (FNDD).
Abdallahi had been democratically elected and had received the support of the military, but he has remained under house arrest since the coup.
No date has been set for elections despite pledges by the army generals to act quickly to restore democracy.