Rouget, 43, who works under the alias Sanders, was arrested on Saturday at the Johannesburg International Airport after stepping off a flight from Harare, Zimbabwe, authorities said.
"We might see more arrests if we can link him to others, but for now our investigations continue," said Sipho Ngwema, spokesperson for the Scorpions special investigation unit.
Rouget, a Frenchman who is a naturalised South African, appeared in court on Monday on charges he violated the Foreign Assistance Military Act by trying to recruit South Africans to fight in the bloody civil war in West Africa, Ngwema said.
He will be the first person charged under a law that bans South African fighters from freelancing in foreign conflicts.
The 1998 law was enacted to put an end to the activities of many former apartheid soldiers who were selling their services across the continent with no regard for the South African government's position in those conflicts.
The law bans the involvement or recruitment of South Africans in foreign conflicts without government permission.
Rouget appeared briefly in a Cape Town court on Monday, and his case was adjourned to August 1, when he will appear in a Pretoria court.
Rouget, who could face a fine, jail time or both, was remanded.
Born in Lyon, France, Rouget had since become a South African citizen, Ngwema said.
For the last two years, he lived in the Ivory Coast, where he was involved in military operations with other South Africans, Ngwema said.
Evidence a problem
That nation, once West Africa's bastion of stability, was plunged into violence after a 1999 coup. The government estimates that more than 3 000 people have been killed in fighting and about one million forced from their homes.
The government was working to trace Rouget's contacts, Ngwema said.
"We might see more arrests if we can link him to others, but for now our investigations continue," he said.
Richard Cornwell, a researcher at the Institute for Security Studies, said Rouget's case would difficult for the state to prove.
"Evidence is always the problem in cases like this. Usually it is hinged on hearsay. Rarely a contract can be produced to prove that someone had been operating as a mercenary," he said.
"People in this business are just too careful," Cornwell said.
According to submissions to South Africa's Truth and Reconciliation Commission by foreign security agencies, Rouget is also linked to a French cell alleged to have killed Dulcie September, the African National Congress' party representative in France, in the mid 1980s, Cornwell said.
The submissions also mention Rouget's name in connection with the attempted murder of the ANC's Godfrey Motsepe in Brussels, Belgium.
Cornwell said Rouget was allegedly part of a group doing reconnaissance missions of the ANC on behalf of the apartheid regime.
According to documents gathered from French authorities, Rouget was a warrant officer in the French Army, which he left in 1984.
A year later, he took on the alias of Sanders and joined the mercenary forces of Bob Denard's Comorian Presidential Guard. Denard had overthrown the Comoros archipelago's second president in 1978.
Rouget later moved to South Africa, where he worked and travelled extensively as representative for two French companies linked to mercenary activity, Cornwell said, citing the documents.