Security guards quickly formed a cordon around Sarkozy as he went into the presidential palace to join other leaders from French-speaking Africa at the funeral for Bongo, who led his oil-rich nation for 41 years.
Several hundred people were allowed into the courtyard to watch leaders arriving and some applauded Sarkozy but boos soon took over. Dozens of people hurled insults such as "We don't want you, leave" at the French president.
One man in the crowd told AFP: "You French, you come here to eat Gabon. All the presidents who have come to this palace have left again with their pockets full and then you criticise us."
The ceremony for Bongo was delayed because mourners were late arriving. About 15 heads of state, mainly from Africa, were at the funeral.
Foreign Minister Bernard Kouchner and former president Jacques Chirac were also present for the funeral, highlighting a major French delegation that reflects the importance that France still gives to its former possessions in Africa.
African Union commission president Jean Ping, a Gabonese, and presidents Denis Sassou Nguesso of Congo-Brazzaville, Francois Bozize of Central African Republic, Paul Biya of Cameroon and Faure Gnassingbe of Togo have arrived in Libreville in recent days.
The leaders from Senegal, Chad, Democratic Republic of Congo, Benin, Burundi, Mali, Equatorial Guinea, Burkina Faso and Sao Tome were also expected.
Bongo died last week aged 73 at a private clinic in Spain.
Gabonese leaders were to speak at an ecumenical service.
It has not been confirmed who will speak on behalf of Bongo's family, but it was considered likely his 50-year-old son Ali Ben Bongo, Gabon's defence minister and a favourite to succeed him, will address the funeral.
Following the funeral the coffin was to be moved outside for a two-hour military procession.
The coffin will then be taken to the city of Franceville, capital of Bongo's native southeastern Haut-Ogouue region, where he will be buried on Thursday.
Bongo came to power in 1967 with French support and ruled over a state that grew rich on abundant oil, while most of the 1.5 million population remained poor.
In his last months, Bongo's relations with Paris were marred by a French probe into his luxury residences in France and a court order to freeze his bank accounts, amid allegations of embezzlement.
His demise raised fears of a power vacuum and initial reports of his death prompted Libreville residents to stock up on fuel, but the government has moved fast to avoid a power vacuum during 30 days of official mourning.
Interim President Rose Francine Rogombe has asked the government to come up with a schedule for presidential elections, which according to the constitution must take place within 45 days of the appointment of an interim president.
Prime Minister Jean Eyeghe Ndong is expected to officially resign on Thursday following Bongo's burial, government sources said, following a constitution court ruling that the appointment of an interim president removed his mandate.