The decision sparked outrage among the relatives and supporters of the former revolutionary leader, whose assassination in 1987 brought current president, Blaise Compaore, to power.
The much-delayed decision enraged a crowd of 100 people who had been waiting outside the court in the capital Ouagadougou, who started chanting "down with the Burkinabe judiciary" and "when will the Burkinabe people know the truth?".
Sankara and 14 of his aides were killed during the 1987 coup. Their bodies were hastily buried in the capital's Dagnoen cemetery before their families had a chance to identify them.
In October 2010, Sankara's relatives filed a legal case to demand the body be exhumed for forensic analysis.
The United Nations Commission on Human Rights had also urged the authorities of Burkina Faso to investigate the former president's murder in 2006.
"We are not happy," the family's lawyer, Benewende Sankara, who is no relation, told reporters after the court's decision that it did not have jurisdiction over the case.
He said they would appeal.
"The Burkinabe judiciary is incapable of giving citizens their rights," singer and anti-regime activist Serge Bambara - better known as "Smokey" - said outside the courtroom.
Compaore helped Sankara seize power in 1983 and has always denied persistent accusations he was involved in the murder of his former comrade-in-arms.
Sankara changed the west African nation's colonial name of Upper Volta to Burkina Faso - which means "the land of upright men" - a year after seizing power in 1983.
Despite his short rule, the young revolutionary leader - who was only 37 when he died - left a lasting mark on African post-colonial politics and was deeply mourned.
Reputedly incorruptible, the charismatic red-bereted captain drew Che Guevara-like adulation on the continent and remains a source of pride for many Burkinabe.