Guns and ammunition were found at Chris Hani Baragwanath Academic Hospital on Sunday.
The Department of Health said it was shocked by the discovery in a doctor's quarters which was vacant and being renovated.
Handcuffs, a police reflector vest and cap were also found.
The Gauteng's acting health MEC, Jacob Mamabolo, said the cache was found by security guards on Sunday morning while they were on routine patrol.
They found 47 bullets, three 9mm handguns, two pepper spray canisters, a police reflector vest and cap as well as handcuffs.
"It is deeply concerning that an unaccounted for assortment of ammunition was found in a space that is meant to look after patients' health care," said Mamabolo, shortly after announcing the return of elective surgeries at the facility in Soweto.
"We call on the police to get to the bottom of this and ensure that if these weapons were used to commit any crime those responsible are brought to book," he added.
The police did not furnish any more information. This will be added when it is received.
Earlier, the Ekurhuleni Metro Police Department said they found a stash of weapons in a room in Yeoville after a tip-off about a possible robbery.