"Since retribution is narrowly focused on 'moral reprobation or outrage against criminal conduct', it fails to adequately reform offending behaviour and to repair harm experienced by victims of crime," he said in a speech prepared for delivery.
"In order for societies to succeed in the fight against crime and eliminate reoffending, victims of crime, offenders, families, and communities need to be active agents."
They had to be heard in determining the sanctions against offenders and how offenders could be assisted in righting their wrongs.
Ndebele was speaking at an African Correctional Services (ACSA) Ministerial Consultative Forum in Pretoria.
Ndelebe said ACSA had to work together to influence courts to use restorative justice and alternative sentencing.
He said Mozambique and Zimbabwe had given more attention to this than South Africa.
"Perhaps, this partly answers the question why South African prisons remain over-populated and far higher than Africa's most populous country Nigeria," Ndebele said.
Over the years, courts had emphasised that a prison sentence was a severe form of punishment which should be imposed only as a last resort and when no other form of punishment would do.
He said there had been concerted efforts to shift from more traditional methods of dealing with crime and offenders to a more restorative form of justice.
South Africa had the highest number of inmates in Africa.
"As at 28 March 2014, we had 157 170 people in custody.
"Of these, 113 458 were sentenced offenders and 43 712 were remand detainees," Ndebele said.
It cost taxpayers around R9 876.35 per month for each inmate.
However, since 2004, the inmate population had been reduced by 31 000 resulting in a saving of more than R1.4bn to the fiscus, he said.
"Working together, there is a lot we, as ACSA, can achieve for the noble calling of corrections in our continent," Ndebele said.
"We must be the architects of a sustainable correctional system in the continent."