[We are] currently ranked ninth in the world in terms of prison population, with approximately 160 000 inmates," he said in a speech prepared for delivery.
At least 30% of those detained were awaiting trial.
Ndebele was speaking at a meeting with senior leadership of the Police and Prisons Civil Rights Union and the Public Servants Association of South Africa in Pretoria.
The problem of over-crowding was a priority, he said.
"That our offender population has remained constant, whether you remove pass laws, group areas, or apartheid laws, should make us search more urgently for answers to the high prison population in South Africa," he said.
The correctional services department hosted a colloquium on overcrowding, alternative sentencing, and the White Paper on Remand Detention in South Africa in November last year.
Violence and unrest
"The recommendations of the colloquium are currently being considered for implementation," the minister said.
In January, three awaiting trial prisoners were killed and 65 others were injured in gang-related violence at the St Albans correctional facility in the Eastern Cape.
The Congress of the People blamed the unrest on overcrowding.
A few days earlier, another riot occurred at the Groenpunt prison in the Free State.
Four warders and around 50 inmates were injured when inmates torched their cells and parts of the prison facility.
The prisoners were reportedly complaining about some of the warders as well as the quality of food they were fed at the prison.
The Inkatha Freedom Party said then the incident reflected the problem of overcrowding in the country's prison facilities.
63 240 criminals serving time outside prison
Of this number, 48 323 were out on parole, and 14 917 were probationers (convicted offenders serving non-custodial sentences), Correctional Services Minister Sbu Ndebele said in a statement.
He called on the public to help re-integrate paroled and released offenders into society as law-abiding citizens.
"As corrections is a societal responsibility, and not just the responsibility of the department of correctional services (DCS), the community forms an integral part of the rehabilitation of offenders on parole to reintegrate them as law-abiding citizens."
Ndebele said the department played its role in attempting to prepare offenders for a life back in the community.
"Measures in place to ensure that released offenders comply with their parole conditions include house detention, monitoring, performance of community service, restriction to magisterial district, refraining from the use of alcohol or illegal drugs, refraining from making contact with a particular person(s), and participating in treatment, development and support programmes," he said.
Electronic monitoring devices and regular visits from parole officers were also used.
"One standard condition is that offenders on parole may not commit any crime, and, if they do, they will return to a correctional centre," said Ndebele.
Convicted offenders needed to undergo rehabilitation programmes while in jail to qualify for parole, Ndebele said. South Africa currently has around 152 550 people behind bars.
Every month about 23 000 inmates leave prisons while 25 000 new inmates are admitted, the department said.