Speaking at a Black Management Forum (BMF) conference - aimed at enhancing black management leadership - Mbeki used a double-storey house as a metaphor for the South African economy.
He said he had been speaking to a senior manager earlier who had said South Africa was like that double-storey house - "without a connecting staircase", he said to laughter from the crowd attending the two-day conference.
Diverging from a prepared text, he said on the upper level there was a modern economy which "for all of its problems it is diversifying".
Then there was another economy were the poor were trapped, where the people were illiterate, without skills and had no access to clean water.
In a clear dig at the mainly white opposition, he said the common argument that 6% growth would solve the unemployment problem in South Africa was "not correct".
Because of the lack of skills and because people in rural areas could not read or write, even with high economic growth they would not "enter the economy".
In a jovial mood and poking fun at himself, Mbeki said that he could hear some people saying "there he goes again", when he urged black managers to put their minds to the problem of black empowerment of the lower storey of the house.
He said available literature on how to break out of poverty may be fine in urban areas but often meant nothing at all in rural environments.
Talking about his experience as chancellor of a tertiary training institution, Mbeki said he became quite depressed by the number of distinctions in such disciplines as biblical studies while one in a hundred was a distinction for a black graduate in the sciences.
Urging South Africans - and black management in particular - to focus on the training of people outside of the humanities, he said South Africa had, nevertheless, made progress during the last nine years of democracy.
But he admitted that there was room "to make a great deal more progress".
Returning to his prepared text, he noted that according to a Human Sciences Research Council survey of 2772 graduates, there were 34% black graduates, 64.6% white graduates and the remainder unknown over an eight-year period in the natural sciences.
In engineering, 89.5% of graduates were white and only 4.3% African, 3.8% Indian and 1.9% coloured.
Progress had been made in the management field, however, with blacks representing 20% of senior management although whites still represented 80%.
At middle management level it was half and half, while at the lower end of the social strata blacks represented 83% of non-permanent employees while whites accounted for just 17%.
These figures for blacks were clearly still low but would have been far lower 10 years ago, he noted.
Mbeki urged the BMF to become involved in advising the public sector in advancing black empowerment and to involve itself in Africa to address the absence of management resources.