Left-hander De Kock smashed a 127-ball 128 while skipper De Villiers hit a fiery 71-ball 108, making a century off just 66 balls, as South Africa posted their highest total against Sri Lanka of 339-5 in windy conditions at Mahinda Rajapaksa Stadium in Hambantota.
The Proteas then restricted the hosts to 257 in 44.3 overs for their first-ever one-day series win on the island by 2-1.
South Africa had won the first one-day in Colombo by 75 runs while Sri Lanka claimed the second by 87 at Pallekele.
Sri Lanka, who needed their record one-day chase, had promised a positive chase by smacking 57 in the sixth over with Kusal Perera (37) and Tillakaratne Dilshan (30) but a middle order slump saw them lose Dilshan, Kumara Sangakkara (36) and Mahela Jayawardene (two) in the space of just six runs.
When Lahiru Thirimanne went for 22 to make it 151-5, Angelo Mathews (58) and Ashan Priyanjan (30) put up some resistance during their 83-run stand but the target proved too much for the hosts.
For South Africa paceman Ryan McLaren took 3-37 while Jean-Paul Duminy and Morne Morkel grabbed two wickets apiece.
De Villiers, who lost a one-day series in Sri Lanka 4-1 last year, was delighted with the victory.
"I am very happy over the win," he said. "It's our first series win here so we will celebrate it tonight."
Sri Lanka skipper Mathews was left to rue missed chances.
"It was very depressing," said Mathews.
"We were pretty flat on the field, let go a few chances and didn't bowl well so we didn't deserve to win."
Earlier De Kock put on 118 for the first wicket with Hashim Amla (48) and another 116 for the second with De Villiers as South Africa improved on their previous highest total of 317-6 against Sri Lanka at Centurion in 2002.
De Kock, dropped at 38 off Sachithra Senanayake, reached his fifth one-day hundred with a sharp single off 108 balls.
The talented left-hander holed out at mid-off in the 41st over but not before bettering South Africa's highest individual one-day score against Sri Lanka made by Boeta Dippenaar (125 not out) at Adelaide in 2006.
In all De Kock hit 12 fours and three sixes but was overshadowed by his skipper who smashed 11 boundaries and four sixes.
De Villiers was in a punishing mood, dancing and moving at the pitch with consummate ease to stroke his way to the third fastest one-day century by a South African behind Mark Boucher's 44-ball effort and his own 58-ball century in the past.
De Villiers hit two boundaries and then took a single off Mathews to complete his 17th one-day hundred.
In between Jacques Kallis missed out, trapped leg-before for just four to end a disappointing series with five runs, leaving a big question mark on his survival until next year's World Cup.
Sri Lanka had packed the side with five spinners, bringing in Rangana Herath but De Villiers and De Kock negated that spin factor with sublime batting.
Ace home paceman Lasith Malinga went for 85 runs in his 10 wicket-less overs.
The teams now play a two-Test series starting at Galle from July 16.Teams:
AB de Villiers (captain), Hashim Amla, Quinton de Kock (wicketkeeper), Jacques Kallis, JP Duminy, David Miller, Ryan McLaren, Wayne Parnell, Dale Steyn, Morne Morkel, Imran Tahir.
Angelo Mathews (captain), TM Dilshan, Kusal Perera, Kumar Sangakkara (wicketkeeper), Mahela Jayawardene, Lahiru Thirimanne, Ashan Priyanjana, Rangana Herath, Sachithra Senanayake, Ajantha Mendis, Lasith Malinga