As it happened: Sri Lanka v SA
South Africa, set a revised target of 176 from 29 overs, were 104-5 in 21 overs when a heavy downpour at the Premadasa stadium forced the match to be called off.
With 121-5 being the par score at that stage, the Proteas lost by 17 runs to suffer their second defeat in the series following the 180-run loss in the first match last Saturday.
Left-arm spinner Herath trapped opener Alviro Petersen leg-before for 24, and had Faf du Plessis caught behind to reduce the Proteas to 69-5 by the 14th over.
David Miller (22 not out) and Ryan McLaren (14 not out) put on 35 runs for the unbroken sixth wicket before rain ruined their hopes of pulling off an upset.
South Africa were dealt another blow as star batsman Hashim Amla injured his left knee while fielding and was rushed to hospital for precautionary scans.
Amla, who had been sidelined for the first match due to a strained back, was unable to bat and faces a race against time to be fit for the third ODI.
South African captain AB de Villiers vowed his team will fight back in the series.
"It's not over yet, there are three big games coming for us now," he said. "This was a much better game for us than the previous one. The bowling did okay, but the batting let us down.
"Hopefully we will get Amla ready for the next match. The injury is not too serious."
Earlier, fast bowler Morne Morkel claimed 3-34 as the hosts were restricted to 223-9 in 49.2 overs after they won the toss and elected to bat.
Sri Lanka's stand-in captain Dinesh Chandimal and opener Tillakaratne Dilshan top-scored with 43 apiece before heavy rain ended the innings with four deliveries still to be bowled.
South Africa regrouped after the heavy defeat in the first match to keep the free-stroking Sri Lankan batsmen in check on the slow wicket.
Sri Lanka lost opener Upul Tharanga in the fourth over before a second wicket stand of 59 between Dilshan and Kumar Sangakkara helped them recover to 66-1 in the 18th over.
Sangakkara, who smashed a career-best 169 on Saturday, hit six boundaries in his 37 before he mistimed a drive off left-arm spinner Aaron Phangiso and was caught in the covers.
South Africa pegged the hosts down to 123-4 in 30 overs as Mahela Jayawardene was bowled by Robin Peterson for 17 and Dilshan was caught behind off Morkel soon after.
Chandimal boosted the total with four boundaries in his 51-ball innings.
Chandimal, who was named man of the match, said he would be happy to hand the captaincy back to Angelo Mathews, who served out a two-match ban for slow over-rates during the tri-series in the West Indies.
"Angelo will be back for the next game, hopefully we can seal the series there," said Chandimal. "I knew the wicket was a bit slow, so it was better to bat first."
Two Sri Lankan umpires officiated after Australia's Rod Tucker, who was due to stand at one end, fell ill before the start.
The third ODI will take place on Friday at the Pallekele International Cricket Stadium at 11:00 SA time
Sri Lanka: Dinesh Chandimal (capt), Lahiru Thirimanne, Tillakaratne Dilshan, Upul Tharanga, Kumar Sangakkara, Mahela Jayawardene, Jehan Mubarak, Rangana Herath, Lasith Malinga, Shaminda Eranga, Thisara Perera.
South Africa: AB de Villiers (capt), Alviro Petersen, JP Duminy, Faf du Plessis, Hashim Amla, Ryan McLaren, David Miller, Morne Morkel, Chris Morris, Robin Peterson, Aaron Phangiso.Umpires: Rod Tucker (AUS) and Ruchira Palliyaguruge (SRI). TV umpire: Richard Kettleborough (ENG) Match referee: Andy Pycroft (ZIM)