But the injury-hit pace bowler finally put those painful memories to rest with a devastating spell that had England reeling on the opening day of the second Test at Lord's on Thursday.
Harris took two wickets, including the prized scalp of Kevin Pietersen, in front of a capacity crowd featuring Queen Elizabeth II, in the process giving Australia renewed hope of regaining the Ashes.
It was a cathartic moment for Harris, who has battled back from a series of ailments that threatened to wreck his career.
Harris emerged as a new ball threat for Australia, after belatedly marking his Test debut aged 30, with six wickets against New Zealand in 2010.
But since then the 33-year-old has been plagued by a series of injuries including a chronic knee problem that will be with him until he retires.
The most frustrating of those injuries came during the 2010-11 Ashes series just as Sydney-born Harris was starting to fulfil his potential.
When he took nine wickets at Perth to help level the series, it seemed Australia had unearthed a real gem.
But Harris suffered a stress fracture to his left ankle in the fourth Test in Melbourne, England went on to win the series 3-1 and the pace bowler has never quite been able to shake off the injury-prone tag.
A shoulder complaint that needed surgery in 2012 left Harris sidelined for several months, while an Achilles injury earlier this year cut short his spell in the Indian Premier League.
Despite such a suspect fitness record, Harris had been in good form for Queensland of late and he showcased his ability to thrive in English conditions with five wickets for Australia 'A' against Gloucestershire last month.
With victory vital at Lord's after Australia's 14-run defeat in the first Test at Trent Bridge, Harris was recalled in place of Mitchell Starc.
It proved an inspired move.
Wearing the baggy green cap for the first time since the final Test against the West Indies in the Caribbean last year, Harris charged in like a man on a mission.
Asked before the series if he had unfinished business with England, Harris said: "That's a very good question and yes it is - we lost pretty badly in that (2010/11) series.
"Obviously I went down halfway through that Test so it would be nice to come back, play well, take some wickets and bring the Ashes home."
And Harris, who had taken 47 wickets at 23.63 from his 12 previous Tests, wasn't going to pass up this chance for revenge.
The Queen, introduced to both teams prior to the start of play, barely had time to get comfortable in her seat before Harris had ripped the heart out of England's top order.
After captain Alastair Cook was dismissed by Shane Watson, Harris struck for the first time, pinning Joe Root lbw.
Even the controversial Decision Review System, which came to England's rescue at Trent Bridge, couldn't deny Harris as third umpire Tony Hill ruled the ball hit the pad before the batsman made contact.
Harris then induced an edge from Pietersen to wicketkeeper Brad Haddin with a perfectly pitched delivery that held its line up the slope.
Harris took a breather with a figures of 5-2-9-2, but returned for another miserly four-over spell that kept England firmly on the back foot at lunch.