Hong Kong-based John Short arrived in Beijing on a commercial flight from Pyongyang just hours after the North's official KCNA news agency announced he was being released.
"I'm really, really tired," a tearful Short said, breaking down in front of reporters at Beijing airport.
He had been arrested week ago after leaving "Bible tracts" in a Buddhist temple in Pyongyang during a tour.
His release came as North Korea fired two short-range missiles into the sea in a sign of the tensions on the Korean peninsula fuelled by ongoing South Korea-US joint military drills.
KCNA said the "generous" decision to release and expel Short had been taken in light of his advanced age and his display of contrition.
The news agency released a copy of his confession, along with photos showing him affixing his thumb in red ink to the document which he also signed and read out.
"I realise that my actions are an indelible hostile act against the independent right and laws of the [North]," the confession read.
"I request forgiveness... and am willing to bow down on my knees," it said.
Confessions and self-criticisms - scripted by the authorities - are normally a prerequisite for detained foreigners seeking release in North Korea.
Observers said Pyongyang likely regarded Short as harmless and had played up his deportation as an act of diplomatic largesse, while continuing to hold a more high-profile detainee - American-Korean missionary Kenneth Bae.
"Holding someone of Short's age and nationality carries no real political upside, and could even become a burden," said Yang Moo-Jin, a professor at the University of North Korean Studies in Seoul.
Described by a North Korean court as a militant Christian evangelist, Bae was arrested in November 2012 and later sentenced to 15 years' hard labour on charges of seeking to topple the government.
He was shown on state TV last month, admitting to "wrongdoings" - but Pyongyang has since given no indication he will be released despite calls from Washington and appeals from his family.
Hawk and dove
Since the beginning of this year, North Korea has played both hawk and dove, offering conciliatory gestures one day and issuing dire threats the next.
In what was seen as a major concession, it allowed the first reunion for three years of families divided by the Korean War to go ahead, even though the event overlapped with the start of the annual South-US drills a week ago.
Pyongyang said it had acted out of humanitarian reasons - the same motive it ascribed to its decision to release Short.
At the same time, the North has flexed its military muscles by firing a total of six short-range missiles into the sea in the space of five days.
"The North is taking a double-faced stance by making conciliatory gestures on one hand and pushing ahead with reckless provocation on the other," South Defence Ministry spokesperson Kim Min-Seok told reporters on Monday.
Short's signed statement stated that US and other Western media reports labelling the North as a closed country without religious freedoms were "inaccurate and wrong".
Although freedom of worship is enshrined in North Korea's constitution, it does not exist in practice and religious activity is severely restricted to officially recognised groups linked to the government.
A UN-mandated commission published a damning report last month, detailing horrific human rights abuses in North Korea and concluding that they could comprise crimes against humanity.
In his statement, Short also admitted distributing religious texts on the Pyongyang subway during a previous tour to the North in 2012.
"I now realise the seriousness of my insult to the Korean people... and for this I truly apologise," it said.
Australia has no diplomatic representation in North Korea, but its foreign ministry said his release was "welcome news".
Short's wife, Karen Short, told AFP in Hong Kong that she was "amazingly thankful".