In a match that lasted more than three hours, the fourth-ranked German came back from a break down twice in the deciding set but was broken again at 3-4. Petkovic, who is now 0-4 against Martic, won the first of her two WTA titles in Bad Gastein in 2009.
Martic will face Patricia Mayr-Achleitner in the second round. The Austrian defeated Tereza Mrdeza of Croatia 6-4, 6-3.
Earlier, second-seeded Annika Beck overcame 12 double faults in a 7-5, 6-3 win over Shahar Peer of Israel. The German, who entered the top 50 for the first time this week after her fourth quarterfinal of the season at the Hungarian Grand Prix, lost her serve four times but saved eight more breakpoints.
Beck next plays Mandy Minella of Luxembourg, who defeated Dia Evtimova of Bulgaria 6-2, 2-6, 6-3.
Also, Viktorija Golubic of Switzerland held serve throughout to defeat No. 5 Kiki Bertens of the Netherlands 6-2, 6-2 and set up a second-round match against Andrea Hlavackova of the Czech Republic, who beat Eleni Daniilidou of Greece 6-3, 6-1.
In her first career WTA match, Austrian wildcard Lisa-Marie Moser edged Romania's Elena Bogdan 7-6 (6), 2-6, 6-4 and will next play top-seeded Mona Barthel.
Also, No. 6 Chanelle Scheepers of South Africa downed Anna Schmiedlova of Slovakia 6-1, 6-4; Estrella Cabeza Candela of Spain defeated Tadeja Majeric of Slovenia 6-1, 6-3; Elina Svitolina of Ukraine beat Michaela Honcova of Slovakia 6-2, 6-2; and Alexandra Cadantu saw off Maria Joao Koehler of Portugal 6-2, 6-4.
Results from day two of the WTA Bad Gastein Open on Tuesday (x denotes seeding):
Lisa-Maria Moser (AUT) bt Elena Bogdan (ROM) 7-6 (8/6), 2-6, 6-4
Andrea Hlavckov (CZE) bt Eleni Daniilidou (GRE) 6-3, 6-1
Viktorija Golubic (SUI) bt Kiki Bertens (NED x5) 6-2, 6-2
Petra Martic (CRO) bt Andrea Petkovic (GER x4) 6-7 (5/7), 7-5, 6-3
Patricia Mayr (AUT) bt Tereza Mrdeza (CRO) 6-4, 6-3
Elina Svitolina (UKR) bt Michaela Honcova (SVK) 6-2, 6-2
Chanelle Scheepers (RSA x6) bt Anna Karolna Schmiedlov (SVK) 6-1, 6-4
Estrella Cabeza (ESP) bt Tadeja Majeric (SLO) 6-1, 6-3
Alexandra Cadantu (ROM) bt Maria Joo Koehler (POR) 6-2, 6-4
Mandy Minella (LUX) bt Dia Evtimova (BUL) 6-2, 2-6, 6-3
Annika Beck (GER x2) bt Shahar Peer (ISR) 7-5, 6-3