The tourists were 60 for no loss in 17.4 overs at the end of the first day with Shane Watson on 36 and Phillip Hughes at 23 as play was halted 15 minutes early due to bad light.
Earlier, Australian fast bowler Ryan Harris captured three wickets, while Trend Copeland and Mitchell Johnson took two apiece as they bowled out Sri Lanka in 64.1 overs.
Angelo Mathews resisted the Australian bowlers to hit his fourth test half-century before being dismissed for 58, which included three sixes and six boundaries in 111 balls.
He was caught behind off Johnson as Mathews single-handedly tried to lift the score without much support from the other end.
Before tea, part-time medium-fast bowler Michael Hussey dismissed Kumar Sangakkara for 48 to break the sixth-wicket partnership which had helped Sri Lanka to recover from 76 for five in the first session.
Resuming after lunch, Sangakkara and Mathews added 52 runs before Hussey struck with his third ball of the only over he has bowled in the test.
Hussey, who also produced an excellent catch to dismiss Mahela Jayawardene when the total was at 14, told reporters: "I was pretty lucky today that the two probably key wickets were in my hands.
"It's a fantastic test match for the bowlers. They are very clear in their minds on what they are trying to do," Hussey added. "But we have to really bat well tomorrow and bat a long time...set the challenge of one or two batsmen going on getting big hundreds."
Opening bowler Harris removed Tharanga Paranavitana for no score in the third over as the opener prodded an edge to wicket keeper Brad Haddin, leaving Sri Lanka on 2-1.
Then Copeland, who claimed Tillakaratne Dilshan as his first career wicket in the debut Galle test, dismissed the Sri Lankan captain again, forcing him to misjudge the ball which hit the top of the off stump.
Dilshan scored only four runs and his dismissal left Sri Lanka defensive at 10 for two wickets.
"They really bowled well at the start," Dilshan said. "This is a good wicket and you can't get out for just 174 when you play with seven batsmen. We have a problem in out middle order and top order. We may change the batting line up in the second innings.
"This is the third time we have got out cheaply. We tried to survive in the first 10-15 overs as we knew the ball was going to move. But unfortunately we lost three wickets in the first 10 overs."
Sangakkara and Thilan Samaraweera led a Sri Lankan recovery in the first session, adding 43 runs for the fourth wicket, but Samaraweera fell for 17, caught by Haddin off Harris in the 24th over when the Sri Lanka total was only 57.
Sri Lanka had won the toss and elected to bat first as left-handed batsman Shaun Marsh and spinner Seekuge Prasanna made their debuts for Australia and the home team respectively.
Australia won the first test in Galle by 125 runs and lead the three-match series 1-0.
