Following a campaign roiled by a series of violent attacks, and interventions by US President Donald Trump's administration, the conservative CDU/CSU bloc won 28.7% of the vote, followed by the AfD with 19.8%, the projection published by ZDF public broadcaster showed.
"Tonight we will celebrate, and from tomorrow we start working. ... The world out there is not waiting for us," Merz, 69, told supporters.
Merz is heading into what are likely to be lengthy coalition talks without a strong negotiating hand. While his CDU/CSU emerged as the largest bloc, it scored its second worst post-war result.
It remains unclear whether Merz will need one or two partners to form a majority. A three-way coalition would likely be much more unwieldy, hampering Germany's ability to show clear leadership.
All of the mainstream parties have ruled out working with the AfD.
Chancellor Olaf Scholz's Social Democrats (SPD) tumbled to their worst result since World War Two, with 16.4% of the vote share, according to the ZDF projection, while the Greens were on 12.3% and the far left Die Linke party on 8.9% of the vote.
The pro-market Free Democrats (FDP) and newcomer Sahra Wagenknecht Alliance (BSW) party hovered around the 5% threshold to enter parliament.
MERZ PROMISES LEADERSHIP
Merz has no previous government experience but has promised to provide greater leadership than Scholz and to liaise more with key allies, restoring Germany to the heart of Europe.
Merz also conditionally supports equipping Ukraine with longer-range Taurus missiles, a step Scholz's government shied away from, and sees Europe as firmly anchored in NATO.
A brash economic liberal who has shifted the conservatives to the right, he is considered the antithesis of former conservative Chancellor Angela Merkel, who led Germany for 16 years.
As such coalition negotiations are certain to be tricky, especially after a campaign which exposed sharp divisions over migration and how to deal with the AfD in a country where far-right politics carry a particularly strong stigma due to its Nazi past.
That could leave Scholz in a caretaker role for months, delaying urgently needed policies to revive Europe's largest economy after two consecutive years of contraction and as companies struggle against global rivals.
It would also create a leadership vacuum in the heart of Europe even as it deals with a host of challenges including US President Donald Trump threatening a trade war and attempting to fast-track a ceasefire deal for Ukraine without European involvement.
Germans are more pessimistic about their living standards now than at any time since the financial crisis in 2008.
Attitudes towards migration have also hardened, a profound shift in German public sentiment since its "Refugees Welcome" culture during Europe's migrant crisis in 2015, that the AfD has both driven and harnessed.
AFD MAKES HISTORY
Sunday's election came after the collapse last November of Scholz's coalition of his SPD, the Greens and pro-market FDP in a row over budget spending.
It has also been overshadowed by the unusually forceful show of solidarity by members of the Trump administration - including Vice President JD Vance and tech billionaire Elon Musk - for the anti-migrant AfD, and broadsides against European leaders.
The 12-year old AfD party took second place for the first time in a national election, according to the exit polls.
"This is a historic result for us," AfD co-leader Alice Weidel said as party supporters waved German flags in celebration.
"We are open to coalition negotiations ..otherwise there will be no change in policy possible in Germany."
Support for the AfD, along with a small but significant vote share for the far left and the decline of Germany's big-tent parties, is increasingly complicating the formation of coalitions and governance.
A note by Capital Economics said it was still unclear at this point what coalition options will be viable.
"Another three party coalition would arguably be bad news as reaching a consensus is likely to be more difficult than with only two parties," it said.
