A surprise "yes" vote would have deprived Paris of a foothold in a region where China is expanding its influence, and dented the pride of a colonial power whose reach once spanned large parts of Africa, Southeast Asia, the Caribbean and the Pacific.
The "no" vote won 53.26% of votes cast votes, a narrower margin of victory than in the 2018 poll.
The result will bring relief to President Emmanuel Macron's government, whose attention had been focused on the coronavirus pandemic in the run-up to the referendum.
"Voters have had their say. They confirmed their wish to keep New Caledonia a part of France. As head of state, I salute this show of confidence in the Republic with a profound feeling of gratitude," Macron said in a televised statement.
New Caledonia became a French colony in 1853. Tensions have long run deep between pro-independence indigenous Kanaks and descendants of colonial settlers who remain loyal to Paris.
Sunday's referendum was the second of up to three permitted under the terms of the 1998 Noumea Accord, an agreement enshrined in France's constitution and which set out a 20-year path towards decolonisation.
France stood ready to organise a further vote within two years if that was the wish of New Caledonians, at which point both sides would have to accept the result, Macron said.
"We have two years to look to the future," the president continued.
The island chain already enjoys a large degree of autonomy but depends heavily on France for matters such as defence and education.
Turnout was high at 85.6% of eligible voters - after a stronger-than-expected independence vote in the 2018 referendum.
New Caledonia lies some 1200km east of Australia and 20,000km from Paris.
Under French colonial rule the Kanaks were confined to reserves and excluded from much of the island’s economy. The first revolt erupted in 1878, not long after the discovery of large nickel deposits that are today exploited by French miner Eramet’s subsidiary SLN.
Today, its economy is underpinned by annual French subsidies of some 1.3 billion euros ($1.5 billion) and nickel deposits that are estimated to represent 25% of the world’s total, and tourism.
The territory has, however, largely cut itself off from the outside world to shield itself from the coronavirus. It has registered only 27 cases of Covid-19.
For a third referendum to take place, a third of the local assembly in New Caledonia must vote in favour of one.
