A man wearing gloves, with his face hidden by a scarf, burst into the hotel in the late morning and demanded to be handed several bags containing jewels and diamond-encrusted watches.
"Everything happened very quickly," a judicial source said, adding that there was no violence.
The Carlton, located on the French Riviera resort's famed Croisette waterfront promenade, has been housing a temporary exhibition of jewels from diamond specialist Leviev since July 20.
The hotel, celebrating its centenary this year, was also where Alfred Hitchcock filmed scenes from the 1955 film "To Catch a Thief", starring Cary Grant as a one-time cat burglar and Grace Kelly as the beautiful Frances.
Nice-Matin newspaper said the stolen jewels were worth 40 million euros ($NZ65 million), a figure that the Carlton's management would not confirm. The prosecutor in Grasse, near Cannes, gave a figure of "several million euros".
Police investigators are studying video surveillance footage in an effort to track down the thief.
The heist came two months after two smaller jewellery robberies hit the annual Cannes film festival, where many of the world's top movie stars are lent gowns and gems to parade on the red carpets and at glamorous parties.
