Chardy, the 28th seed, put up little fight in the evening sunshine on Centre Court with the third round match wrapped up in 86 minutes.
Clinical Djokovic, who eased past the same player in the first round two years ago on his way to a first Wimbledon title, is yet to drop a set in three one-sided contests so far.
His next opponent, evergreen Tommy Haas, should put up more resistance in the last 16.
German Haas surprisingly beat the Serb in straight sets at the Sony Open in Miami in March, although Djokovic took revenge in the quarter-finals of the French Open earlier this month.