The internationally recognised artist regards himself as a "southerner", with strong family links to the South.
He has temporarily opened the shop in Queen St, which will remain open until the third week of this month.
He has been delighted by the response from visitors, who have been "quite stunned".
"This old shop appeals to me. This is sort of me. I feel comfortable in here. I've got the kettle in the corner and paintings over here," he said gesturing to some of his spectacular work.
Mr Caley has a passion for Maori culture and has a collection of large oil portraits of Maori.
New Zealand nearly lost the collection - he received a "magnificent offer" last year and nearly accepted.
The offer would have meant money would not have been an issue for the rest of his life, Mr Caley said, but he turned it down.
One of his most special commissions, a portrait of Witarina Harris, was officially unveiled last year.
Mrs Harris died in 2007 aged 101. Mrs Harris, then Witarina Te Miriarangi Parewahaika Mitchell, played Princess Miro in the Hollywood movie Under the Southern Cross, later known as The Devil's Pit, in 1929.
Mr Caley was asked to do the painting by Sir Howard Morrison and he said Mrs Harris' family and the public were thrilled with the result.
He had no idea why art critics "had a go" about the painting.
"I was thrilled. It had everything you could possibly ask for."
Negative comments from art critics in New Zealand were nothing new for Mr Caley, who defended his style, saying it was "certainly not chocolate box".
He had been advised if he stopped his portraits, his career would take off, but he could not.
"I can't. It's part of me," he said.
The critics needed to open their eyes and learn it was not photographic painting but a very modern style, he said.
He enjoyed the mysteries of abstract art, which he combined with realism.
Mr Caley would like to see a mixed cultural gallery established - showcasing Maori as well as Pakeha art - and Waimate was one possible venue.
Financial support would be needed, but whatever town it was done in, it would "go to the lights".
He could turn a gallery into an "icon", he said.
Mr Caley, who has travelled widely, said he had loved his life.
"I just do my thing and if it works and goes well, I'm happy. If it doesn't work and it goes badly, I'm still happy. I've always done what I feel I should do."
