Oliver Joel Fry, 28, appeared in the Dunedin District Court after admitting that as a result of his drug use, he had no recollection of his offending in 2022.
Details of his "dreadful childhood" were shared with the court and Judge Jim Large said it might explain his behaviour, but not excuse it.
On two occasions in August, last year, the defendant and the child were watching cartoons on his bed.
He put his arm around the victim, moved his hands down her body and violated her, despite her clear protests.
"Please don’t do that, I don’t like it," the girl said.
Fry apologised after one of the assaults, saying he had forgotten how old she was.
A month later, the victim’s father noticed the door to his home was locked, which he found "unusual", court documents said.
Once he was inside, he told the defendant and his daughter he was heading out to meet someone.
Instead, he waited outside the property, watching through the window.
As his daughter cut the defendant’s hair in the bathroom, Fry kissed her and touched her inappropriately.
As the pair moved to the kitchen he saw the defendant grabbing at the victim’s top and groping her.
He smacked her on the bottom several times before they went to his bedroom.
The curtains were closed, but the man heard Fry refer to his genitals, saying they "did not work" due to his medication.
Fry instructed the girl to remove her tights and put on a dress, which upset her.
The victim’s father entered the house and told his daughter it was time for bed.
The next morning he confronted Fry but the man denied assaulting the child.
After slapping the defendant in the face, the man threw him out of the house.
Counsel Alex Bligh said her client was "tortured by his offending" and was mortified to have become "the thing he despised".
Judge Large said the man’s expressions of remorse were "too little, too late".
The 11-year-old provided a statement to the court detailing the trauma she had experienced, stating: "Life sucks because of him."
She had trouble controlling her moods and felt "dirty".
"It makes her feel sick ... She can’t eat some foods, because that’s the food you shared with her," Judge Large said.
"I accept that you have had a difficult background yourself, but you need to hear and take on board the effects your actions have had on your victim."
Fry was convicted of two counts of sexual conduct with a child under 12 and a charge of wilful damage, which arose from a separate incident.
He was sentenced to 16 months’ imprisonment.
