Prime Minister Jacinda Ardern says she retains confidence in the Speaker, but has expressed "serious concerns" to him about his conduct in Parliament last night.
In a fiery debate, Trevor Mallard told MPs that the man he wrongly accused of rape in 2019 was guilty of a serious sexual assault.
National MP Chris Bishop then labelled Mallard a bully and said his "display of petulance and contempt and hate" made him unsuitable to be Speaker.

"Nor do I consider it to have met the needs of the victim in this situation. The Speaker acknowledges he did not meet his own standards either."
Ardern added that she also believed the behaviour of members of the Opposition was "inappropriate".
"Issues of this serious nature should not be litigated in Parliament in such a manner. It was wrong.
"Parliament rightly needs to set a standard for others to follow. The Francis Review and its recommendations, including the introduction of specific Behavioural Standards for all who support the work of Parliament, offer a blueprint for best practice and I believe Parliament should be focused on that.
Ardern has asked Mallard to reconvene a cross-party working group to consider how the Behavioural Standards can be given practical effect when MPs are dealing with sensitive staff conduct matters such as sexual assault.
"Parliament must continue to maintain its right to hold government ministers and the Speaker to account for actions. However, this can be done in a robust and respectful appropriate.
"I urge all parties and MPs to adopt a bipartisan approach to ensure Parliament is a good and safe place for staff to work," Ardern said.
Hipkins calls on MPs to 'step back'
Earlier today, senior Labour minister Chris Hipkins - who is also leader of the House - called on all MPs to "step back" from the dispute, but said Labour maintained confidence in Mallard.
Hipkins said he sat through last night's debate and disapproved of the conduct of all involved.
"I don't think it reflected well on Parliament as a whole. I don't think it reflected well on pretty much anybody that was taking part in the debate."
Hipkins said the debating chamber was not the best place to deal with "these types of issues".
"I don't think that was a victim-centric approach. I don't think it was fair on the person who the allegations were against either.
"I think the whole of Parliament needs to step back from this... Continuing down this road isn't the appropriate course of action for anybody at Parliament."
Hipkins said he would raise the debate with Mallard, but: "the Labour Party continues to support the Speaker".

Mallard unfit for role, says Collins
National Party leader Judith Collins has again called for Mallard to be removed as Speaker, describing his behaviour last night as a disgrace and saying he is "temperamentally unfit" for the role.
In what has been described as a "tawdry, sordid" affair by the Opposition, Mallard accused a former parliamentary staffer - to whom he had previously apologised for claiming he was a rapist - of sexual assault.
Appearing before the House last night as the minister responsible for Parliamentary Service, Mallard told MPs that the man he wrongly accused of rape in 2019 was guilty of a serious sexual assault.
He rebutted allegations that he had ruined the man's life, saying "that man's life was destroyed when he sexually assaulted a woman".
"That's what did it... I will support the woman and what she said, I will support the investigation that found that he seriously assaulted her, and I will support the police and their investigation and the results of that."
Collins said the Speaker seemed to lose control of his behaviour and his emotions last night.
"He continued down the path of using Parliament to defame a former staff member to whom he has already had to apologise publicly and to whom the taxpayer had to fund or pay out a large amount of money," she said.
Collins said it was important for people to go through a proper process when dealing with allegations.
"That is why we have courts, that's why we don't have kangaroo courts where people never get a chance to have their say. What I saw last night was a Speaker who is entirely temperamentally unsuited for the role."
There was also discussion in Parliament about why the cases had not either been reported to police or resulted in a prosecution.

In other cases police could decide not to proceed if there was not a great chance of success, he said.
"I want to say that in two cases that I have been dealing with there is one of each that fits into that category," Mallard said.
But Collins said National was championing proper process when raising these types of complaints and the Speaker should not have behaved in the way he did last night.
"Utterly bullying, entirely at times almost incoherent with anger at being questioned, we saw a man who is temperamentally unfit for the job."
While he had apologised for his mistake, Mallard said it did not "erase the fact that there were serious allegations from staff members that had to be taken seriously", and he believed the victims.
That "mistake" cost the taxpayers more than $330,000 following a defamation settlement, and an apology from Mallard for the distress and humiliation caused.
The Opposition has repeatedly tried to have him removed, saying Mallard continued to make allegations of rape when he knew that was not the case.
Collins said she is writing to the Prime Minister today asking her to remove Mallard from office. She said Ardern could not just say it was a matter for Parliament.
"She controls Parliament with her numbers, the National Party will obviously support her, I believe the ACT Party will too - it is simply unacceptable that this man is in a position of judge and jury against MPs and their behaviour and the staff members of parliamentary service," Collins said.
Collins said that the Speaker's role was to be a very careful judge of behaviour but Mallard's behaviour last night showed he was unfit for the job.
"I say that on the basis of his temperament - what we saw was a display of anger, of petulance, of vindictiveness. I have never seen that behaviour to that extent in Parliament from any MP, let alone the Speaker.
"A Speaker needs to be of the same temperament as a judge, a speaker needs to have a temperament where they don't actually make the situation worse, that's what we're seeing from Trevor Mallard."

Bishop criticises Mallard
Throughout yesterday's fiery debate, National MP Chris Bishop continued to press Mallard, asking why he "did not clean up the matter immediately" once he considered he had made a mistake.
He criticised Mallard for using parliamentary privilege - which protects MPs from defamation - "to relitigate what happened".
"My challenge to the Speaker is to go and say those words outside this House. I'm willing to say that he will not do that".
At one point, Bishop yelled at the Speaker he was "a bully" and that he had no confidence in his ability to oversee reforms to improve the culture of Parliament.
Bullying and harassment was found to be rife in Parliament in the independent external review by Debbie Francis, released in May 2019, outlining incidents of sexual harassment and assault.
That report is what sparked the rape allegation controversy which have plagued Mallard over the last 18 months.
Labour MPs did their best to run interference last night, with Kieran McAnulty trying to draw a line in the sand around politicians not using their "position to politicise these [allegations of sexual assault], to drag them out, and revictimise those who are accusing of sexual assault".
Labour MP for Northland Willow-Jean Prime told Parliament at some points Opposition MPs' comments sounded like "she asked for it, her skirt was too short, she was drunk," drawing the fury of Bishop.
Deputy Prime Minister Grant Robertson was working on the Budget and was not following the annual review debate.
Robertson said there were obviously some "robust exchanges" but he was not in a position to comment on it and nor would he comment on matters dealt with under Parliamentary privilege.
"I have zero involvement in what happened in the House, last night because I wasn't there.
"Obviously, the annual review debate is the opportunity for both the Speaker in his role looking after Parliamentary Service to speak about what's happened, and also for Parliamentarians to hold him to account.
"From the perspective of the Labour Party, these issues as such that they can be talked about in public were covered by Mr Mallard when he apologised for the comment that he had made previously, from our perspective that part of the matter rested there.
"What happened in Parliament last night is ultimately a matter for the Speaker and for Parliament."