"We are facing a terrorist monster," Iraq's human rights minister, Mohammed Shia' Al Sudani, told the U.N. Human Rights Council which adopted a resolution tabled by Iraq and France at an emergency sitting of the 47-member state forum in Geneva.
The Council aims to send 11 investigators, with a total budget of $US1.18 million, to report back by March 2015.
Islamic State, which declared a "caliphate" in June in parts of Iraq and Syria under its control, has been cited as a major security threat by Western governments since posting a video in August of the beheading of U.S journalist James Foley.
The Sunni militants have driven more than 1.2 million people from their homes this year, the United Nations says. At least 1,420 people were killed in sectarian violence in Iraq in August alone, UN figures showed.
UN Deputy High Commissioner for Human Rights Flavia Pansieri said there was "strong evidence" Islamic State, also known as ISIS, and allied groups had carried out targeted killings, forced conversions, sexual abuse and torture in Iraq.
"The reports we have received reveal acts of inhumanity on an unimaginable scale," she said.
ETHNIC CLEANSING
Pansieri voiced concern at the persecution of Christians, Yazidis, Shia, and Turkmen, saying such "ethnic and religious cleansing" may amount to crimes against humanity.
Children belonging to targeted minorities have been forcibly recruited and positioned on front lines to shield fighters or made to donate blood, she said. Women are beaten for breaking rules requiring them to be veiled and escorted by men.
Al-Sudani told the session that Islamic State, "oozing with barbarity", was threatening the makeup of his country.
"The land of ancient Babylon is subjected to threats, starting with threats to its very independence. They are attempting to change its demographic and cultural composition," he said, adding: "Acts by ISIS threaten not only Iraq but the whole region and world."
Pansieri raised concerns that Iraqi government forces had also engaged in acts that may amount to war crimes.
She said government-allied militias had opened fire on a mosque in Khanaqin district, northeast of Baghdad, killing 73 men and boys. Iraqi soldiers had shelled towns and carried out air strikes killing and injuring dozens of civilians, she added.
The US envoy to the rights forum, Keith Harper, urged Iraq's prime minister designate, Haider al-Abadi, to form a multi-ethnic government that would investigate allegations against government forces and "terrorist groups".
"The stories that have emerged from ISIL's bloody assault on Iraq are the ones of nightmares. Christians and others have been driven from their homes with the threat of 'convert or die'," Harper said.
"The Yazidis have been buried alive, beheaded or killed in mass executions."
Al Sudani, asked whether the UN investigators would look into alleged crimes by government forces, said they would focus on those committed by the Sunni militants and that the Baghdad government would look into allegations against state forces.
"These are allegations of crimes perpetrated by the government and we have a transparent investigation by our own government which will soon be deposited with all concerned members of the Human Rights Council," he told Reuters