The Otago Daily Times reported last month the department had declined an Official Information Act request to release the New Zealand Rugby Union-supplied documents.
Mr Tremain said he had confidence in his department, and "the decision not to release the information was made by the Department of Internal Affairs".
"The rationale for the decision not to release was that it might prejudice the supply of similar information. Requesters have the right to seek an investigation and review of the decision by the Department to refuse to release the information."
The ODT has lodged a complaint with the Office of the Ombudsman.
Labour's internal affairs spokeswoman Ruth Dyson said the refusal by the department to release the documents aroused suspicion.
"Internal Affairs' refusal to release information, especially on such weak grounds, arouses public suspicion. What is being hidden from us and why?
"Club members and local communities deserve to know how their clubs are funded."
If there was a valid reason to withhold information concerning the funding of the ORFU, the department should make that clear, she said.
A spokesman confirmed while the union was not under investigation, its biggest funder, The Trusts Charitable Foundation, (TTCF), was having its audit finalised.
This year, the ODT reported the union was involved in buying three Auckland bars, and entered in an alleged funding relationship with TTCF.
