Graham, who became chairman of the Otago Rugby Football Union earlier this year, says off-field structures had been put in place, but things on the field had not worked as well as they should have.
"Obviously, it is a major concern for everybody.
I think it is going to take some time to get the on-field stuff sorted out and we are working hard on that," Graham said.
"As a board and a union we've got to stand back and support the team. I believe we've got the players and the team."
Otago has endured a miserable start to the season, losing its first five games to sit rock bottom of the ITM Cup.
But Graham said the difference between winning and losing was small.
"Look at the performance of Southland.
"There was not a lot between the two sides when they played. And now they are unbeaten and we haven't won a game.
"There is not much between the teams. It is just key things letting us down in games.It is causing us a headache. The frustrating thing is we are there or thereabouts.
"It comes down to two or three mistakes in games and at this level that is all it takes. We should never have lost to North Harbour. We should have really beaten Taranaki.
"But at the end of the day we did not. We need to find the answers sooner, rather than later."
Graham said getting into the top seven would come from playing better and eliminating errors.
The union was working hard to secure long-term issues, he said.
"We have got a long-term strategy and we are working towards that. But that is going to take two to three years.
"We have got some good young kids around who will be good players two or three years down the road.
"But this is about the here and now and the best we can do with what we've got."
Graham and deputy chairman Laurie Mains, both former Otago players and coaches, have been giving as much assistance as possible to coaches Phil Mooney and David Latta, and Graham gave his full backing to the coaches.
"We have got two new coaches at this level who are finding their feet at this level.
"Myself and Laurie are helping out as much as we can, but it comes down to the 22 guys who go out on to the field."
With the side losing, the inevitable talk had started - Otago doesn't train hard enough; the players don't care about losing; there are better local players around.
But Graham said he watched the team train, and it carried plenty of intensity.
He felt the players were the best in the province, though he admitted the union had not done the best in recruitment over the past two or three years.
"The talent is there but it is a hard, gutsy sort of competition.
"You've got to be on your game. There is a real will to win with these guys but a lot of them are still finding their feet.
"They are on a learning curve on how to play winning rugby. They play well for 20 or 30 minutes but you need to play more than that.
"We are not a mile away from winning."
The appointment of Jamie Joseph and Simon Culhane as Highlanders coaches would also help Otago as it would create pathways for players to come to Otago, he said.
Graham said he was pleased the rugby public was not sitting back and was interested in what was happening with the side.
"The Otago rugby public still really cares. A lot of people still care.
"And it is our top priority to get it right."
