The Dunedin co-educational school made the decision because it feared it did not have the player depth to match other teams and also had concerns over player safety.
Highlanders First XV committee chairman Craig Morton said it was disappointing the side had pulled out but he could understand the reasons. It would not be replaced, which would create a double bye in the northern section in the opening round.
Kavanagh College had struggled in the competition last season, finishing 11th.
Morton said no other Dunedin school was willing to replace it.
A suggestion of getting some Dunedin co-educational schools to combine to make up a team had not gained any traction with schools.
Kavanagh's withdrawal has left just three Dunedin schools in the competition - King's, Otago Boys' and John McGlashan.
Kavanagh College principal Tracy O'Brien said a few players did not come back for their final year at school and a few others had left.
The bulk of the side would have been year 11 pupils and he did not want to expose them to the tough physical nature of the game. There were safety concerns about them playing against bigger and older boys.
He confirmed the side had only half a dozen year 13 pupils and did not have a second XV. The other sides at the school were under-15 and under-14 teams.
The first XV would play in the metropolitan under-18 competition and he hoped Kavanagh College would be back in the Highlanders First XV competition next year.
O'Brien said it was becoming hard for co-educational schools to compete against all-boys schools which put an emphasis on sport and poured resources into teams.
Kavanagh would have to qualify for next year's competition.
It had the bye in the first round last week and played Dunstan High School, which also had a bye, in a friendly, and lost by more than 50 points. It had also lost narrowly to Taieri College in pre-season.
The side was due to play Otago Boys' on Saturday.
There was a possibility a Southland side, such as St Peter's College in Gore or Central Southland College, would be added to the second round to eliminate a bye in the bottom section.
