The fire scorched an area of land behind the camp site at Ocean View late on Friday, leaving it blackened and burned the next morning.
Israeli camper Tomel, who asked that just his first name was used, said he heard a vehicle arrive shortly after 11pm.
He then saw fireworks set off, which were placed in a can in the middle of the camping area.
He did not think those who set them off intended to start a fire, but they left straight after they were lit.
Brighton volunteer chief fire officer Grant Tapp said when officers arrived the fire was moving south towards nearby houses.
More trucks and resources were called in to help out, with units arriving from Roslyn, Lookout Point and Mosgiel.
The area burned ended up being about 50sq m, though it was not too hard to put out because of good access.
Some freedom camping vehicles had to be moved for hoses to get in, and "we didn't want to disturb the evidence of the fireworks in the middle".
Mr Tapp also understood the fireworks had been let off by people who arrived in a car and left straight after.
He was not sure if the incident was a prank, but police were involved and had taken evidence from the scene.
He said the perpetrators had put people at risk in the camping area, as well as those in nearby houses.
