Sergeant Matt Lee, of Dunedin, said police were called to Truby King Reserve near the former Seacliff asylum yesterday afternoon.
Two members of the public discovered what appeared to be a human femur bone and an old patient tag in a "shallow grave" within the grounds.
Police seized the bone for examination and it was found to be the remains of a dog, Sgt Lee said this afternoon.