Small, a key member of the Springboks side who lifted the 1995 Rugby World Cup trophy, scored 20 tries in 47 tests between 1992 and 1997.
Small was praised for his performance in the World Cup final, where his brave tackles on giant New Zealand winger Jonah Lomu helped the Boks to a surprise victory at Ellis Park.
"He always played with the type of passion and courage that encapsulate what Springbok rugby stands for, and he lived his life in the same way," South Africa Rugby president Mark Alexander said in a statement.
"At 50 years old, James Small died too young. Our thoughts and condolences are with his family and friends during this very sad time."
Known for his aggressive style of play, Small was the first South African to be sent off in a test when it was rare in rugby, banished in a match against Australia in 1993.
He had numerous run-ins with opposition players, and sometimes his own team mates, earning a reputation for his feisty personality that lived on after his career.
Small is the third member of the world champion team to pass away after loose-forward Ruben Kruger succumbed to brain cancer in 2010 and scrumhalf Joost van der Westhuizen died after a long battle with motor neurone disease two years ago.
