Now a museum and popular tourist attraction, the windswept lump of rock in shark-infested waters off Cape Town kept black agitators in isolation for three decades until president F.W. de Klerk began dismantling white-minority rule in 1990.
Mandela, who died on December 5 (local time) aged 95, was first sent to Robben Island for a brief period in 1962 for minor political offences, then returned two years later for a life sentence after being convicted of sabotage and conspiracy to overthrow the state.
Aged 46 when he began his term, Mandela was sentenced with other leading members of the African National Congress to years of hard labour, breaking rocks in a limestone quarry.
Chained in rows of four, prisoners worked eight to 10 hours a day, five days a week. The harsh glare of the sun on the white rocks, combined with the dust, caused permanent damage to Mandela's eyes.
Despite the hardship, Mandela did not direct his frustrations at his immediate captors.
"He was always friendly, polite and helpful," Christo Brand, a prison warder who was with Mandela from 1978 until his release in 1990, told Reuters during a recent visit to the island.
"He became like a father to me. If I needed some help and assistance with something, he was always there for me," said Brand, who now helps at the Robben Island museum.
In his autobiography, "Long Walk to Freedom", much of which was written covertly in his cell on the island, Mandela recalls the loneliness and isolation felt by inmates.
"Robben Island was without question the harshest, most iron-fisted outpost in the South African penal system," he wrote.
"ANOTHER COUNTRY"
Lying 10 km (6 miles) from the glitz and glamour of the resort city Cape Town, the islet looks deceptively close from the shore.
But as you make the 30-minute boat trip across the frigid waters of the southern Atlantic, the flat silhouette dips below the level of the horizon, and the distance seems to grow.
"Journeying to Robben Island was like going to another country," Mandela wrote. "Its isolation made it not simply another prison but a world of its own, far removed from the one we had come from."
The road from the sheltered Murray Harbor, where prisoners were corralled after disembarking, leads to a grey stone building surrounded by barriers topped with barbed wire.
Marched to the single storey maximum security wing, prisoners were mockingly greeted by the words "Welcome to Robben Island: We serve with pride" painted on an arch over the road.
In B Section, visitors are guided by former political prisoners to Mandela's single cell. A tiny table, with metal cup and plate, and a dented ablution tin remain as they were decades ago.
A small window overlooks the courtyard, while a makeshift bed with three grey blankets atop a sisal mat provide scant protection from the cold, bare cement floor.
"I could walk the length of my cell in three paces. When I lay down, I could feel the wall with my feet and my head grazed the concrete on the other side," wrote Mandela, once a lanky amateur boxer.
Mandela's prison number was 466/64, made famous now in global campaigns to raise money to fight HIV/Aids and for his beloved Children's Foundation.
LAWYER'S TRAINING
Conditions during his 18 years on the island fluctuated, echoing the broader political climate on the mainland, as well as the whim of officials overseeing the jail. A particularly harsh regime was implemented after prime minister Hendrik Verwoerd was assassinated in September 1966.
But Mandela used his training as a lawyer to hamper many of the attempts by warders to harass prisoners, and insisted on their right to study for university degrees. He also made a point of trying to talk to prison guards, most of them Afrikaans-speaking whites.
Besides his iron will and principled stance, Mandela's easy charm and generosity helped to chip away at the prison's strict rules and disciplinarian barriers.
Occasionally, prisoners were allowed to cook themselves stews of clams, mussels, crayfish and abalone during lunch breaks while they harvested seaweed for export to Japan.
They were also able to carry out ritual circumcisions, an important part of tribal custom marking initiation into manhood.
"The inmates seemed to be running the prison, not the authorities," Mandela said.
The island's first political prisoner, a local chief named Autshumato, was imprisoned in 1658 but escaped a year later with fellow prisoners in a stolen boat, one of the handful of successful escapes that historians record.
In the 1800s, British colonial authorities sent lepers and the mentally ill to the island, which was named by Dutch colonialists after the seals - "robben" in Dutch - that bred there.
Currently about 200 people, including former political prisoners and museum staff, live on the island, which was declared a World Heritage Site by UNESCO in 1999.
To help complete its transformation from a forbidding place of exile, the island now hosts weddings on Valentine's Day, attracting couples from around the world who want to exchange vows in the main village's historic Garrison Church.