The gold medal winner in Beijing four years ago has failed to recover from a knee problem, and also misses out on the honour of carrying his nation's flag at the opening ceremony.
The Spanish tennis federation (RFET) said Feliciano Lopez would be his replacement in the singles, while the country's Olympic committee (COE) said a meeting on Friday would decide on a replacement flag bearer for the July 27th parade.
"I'm not in a condition to be able to compete," Nadal, 26, said in a statement. "It's one of the saddest moments of my career."
Nadal won a record seventh French Open title in June but crashed out in the second round at Wimbledon in a massive upset against lowly ranked Czech Lukas Rosol.
Earlier this month, he pulled out of a charity match against Novak Djokovic at Real Madrid's Bernabeu stadium saying he was suffering from tendinitis and needed to rest.
Nadal had hoped to return to Wimbledon this month to defend his Olympic title and his withdrawal echoes his plight of 2009 when a similar knee problem meant he had to withdraw from the grasscourt grand slam, a year after winning it for the first time in an epic final against Roger Federer.
"I have to think of my colleagues, I cannot be selfish and I have to think of the well-being of Spanish sport, especially Spanish tennis, and let a colleague play who has had a better preparation and is in a state to compete," the world number three added.
"I pushed until the last moment in my preparations, in training, but it wasn't to be.
"Today is one of the saddest days of my career as it was one of my biggest dreams, and perhaps the most anticipated moment to be the flag bearer at the opening ceremony for the Games in London.
"You can imagine how difficult it has been to take this decision."
Messages of support quickly flooded in for the Mallorca native, one of the first coming from compatriot David Ferrer.
"It's a shame about Rafa, the most important thing is that he recovers soon. He is the best... Keep your spirits up," the world number five wrote on his Twitter account
The COE president Alejandro Blanco said Nadal was devastated.
"It is a sad day for him but once again he has proved what a great person he is," Blanco told Spanish radio.
"If he isn't 100 percent, if he isn't able to go for the victory, if he isn't convinced he can be Olympic champion he prefers to let another person try."
RFET president Jose Luis Escanuela said in a statement: "It is the worst news we could have. Rafa has always been an example of perseverance and tenacity... we wish him the quickest of recoveries."
The debate as to who would replace Nadal as flagbearer had already started in the local media.
Sailing duo Iker Martinez and Xabi Fernandez, who have won a gold and a silver, were in the running along with basketball silver medalist Pau Gasol.
The tennis competition at the All England Club begins on July 28 with Serbia's Djokovic the top seed.
