Nadal won 40 percent of his first serves in the opener as he struggled with the high humidity on Arthur Ashe Stadium court, with Thiem becoming the first player to 'bagel' the Spaniard at Flushing Meadows since Andy Roddick 14 years ago.
"It's been a great battle," an exhausted Nadal said. "The conditions were tough, very humid for me."
Thiem, who lost to Nadal in the French Open final but had beaten the Spaniard three times in 10 previous meetings, struck 74 winners, including 24 with his stylish single-handed backhand, to keep the champion under constant pressure.
"I'm very sorry for Dominic," Nadal added. "He's a close friend on tour, a great guy and I wish him the very best ... He is young and has plenty of time to win tournaments."
After Nadal roared back to take a 2-1 lead, the fourth set went to a tiebreak. Thiem raced to a 3-0 lead before Nadal fought back, but the Austrian held his nerve to level the match.
There were no breaks of serve in the deciding set and a second tiebreak ensued. The two were neck-in-neck at 5-5 when Nadal carved out a match point with his forehand.
Thiem's wayward overhead smash handed the match to Nadal, who stood midcourt with his head thrown back and arms raised in the air in celebration.
"It's going to be stuck in my mind forever," Thiem said.
"I'm going to remember this match, for sure. It's cruel sometimes tennis, because I think this match didn't really deserve a loser. But there has to be one.
"If we skip the first set, it was a really open match from beginning to the end. The way it ended up in the fifth set tiebreaker, from there it's 50-50. He made one more point than me."
Nadal paid tribute to the crowd for hanging on until the early hours of Wednesday morning.
"Thank you very much everybody for staying here tonight," Nadal said after the four hour, 49 minute marathon.
"It's been an amazing feeling. I'm playing a lot of hours this year at this beautiful stadium."
Next up for the top seed is a semi-final clash with big-serving Argentine and third seed Juan Martin del Potro.