Scolari's men were favourites to win the tournament but were demolished 7-1 in the semi-finals by eventual champions Germany. To compound their misery, they lost the third-place playoff 3-0 against the Netherlands.
CBF president Jose Maria Marin confirmed Scolari's much expected departure in a statement and thanked the experienced coach for helping "rescue the Brazil team's self-esteem."
"Scolari and all his backroom staff deserve our respect and thanks," Marin said in a short statement published on the CBF's website. "They were responsible for returning to the Brazilian people a love for the national side, even if they did not achieve our highest aim."
Scolari, who led Brazil to the last of their record five World Cups in 2002, took over the post for a second time in November 2012 and lost just five of his 29 games in charge.
He led Brazil to the Confederations Cup last year with a memorable 3-0 defeat of then world champions Spain in the final.
He was hugely popular in Brazil for his gruff charm and also with the players, who looked up to him as a father figure.
He had the support of the nation going into the World Cup and there was very little opposition to his squad selection, a rarity in Brazil, which is famously known as the "nation of 200 million coaches."
Marin did not say who might take over but said a press conference will be held on Thursday.
One Brazilian sports channel reported Alexandre Gallo, the coach of the Brazil under-20 side, would take over until the end of the year.
Among the other names mentioned are Tite, who led Corinthians to the Libertadores Cup and Club World Cup in 2012, and Muricy Ramalho, the current coach of Sao Paulo.
Brazil have four friendlies planned in September and October against Ecuador, Colombia, Argentina and Turkey. Their next competitive fixtures will be in next year's Copa America in Chile.
However, there are also calls for a foreign manager to be appointed for the first time in Brazil's history. The CBF has always resisted that idea, with reports they refused to even talk with Spaniard Pep Guardiola before he joined Bayern Munich.
"Whoever the new man is, he should live in Europe so he can follow or principal players from close up," Carlos Alberto Torres, the captain of the legendary 1970 side, tweeted on Monday.
Many others believe the malaise goes much deeper than the coach and want reforms of the CBF.
The organisation voted to elect a new president in April - Marco Polo Del Nero was the only candidate - but he will not take over from 82-year-old Marin until next year.
That early election was a strategic decision made to protect them from any possible problems in the 2014 World Cup, said Common Sense FC, a group of more than 1,000 professionals who are pressing for reforms within the game.
"The dispute for the presidency is incontestable proof that the concepts of elections and democracy are not necessarily members of the same family," Common Sense FC said in a statement released Monday morning.
"Brazilian football - despite the sad end to the 2014 World Cup - will remain in the hands of the men who brought us here."
