The repeatedly vandalised billboard outside St Matthew-in-the-City Anglican Church featured pictures of a sombre Mary and Joseph in bed, with the caption "Poor Joseph. God was a hard act to follow".
After four attacks on the billboard, the church took it down.
The ASA received a number of complaints, saying the billboard was offensive, insensitive, blasphemous, ridiculed the beliefs of Christians, perverse, and distorted children's perceptions of Christmas.
However, it chose not to uphold the complaints and sided with the church, saying the billboard may have been controversial, but it wasn't obscene.
It had no naked people on it, depicted no sexual acts and "had been prepared with a due sense of social responsibility to consumers and to society", the ASA said.
