Ancient Syria
A concept superficially similar to the biblical scapegoat is attested in two ritual texts in archives at Ebla of the 24th century BC. They were connected with ritual purifications on the occasion of the king's wedding. In them, a she-goat with a silver bracelet hung from her neck was driven forth into the wasteland of 'Alini'; "we" in the report of the ritual involves the whole community. Such 'elimination rites', in which an animal, without confession of sins, is the vehicle of evils (not sins) that are chased from the community are widely attested in the Ancient Near East.
Ancient Greece
The Ancient Greeks practiced a scapegoating rite in which a cripple or beggar or criminal (the pharmakos) was cast out of the community, either in response to a natural disaster (such as a plague, famine or an invasion) or in response to a calendrical crisis (such as the end of the year). The scholia refer to the pharmakos being killed, but many scholars reject this, and argue that the earliest evidence (the fragments of the iambic satirist Hipponax) only show the pharmakos being stoned, beaten and driven from the community.
The Bible
The scapegoat was a goat that was designated "for Azazel" and driven off into the wilderness as part of the ceremonies of the Day of Atonement, that began during the Exodus with the original Tabernacle and continued through the times of the temples in Jerusalem. The rite is described in Leviticus 16.
Since this goat, carrying the sins of the people placed on it, is sent away to perish, the word "scapegoat" has come to mean a person, often innocent, who is blamed and punished for the sins, crimes, or sufferings of others, generally as a way of distracting attention from the real causes.
In Christian theology, the story of the scapegoat in Leviticus is interpreted as a symbolic prefiguration of the self-sacrifice of Jesus, who takes the sins of humanity on his own head, having been driven into the 'wilderness' outside the city by order of the high priests. (Also see John 1:29 and Hebrews Chps. 9-10)
Elsewhere
Scapegoating in its most brutal form was carried out in tribal communities of the ancient world like the Vikings, where a person not meeting the physical or spiritual criteria of the community was simply put to death.