The foundation of this teaching is laid by the Gemara itself: Nedarim 22b records a trilogy of rulings — that one who gets angry treats the Divine Presence as of no account, that anger causes a person to forget his learning and accumulate foolishness, and that his sins are acknowledged to outnumber his merits — all derived from explicit pesukim.
Building on this, Orchot Tzadikim 12:4 articulates the inner mechanism: anger drives a person's intellect out from within him, multiplying quarrelsome and contentious speech, so that it becomes impossible for someone given to anger to be saved from grave sins.
Mesillat Yesharim 11 extends the analysis, describing the anger-prone person as one whose heart is no longer with him and whose judgment is utterly lost — likened to a predatory beast — and applies to him the pasuk 'you who tears his soul in his anger' (Iyov 18:4), making explicit that the destruction is first and foremost self-directed.
Sefer Charedim, Positive Commandments 1:39 draws on the Zohar to add a spiritual dimension: when a person becomes angry, the holy soul departs and an evil spirit takes its place, so that the angry person effectively becomes an idolatrous altar within himself, and Shenei Luchot HaBerit, Shaar HaOtiyot, Ratzon:1 similarly cites the teaching that anger uproots all three levels of holiness — nefesh, ruach, and neshamah — simultaneously.
Chafetz Chaim, Introduction to the Laws of Lashon Hara acknowledges in passing that anger, together with other corrupt traits, corrupts and destroys the form and majesty of the soul — even though no explicit negative commandment of the 613 is stated against it — confirming that the mussar tradition consistently frames anger as an internal corrosive force on the person himself.