The pasuk in Devarim 20:19 — "כי האדם עץ השדה" — supplies the textual anchor for the entire comparison, and Netzach Yisrael (Netzach Yisrael:7) builds directly on it, explaining that man is an inverted tree: whereas an ordinary tree is rooted below in the earth because it draws its life from the ground, the human being is rooted above because his soul — his true root — comes from the heavens, with his body serving as the trunk and his limbs as branches.
Chazal develop the comparison as a moral measure: Pirkei Avot 3:17 teaches that one whose deeds exceed his wisdom resembles a tree with few branches but many roots, unmoved by any wind, while one whose wisdom exceeds his deeds resembles a tree with many branches but shallow roots, easily uprooted — and Avot DeRabbi Natan 22:1 applies the same contrast specifically to Torah study joined with good deeds versus Torah study without them.
The anchor image drawn upon by both Pirkei Avot and Avot DeRabbi Natan is Tehillim 1:3, which portrays the person of integrity as "a tree planted beside streams of water, which yields its fruit in season, whose foliage never fades," and מלבי"ם, הכרמל explains why the metaphor uses a tree (עץ) rather than grass: the tree endures for many years, blossoms, and bears seed year after year, whereas grass springs up briefly and perishes — so the perfect person is likened to a tree whose roots are planted above and whose branches extend downward, the brain being the root from which sensation flows to the limbs as branches.
Kiddushin 40b extends the analogy into the realm of reward and suffering, comparing the righteous to a tree rooted entirely in a pure place whose branches alone extend into impurity — when those branches are cut the whole tree stands pure — thereby using the tree's structure to illuminate how present affliction purifies the righteous for the World-to-Come.