The Torah itself provides the foundation: Bamidbar 15:38-40 commands Israel to place tzitzit on the corners of their garments throughout their generations so that they will remember all the commandments, and Devarim 22:12 similarly enjoins making twisted cords on the four corners of one's garment.
Ibn Ezra (Bamidbar 15:27) reflects interestingly on the practice, observing that those who pray in a tallit do so because they will be reciting the Shema passage 'and it shall be for you tzitzit' — yet he notes that strictly speaking, a person is arguably more obligated to wear tzitzit during the rest of the day than at prayer time specifically, since it is precisely during non-prayer hours that one needs the reminder not to sin.
Mishneh Torah, Hilkhot Tzitzit 3:9-12 establishes the normative basis for the custom: while no one is technically required to acquire a four-cornered garment, it is unbecoming for a pious person to exempt himself from tzitzit entirely, and one must take special care to be wrapped in a tzitzit garment during prayer — so much so that it is a great disgrace for a Torah scholar to pray without being wrapped.
Shulchan Arukh, Orach Chayim 91:3-6 codifies this as the way of Torah scholars and their students, who pray only while wrapped; the Rema adds that at prayer one should adorn oneself in fine garments, analogous to a servant standing before his master, and Mishneh Torah, Prayer and the Priestly Blessing 5:1-5 similarly requires that one arrange and beautify one's clothing before prayer, as the verse says 'bow to God in resplendent holiness.'
Several Acharonim explain the spiritual rationale for head-covering with the tallit specifically: Kaf HaChayim (Orach Chayim 8:10) records that covering one's head with the tallit during prayer is meant to induce awe, since such covering humbles the heart and brings a person to fear of Heaven, and Mishnah Berurah 8:1-4 states this same reason explicitly, adding that the Bach requires the tallit to remain on one's head from the beginning of prayer to its end.
Derekh Hashem, Part Four, On the Daily Order of Prayer 6 offers a deeper rationale: beyond tzitzit being a perpetual commandment, the Sages fashioned wrapping in a tallit for prayer as one of the formal enactments of the prayer service, and the meaning of the wrapping is the acceptance of the Divine yoke — taking hold of God's work of repairing the world.
Sefat Emet, Shelach adds that the act of wrapping in a tallit represents the gathering of all of a person's inner forces into one place, with the tzitzit at the corners serving as the point of connection that keeps the entire garment — and by analogy, the entire person — from dispersing, drawing all things inward toward their root.
Arukh HaShulchan, Orach Chayim 8:1-5 reflects the practical settled custom: all of Israel wears a tallit katan throughout the day, and a tallit gadol specifically at the time of prayer, and Kitzur Shulchan Arukh Yalkut Yosef, Orach Chayim 8:3 adds that it is proper to use a tallit gadol rather than relying on the tallit katan alone, partly because the tallit katan sometimes does not meet the required size for the mitzvah.
Kitzur Shulchan Arukh Yalkut Yosef, Orach Chayim 17:4 records that among Sephardim and Edot HaMizrach, the widespread custom is for young men who have reached the age of mitzvot to wrap in a tallit gadol at Shacharit even if they already wear a tallit katan underneath, and even if they are not yet married.
Mishneh Halachos (Mishneh Halachos 15:23) grounds the cantor's wrapping in a tallit in a Talmudic image of the Holy One Himself, as it were, wrapping like a cantor and showing Moshe the order of forgiveness, and further derives from the principle of kavod tzibur that one should not stand before the congregation without a tallit, though some communities were lenient when praying without a minyan.
