The root of the practice traces to Chazal: Kiddushin 31a records Rav Huna son of Rav Yehoshua refusing to walk four cubits bareheaded because the Divine Presence rests above his head, and Shabbat 118b lists this same practice among the acts for which he expected reward, alongside other exacting personal stringencies.
The Rambam (Mishneh Torah, Prayer 5:5) codifies that one must not stand in prayer with an uncovered head or uncovered legs where local practice treats this as disrespectful before dignitaries, and that the custom of the sages and their students was never to pray except while wrapped, rooting the requirement in the honor due to prayer itself rather than treating it as mere stringency.
Shulchan Arukh, Orach Chayim 91:1 and Shulchan Arukh, Orach Chayim 91:5 record a dispute among the Rishonim as to whether it is outright forbidden to mention God's name with an uncovered head or whether one should merely object to entering a synagogue bareheaded, and note that straw-woven hats count as a valid covering while merely placing one's hand on the head does not; the Beit Yosef (Orach Chayim 91:4-5) traces this same machlokes, citing the Kolbo's view that going bareheaded is only a matter of piety against Rabbeinu Yerucham's ruling that it is an outright prohibition to bless with the head uncovered, and concludes in favor of the stricter position.
Shulchan Arukh HaRav (Orach Chayim 91:5-6) adds historical color, describing how the early sages, in times of peace, adorned themselves in fine garments for prayer out of honor before God, while in times of distress they stood instead like a servant before his master, divesting themselves of mantles so as not to appear important, out of fear and trembling.
Later acharonim sharpen the law: Magen Avraham 91:3 and Arukh HaShulchan, Orach Chaim 91:5 clarify that while refraining from going bareheaded during the day generally is a matter of piety, doing so specifically during prayer is treated as actual law, not dependent on local custom, even in a place where standing bareheaded before dignitaries is normal; Mishnah Berurah 91:10-12 and Mishnah Berurah 8:4 apply this practically, requiring one to wear an actual hat during prayer as one would in the street, and ruling that a small skullcap worn beneath the hat should not be removed even when the tallit also covers the head.
The Sulam on Zohar (Sulam on Zohar, Pinchas 625) connects the law of covering the head during blessings and the Divine Name to the Talmudic teaching that a scholar may not walk four cubits bareheaded because the Divine Presence rests above him, grounding the practice in the pervasive honor owed to God's glory that fills the world.
The Divrei Yatziv (Divrei Yatziv OC 60) cites Rav Ashi and the Rosh to rule that even when one's head is covered by a small turban-like covering, one should not recite a blessing with the head otherwise uncovered, and he reads the Zohar's prohibition as referring specifically to praying without a proper hat, reinforcing the stringent reading of the law over the lenient one.
Among later poskim, Kitzur Shulchan Arukh Yalkut Yosef, Orach Chaim 91:7 permits removing one's hat during Shacharit in order to properly position the head tefillin, retaining only a skullcap, but requires keeping the hat on during Minchah's Amidah and during Maariv's Shema and prayer, when one accepts the yoke of Heaven; Kitzur Shulchan Arukh Yalkut Yosef, Orach Chaim 98:16 notes the custom in the yeshivot to wear a hat and suit jacket specifically for prayer, likening it to the honored garments of the Kohanim; and B'Mareh HaBazak Volume VI 27:2 rules that one who habitually wears a hat before respected people or in the street should likewise wear it for prayer and brachot, while one who does not normally wear a hat need not adopt it even for prayer, and that covering the head with the tallit itself suffices if preferred.
Kitzur Shulchan Arukh Yalkut Yosef, Orach Chaim 2:16 further distinguishes between the piety of wearing a large kippah covering most of the head generally, and the more exacting standard urged specifically during Shema, prayer, and Birkat HaMazon, where one should strive, where possible, to wear a hat or at least a kippah covering most of the head.