The practice of burying a man in his tallis is already treated as settled law by the Shulchan Arukh (Yoreh De'ah 351:2), which rules that one buries the dead only in a tallis that has tzitzit, with the gloss adding that the accepted practice is to bury him with tzitzit — only that one first disqualifies them or wraps one of the corners.
The Gemara in Bava Kamma 17a records that a Sefer Torah was placed on the bier and people said 'he fulfilled what is written in this,' and notes that today we bring it out but do not lay it on the bier — a distinction that underlies the treatment of sacred objects at burial, and that implicitly explains why tefillin, as a holy object one brings near the deceased during the procession but does not bury with him, follows the same principle.
The Kaf HaChayim (Orach Chaim 23:5–7) provides the positive rationale for the tallis: while everyone knows the dead are exempt from the commandments, the tzitzit are placed on him as a sign of belief in what Chazal taught — that the garment that descends with a person to the grave will rise with him, so that when he rises at the resurrection he will be clothed in a valid tallis.
ילקוט ראובני, שלח לך ועיין שם עוד quotes Rabbeinu Tam's explanation, heard from the elders of Lorraine, that tzitzit constitute testimony that one fulfilled the entire Torah — since tzitzit numerically equals 600, plus eight threads and five knots, totaling 613 — and therefore removing them from a deceased person's tallis would amount to bearing false witness; conversely, this logic supports burying him with tzitzit rather than with tefillin, whose absence from the burial is not a withdrawal of testimony in the same way.