The earliest statement of the rule appears in Mishnah Sanhedrin 3:3, which lists the dice-gambler (משחק בקוביא) among those categorically disqualified from serving as a witness or judge.
The Rambam (Mishneh Torah, Testimony 10:4) explains the disqualification: a gambler who has no other trade is presumed to subsist on winnings that carry the taint of near-theft (אבק גזל), because he is not engaged in the productive settlement of the world.
The Tur (Choshen Mishpat 34:24) extends this reasoning further, noting that such a person — having no other livelihood — does not appreciate how hard people labor for money, and is therefore liable to testify falsely in order to deprive another of his funds; if he does have another trade, he remains fit to testify.
The Shulchan Arukh (Choshen Mishpat 34:16) broadens the category beyond dice to any game in which the winner takes from the loser — including games played with nutshells or with animals — provided gambling is the person's sole occupation.
The Arukh HaShulchan (Choshen Mishpat 34) adds a further nuance in the name of the Rema: even for rabbinic prohibitions generally, disqualification as a witness requires that the transgression involve the desire for monetary gain, since it is specifically the willingness to sin for money that raises the suspicion one would also lie for money.