The clearest ruling comes from the Shulchan Arukh (Orach Chayim 124:3), which states that if individuals in the congregation are prolonging their prayers, the chazan need not wait for them — even if those individuals are the prominent people of the city — implying that the chazan may begin chazarat hashatz once the main body of the congregation has finished, without waiting for every last person.
The underlying requirement is that nine of the ten needed for a minyan be paying attention to the chazan's blessings: the Shulchan Arukh (Orach Chayim 124:3) rules that if nine are not attentive, the chazan's blessings are close to being recited in vain, and therefore each person should act as though he is the critical ninth.
The Mishneh Torah, Prayer and the Priestly Blessing 8:4 adds a related principle: even those who have already fulfilled their own prayer obligation can complete the minyan of ten for this purpose, provided that the majority of the ten have not yet prayed — pointing to the underlying structure that a majority of the quorum must still be in need of the repetition.
The Mishnah Berurah (124:13) records a widespread custom to wait specifically for the av beit din to finish his prayer before the chazan begins, explaining that since most people pray hurriedly and the av beit din prays word by word, waiting for him ensures the congregation can properly recite Kedushah — but notes that where there is no av beit din, one would similarly wait for whoever prays carefully word by word, though not for someone who is simply taking an unusually long time.