The Kli Yakar interprets 'ben zekunim' as indicating that Yosef had the mind of an elder even in youth — i.e., wisdom beyond his years — and this intellectual and spiritual precocity was the basis of Yaakov's special love.
וְיִשְׂרָאֵל אָהַב אֶת יוֹסֵף מִכָּל בָּנָיו כִּי בֶן זְקֻנִים הוּא לוֹ. אַף עַל פִּי שֶׁנֶּאֱמַר (בראשית לז:ב) ״וְהוּא נַעַר אֶת בְּנֵי בִלְהָה״, שֶׁהָיָה עוֹשֶׂה מַעֲשֵׂה נַעֲרוּת כְּמוֹתָם, מִכָּל מָקוֹם אֲהֵבוֹ יַעֲקֹב, לְפִי שֶׁבִּהְיוֹתוֹ אֶצְלוֹ הָיָה מִתְנַהֵג אֶת עַצְמוֹ כְּזָקֵן וְרָגִיל וּפִרְקוֹ נָאֶה, וְלֹא יָדַע הַנְהָגָתוֹ שֶׁעִם אֶחָיו. עַל כֵּן אָמַר ״כִּי בֶן זְקֻנִים הוּא לוֹ״, בְּמִלַּת ״לוֹ״ הוֹרָה שֶׁדַּוְקָא לוֹ הָיָה בֶּן זְקֻנִים וְלֹא לְאֶחָיו, לְפִי שֶׁבִּהְיוֹתוֹ אֶצְלָם עָשָׂה מַעֲשֵׂה נַעֲרוּת כְּמוֹתָם, וּבִהְיוֹתוֹ אֵצֶל אָבִיו הַזָּקֵן הָיָה עוֹשֶׂה מַעֲשֵׂה זִקְנָה. וְאוּלַי לֹא רָצָה לְשַׁנּוֹת מִן הַמִּנְהָג וְיָדַע לְהִתְנַהֵג עִם הַנְּעָרִים וְעִם הַזְּקֵנִים, לְכָךְ נֶאֱמַר ״הוּא לוֹ״. יֵשׁ אוֹמְרִים, שֶׁכָּל מַה שֶּׁלָּמַד מִן שֵׁם וָעֵבֶר מָסַר לוֹ. וְלָמָּה לֹא מָסַר לְכָל בָּנָיו מַה שֶּׁלָּמַד, אֶלָּא וַדַּאי לְפִי שֶׁכָּל בָּנָיו מָאֲסוּ בַּחָכְמוֹת וְלֹא נִתְחַבְּרוּ אֶל יַעֲקֹב לִשְׁמֹעַ מִמֶּנּוּ, אֲבָל יוֹסֵף ״בֶּן זְקֻנִים הוּא לוֹ״ שֶׁתָּמִיד הוּא פָּנָה לוֹ לְיַעֲקֹב וְנִכְסַף לִשְׁמֹעַ מִפִּיו מַה שֶּׁלָּמַד מִשֵּׁם וָעֵבֶר, עַל כֵּן אֲהֵבוֹ. וְתַרְגּוּמוֹ ״בַּר חַכִּים הוּא לוֹ״, כִּי בִּישִׁישִׁים חָכְמָה. זֶה שֶׁאָמְרוּ (בראשית רבה פד ח) שֶׁהָיָה זִיו אִיקוֹנִין דּוֹמֶה לוֹ, כִּי ״חָכְמַת אָדָם תָּאִיר פָּנָיו״ (קהלת ח יג). וְיֵשׁ רֶמֶז כָּאן לְמַה שֶׁאָמְרוּ רַבּוֹתֵינוּ זִכְרוֹנָם לִבְרָכָה (קידושין מ:) נַעֲנוּ כֻּלָּם וְאָמְרוּ תַּלְמוּד גָּדוֹל שֶׁמֵּבִיא לִידֵי מַעֲשֶׂה, כָּךְ הִזְכִּיר כָּאן שְׁלֵמוּת שֶׁל יוֹסֵף בַּפָּסוּק ״הוּא הָיָה רֹעֶה אֶת אֶחָיו בַּצֹּאן״ (בראשית לז ב), לְמַעֵט כָּל מִינֵי רְעִיָּה הָאֲסוּרִים, וְאַף עַל פִּי כֵן לֹא אֲהֵבוֹ כִּי אִם מִצַּד ״כִּי בֶן זְקֻנִים הוּא לוֹ״, בַּר חַכִּים, כִּי גָדוֹל הַתַּלְמוּד כָּאָמוּר.
And Israel loved Joseph more than all his sons because he was a child of his old age to him. Even though it is said and he was a youth with the sons of Bilhah — meaning he would act youthfully like them — nevertheless Jacob loved him because when Joseph was with him, he would conduct himself like an elder, was well-practiced [in proper behavior], and had a pleasant youth. Jacob did not know how Joseph conducted himself with his brothers. Therefore it says because he was a child of his old age to him — the word to him indicates that specifically for Jacob he was a child of old age, but not for his brothers. This is because when Joseph was with them, he would act youthfully like them, but when he was with his elderly father, he would act with elderly maturity. Perhaps he did not want to deviate from custom and knew how to conduct himself appropriately both with youths and with elders. This is why it specifically states he was to him. Some say that [Jacob] transmitted to [Joseph] everything he had learned from Shem and Eber. And why didn’t he transmit what he learned to all his sons? Rather, certainly because all his sons rejected wisdom and did not connect with Jacob to learn from him. But Joseph was a son of his old age to him, as he always turned to Jacob and yearned to hear from him what he had learned from Shem and Eber. Therefore he loved him. And [Onkelos] translates [ben zekunim] as “a wise son to him,” for wisdom is found in the elderly. This is what they meant when they said that his facial features resembled him, for a person’s wisdom illuminates their face. And there is a hint here to what our Sages said, “They all responded and said: Study is greater because it leads to action.” Similarly, here it mentions Joseph’s perfection in the verse he was shepherding the flock with his brothers, excluding all forbidden types of shepherding, and nevertheless [Jacob] only loved him because he was a son of his old age — a wise son — for study is greater, as mentioned.