The Alter Rebbe teaches that worldly pleasures, even when permitted, engage the 'animal soul' and occupy the heart; a person who minimizes indulgence in them frees the divine soul to cleave to God more completely.
אַךְ נֶפֶשׁ הַחִיּוּנִית הַבַּהֲמִית שֶׁבְּיִשְׂרָאֵל שֶׁמִּצַּד הַקְּלִיפָּה, הַמְלוּבֶּשֶׁת בְּדַם הָאָדָם כַּנִּזְכָּר לְעֵיל, וְנַפְשׁוֹת בְּהֵמוֹת וְחַיּוֹת וְעוֹפוֹת וְדָגִים טְהוֹרִים וּמוּתָּרִים בַּאֲכִילָה, וְקִיּוּם וְחַיּוּת כָּל הַדּוֹמֵם וְכָל הַצּוֹמֵחַ הַמּוּתָּר בַּאֲכִילָה, וְכֵן קִיּוּם וְחַיּוּת כָּל הַמַּעֲשֶׂה דִּבּוּר וּמַחֲשָׁבָה בְּעִנְיְינֵי עוֹלָם הַזֶּה שֶׁאֵין בָּהֶם צַד אִיסּוּר, לֹא שֹׁרֶשׁ וְלֹא עָנָף מִשַּׁסַ״ה מִצְוֹת לֹא תַעֲשֶׂה וְעַנְפֵיהֶן דְּאוֹרַיְיתָא וּדְרַבָּנָן, רַק שֶׁאֵינָן לְשֵׁם שָׁמַיִם – אֶלָּא רְצוֹן הַגּוּף וְחֶפְצוֹ וְתַאֲוָתוֹ, וַאֲפִילוּ הוּא צוֹרֶךְ הַגּוּף וְקִיּוּמוֹ וְחַיּוּתוֹ מַמָּשׁ, אֶלָּא שֶׁכַּוָּונָתוֹ אֵינָהּ לְשֵׁם שָׁמַיִם כְּדֵי לַעֲבוֹד אֶת ה׳ בְּגוּפוֹ – לָא עֲדִיפֵי מַעֲשֶׂה דִּבּוּר וּמַחֲשָׁבוֹת אֵלּוּ מִנֶּפֶשׁ הַחִיּוּנִית הַבַּהֲמִית בְּעַצְמָהּ, וְהַכֹּל כַּאֲשֶׁר לַכֹּל, נִשְׁפָּע וְנִמְשָׁךְ מִמַּדְרֵגָה הַשֵּׁנִית שֶׁבִּקְלִיפּוֹת וְסִטְרָא אָחֳרָא, שֶׁהִיא קְלִיפָּה רְבִיעִית הַנִּקְרֵאת ״קְלִיפַּת נוֹגַהּ״, שֶׁבָּעוֹלָם הַזֶּה, הַנִּקְרָא ״עוֹלַם הָעֲשִׂיָּה״, רוּבּוֹ כְּכוּלּוֹ רָע, רַק מְעַט טוֹב מְעוֹרָב בְּתוֹכָהּ [שֶׁמִּמֶּנָּה בָּאוֹת מִדּוֹת טוֹבוֹת שֶׁבַּנֶּפֶשׁ הַבַּהֲמִית שֶׁבְּיִשְׂרָאֵל, כְּמוֹ שֶׁנִּתְבָּאֵר לְעֵיל].
On the other hand, the vitalizing animal soul in the Jew, that which is derived from the aspect of the kelipah, which is clothed in the human blood, as stated above, and the “souls” of the animals, beasts, birds, and fish that are clean and fit for [Jewish] consumption, as also the existence and vitality of the entire inanimate and entire vegetable world which are permissible for consumption, as well as the existence and vitality of every act, utterance, and thought in mundane matters that contain no forbidden aspect—being neither root nor branch of the 365 prohibitive precepts and their offshoots, either on the explicit authority of the Torah or by Rabbinic enactment—yet are not performed for the sake of Heaven but only by the will, desire, and lust of the body; and even where it is a need of the body, or its very preservation and life, but his intention is not for the sake of Heaven, that is, to serve G–d thereby —all these acts, utterances, and thoughts are no better than the vitalizing animal soul itself; and everything in this totality of things flows and is drawn from the second gradation [to be found] in the kelipot and sitra achara, namely, a fourth kelipah, called kelipat nogah. In this world, called the “World of Asiyah (Action),” most, indeed almost all, of it [the kelipat nogah] is bad, and only a little good has been intermingled within it [from which come the good qualities contained in the animal soul of the Jew, as is explained above.]