From Kitab az-Zuhd by Imam Ahmad ibn Hanbal, and also in the Musannaf of Ibn Abi Shaybah.
When Umar ibn al-Khattab radiallahuanhu (may Allah be pleased with him) was walking in the market, he passed by a man who was supplicating, “O Allah, make us of Your ‘few’ servants! O Allah make us of Your ‘few’ servants!”
So ‘Umar said to him, “Where did you get this dua’ (supplication) from?” And the man said, “Allah in His Book says ‘And few of My servants are grateful.'(Qur’an 34:13)” So ‘Umar wept and admonished himself, “The people are more knowledgeable than you, O Umar! O Allah make us of Your ‘few’ servants.”
Sometimes when you advise someone to leave a sin, they respond with “But most people do it, it’s not just me!” But if you look for the words “most people” in the Qur’an, you will find that most people “do not know” (7:187), “do not give thanks” (2:243) and “do not believe” (11:17). And if you look for “most of them”, you will find that most of them are “defiantly disobedient” (5:59), “ignorant” (6:111), “turning away” (21:24), “do not reason” (29:23), and “do not listen” (8:21).
So be of the “few”, whom Allah says about them:
“And few of My servants are grateful.” (34:13)“
But none had believed with him, except a few.” (11:40)
“In the Gardens of Pleasure, A [large] company of the former peoples, And a few of the later peoples.” (56:12-14)
Ibn al-Qayyim (rahimahullah) said, “Go on the path of truth and do not feel lonely because there are few who take that path, and beware of the path of falsehood and do not be deceived by the greatness of the perishers.”