Since Easterlin pointed out that economic growth in nations does not guarantee increasing happiness for the average citizen, the underlying reason has remained controversial. The present study focuses on income inequality to explain the “Easterlin Paradox,” ignoring the permanent inequality that long-term wealth accumulation brings. Based on social comparison theory, the literature aims to determine how wealth inequality, which accompanies economic growth, diminishes one’s happiness (inequality aversion). Specifically, we conduct this study in which we split the wealth inequality into the upward wealth inequality and the downward wealth inequality as measures of upward comparison and downward comparison, respectively. The upward wealth inequality measures the average gap between one and the better-off in wealth while the downward wealth inequality measures the average gap between one and the worse-off in wealth. Furthermore, the heterogeneity of the area of respondent is analyzed and the family life cycle is tested as a moderator. The main findings of the paper are as follows: (1) The empirical test results of hypothesis 1 indicate that the upward wealth inequality aversion (jealousy effect: people envy who is richer than themselves) is stronger than the downward wealth inequality inclination (proud effect: people enjoy having a superior position in the wealth hierarchy). It is due to the psychological preference: loss aversion. As an increase in upward distance implies a loss in relative status and an increase in downward distance implies a gain in relative status, people focus more on loss rather than gain. (2) The empirical test results of hypothesis 2 indicate that residents who live in rural areas do not have a proud effect as much as those who live in urban areas. There is a huge urban-rural wealth gap in China. With the expansion of the social network, people living in rural areas realize that even he is almost the rich in rural areas but still the lower classes in the whole society. It is hard for rural residents to have a proud effect. (3) The empirical test results of hypothesis 3 indicate that family members have the strongest upward inequality aversion in the middle-stage phase of the life cycle (when the family head is approximately 50). During the family life cycle, inequality aversion will be different in different life stages due to the changes in economic status expectations. At the beginning of the family life cycle, family members assume their life has limitless possibilities, and they have high expectations for the future. Logically, they can be easily satisfied by achieving a little more than their peers. In later periods, with increasing age, the members will pay more attention to health instead of wealth. The results shed light on how macroeconomics influence changes in individual psychology.