In the current era, the rapid spread of Internet technology has combined with various traditional industries; this provides new research perspectives and solutions for current problems, such as those in the elderly care industry. Elderly health is an important social problem in various countries, and governments have turned to the internet for new methods and better solutions. However, internet-use behavior has a certain influence on the elderly’s health status. This study investigates the effects of internet use on the elderly’s physical health, mental health, and self-rated health, along with the moderating role of individual cognitive ability in the above relationship. This study uses data from the Chinese General Social Survey (CGSS) in 2012 and 2015 as samples for analysis via the hierarchical regression method. The sample is from China and had 2821 and 3185 valid respondents in 2012 and 2015, respectively. Results show that internet use significantly affects the physical and mental health of the elderly and does not significantly affect self-rated health. In addition, individual cognitive ability plays a negative moderating role between internet use and physical and mental health. Finally, on the basis of results analysis and discussion, this study provides new recommendations to achieve targeted health improvements.