Background The COVID-19 pandemic continues to pose an international public health threat. Prevention is of paramount importance to protect the high-risk group of older adults until specific treatments for COVID-19 become available; however, little work has been done to explore factors that promote preventive behaviors among this population. Objective This study aims to investigate the knowledge, perceived beliefs, and preventive behaviors towards COVID-19 of older adults in China and determine the factors that influence their practice of preventive behaviors. Methods From February 19 to March 19, 2020, a cross-sectional, web-based survey was administered to Chinese older adults in all 31 provinces of mainland China using a convenience sampling method to assess the respondents’ knowledge, perceived beliefs, and preventive behaviors towards COVID-19. Standard descriptive statistics and hierarchical linear regression analyses were conducted to analyze the data. Results A total of 1501 participants responded to the survey, and 1263 valid responses (84.1%) were obtained for further analysis. The overall correct rate on the knowledge questionnaire was 87%, overall positive beliefs regarding COVID-19 were found, and the mean behavior score was 13.73/15 (SD 1.62, range 5-15). The hierarchical linear regression showed that respondents who were married or cohabitating and who lived in areas with community-level control measures were more likely to practice preventive behaviors (P<.01). Knowledge (β=0.198, P<.001), perceived susceptibility (β=0.263, P=.03), perceived benefits (β=0.643, P<.001), and self-efficacy in preventing COVID-19 (β=0.468, P<.001) were also found to be significantly associated with preventive behaviors. Conclusions Most older residents had adequate knowledge and positive beliefs regarding COVID-19 and engaged in proactive behaviors to prevent the disease. Knowledge and beliefs were confirmed to be significantly associated with behavior responses. Our findings have significant implications in enhancing the effectiveness of COVID-19 prevention programs targeting the older population; these programs must be continued and strengthened as the epidemic continues.