Using machine learning to analyze longitudinal data: A tutorial guide and best‐practice recommendations for social science researchers

Abstract

This article introduces the research community to the power of machine learning over traditional approaches when analyzing longitudinal data. Although traditional approaches work well with small to medium datasets, machine learning models are more appropriate as the available data becomes larger and more complex. Additionally, machine learning methods are ideal for analyzing longitudinal data because they do not make any assumptions about the distribution of the dependent and independent variables or the homogeneity of the underlying population. They can also analyze cases with partial information. In this article, we use the Household, Income, and Labour Dynamics in Australia (HILDA) survey to illustrate the benefits of machine learning. Using a machine learning algorithm, we analyze the relationship between job-related variables and neuroticism across 13 years of the HILDA survey. We suggest that the results produced by machine learning can be used to generate generalizable rules from the data to augment our theoretical understanding of the domain. With a technical guide, this article offers critical information and best-practice recommendations that can assist social science researchers in conducting machine learning analysis with longitudinal data.

Artificial intelligence for dementia prevention

“…For example, hypertension and obesity are both associated with increased future dementia risk during mid‐life, but weight and blood pressure have been shown to decrease in later life in those with or developing dementia, indicating that in later life, changes in these risk factors are consequences of disease progression 64,65 . ML methods that consider the changing importance of risk factors using longitudinal data could be used to understand different disease trajectories and disease heterogeneity 66 …”