In this critical review, we examine the application of predictive models, e.g. classifiers, trained using Machine Learning (ML) to assist in interpretation of functional neuroimaging data. Our primary goal is to summarize how ML is being applied and critically assess common practices. Our review covers 250 studies published using ML and resting-state functional MRI (fMRI) to infer various dimensions of the human functional connectome. Results for hold-out (“lockbox”) performance was, on average, ~13% less accurate than performance measured through cross-validation alone, highlighting the importance of lockbox data which was included in only 16% of the studies. There was also a concerning lack of transparency across the key steps in training and evaluating predictive models. The summary of this literature underscores the importance of the use of a lockbox and highlights several methodological pitfalls that can be addressed by the imaging community. We argue that, ideally, studies are motivated both by the reproducibility and generalizability of findings as well as the potential clinical significance of the insights. We offer recommendations for principled integration of machine learning into the clinical neurosciences with the goal of advancing imaging biomarkers of brain disorders, understanding causative determinants for health risks, and parsing heterogeneous patient outcomes.

Background: Neuropsychiatric symptoms are common following traumatic brain injury (TBI), but their etiological onset remains unclear. Mental health research implicates neuroinflammation in the development of psychiatric disorders. The presence of neuroinflammatory responses after TBI thus prompts an investigation of their involvement in the emergence of neuropsychiatric disorders postinjury. Objective: Review the literature surrounding the role of neuroinflammation and immune response post-TBI in the development of neuropsychiatric disorders. Methods: A search of scientific databases was conducted for original, empirical studies in human subjects. Key words such as "neuroinflammation," "TBI," and "depression" were used to identify psychopathology as an outcome TBI and the relation to neuroinflammatory response. Results: Study results provide evidence of neuroinflammation mediated post-TBI neuropsychiatric disorders including anxiety, trauma/stress, and depression. Inflammatory processes and stress response dysregulation can lead to secondary cell damage, which promote the development and maintenance of neuropsychiatric disorders postinjury. Conclusion: This review identifies both theoretical and empirical support for neuroinflammatory response as feasible mechanisms underlying neuropsychiatric disorders after TBI. Further understanding of these processes in this context has significant clinical implications for guiding the development of novel treatments to reduce psychiatric symptoms postinjury. Future directions to address current limitations in the literature are discussed.

The replication crisis poses important challenges to modern science. Central to this challenge is re-establishing ground truths, or the most fundamental theories that serve as the bedrock to a scientific community. However, the goal to identify hypotheses with the greatest support is non-trivial given the unprecedented rate of scientific publishing. In this era of high-volume science, the goal of this study is to sample from one research community within clinical neuroscience (traumatic brain injury) and track major trends that have shaped this literature over the past 50 years. To do so, we first conduct decade-wise (1980-2019) network analysis to examine the scientific communities that shape this literature. To establish the robustness of our findings, we utilized searches from separate search engines (Web of Science; Semantic Scholar). As a second goal, we sought to determine the most highly cited hypotheses influencing the literature in each decade. In a third goal, we then searched for any papers referring to “replication” or efforts to reproduce findings within our >50,000 paper dataset. From this search, 550 papers were analyzed to determine the frequency and nature of formal replication studies over time. Finally, to maximize transparency, we provide a detailed procedure for the creation and analysis of our dataset, including a discussion of each of our major decision points, to facilitate similar efforts in other areas of neuroscience. We found that the unparalleled rate of scientific publishing within the brain injury literature combined with the scarcity of clear hypotheses in individual publications are a challenge to both evaluating accepted findings and determining paths forward to accelerate science. Additionally, while the conversation about reproducibility has increased over the past decade, the rate of published replication studies continues to be a negligible proportion of the research. Meta-science and computational methods offer the critical opportunity to assess the state of the science and illuminate pathways forward, but ultimately there is structural change needed in the brain injury literature and perhaps others.

Machine learning offers a promising set of prediction tools that have enjoyed more recent application in network neuroscience. Computer algorithms hold the potential to uncover hidden patterns and guide scientists and practitioners alike. In this NETN Perspectives, we examine the current application of predictive models, e.g., classifiers trained using machine learning (ML), within the clinical network neurosciences. Our primary goal is to summarize how ML is being applied and critically assess the most common practices. Our review covers 118 studies published using ML and functional MRI (fMRI) to infer various dimensions of the human functional connectome. We identify several important methodological challenges in this literature. More than half of the studies focused almost exclusively on maximizing the accuracy of classifying brain functional connectomes into one of several predetermined categories (e.g., disease versus healthy), with significantly less emphasis on offering insights into the how features of the functional connectome combine to orchestrate complex brain functions. There was also a concerning lack of transparency across many of the key steps in training and evaluating predictive models using machine learning. The summary of this literature underscores the importance of external validation (i.e., lockbox or test-set data) and highlights several methodological pitfalls that can be addressed by the imaging community. We argue that ideally studies can be motivated by the reproducibility and generalizability of the findings, and the potential clinical significance of the insights. We offer recommendations that might move the community toward a more principled integration of machine learning into clinical neuroscience with the goal of advancing imaging biomarkers of brain disorders, understanding causative determinants for health risks and parsing heterogeneous patient outcomes.