The study of functional brain networks has grown rapidly over the past decade. While most functional connectivity (FC) analyses estimate one static network structure for the entire length of the functional magnetic resonance imaging (fMRI) time series, recently there has been increased interest in studying time-varying changes in FC. Hidden Markov models (HMMs) have proven to be a useful modeling approach for discovering repeating graphs of interacting brain regions (brain states). However, a limitation lies in HMMs assuming that the sojourn time, the number of consecutive time points in a state, is geometrically distributed. This may encourage inaccurate estimation of the time spent in a state before switching to another state. We propose a hidden semi-Markov model (HSMM) approach for inferring timevarying brain networks from fMRI data, which explicitly models the sojourn distribution. Specifically, we propose using HSMMs to find each subject's most probable series of network states and the graphs associated with each state, while properly estimating and modeling the sojourn distribution for each state. We perform a simulation study, as well as an analysis on both task-based fMRI data from an anxiety-inducing experiment and resting-state fMRI data from the Human Connectome Project. Our results demonstrate the importance of model choice when estimating sojourn times and reveal their potential for understanding healthy and diseased brain mechanisms.Interest in studying brain networks, where the brain is viewed as a system 2 of interacting regions (nodes) that produce complex behaviors, has grown 3 tremendously over the past decade. Networks have become a popular ap-4 proach towards illustrating both the physiological connections (structural 5 networks) and the coupling of dynamic brain activity (functional networks) 6 linking different areas of the brain. It is within this paradigm shift that sci-7 entists have begun investigating how networks behave in healthy brains and 8 how they are altered in neurological and psychiatric disorders [1]. 9 The study of functional connectivity (FC), or the undirected association 10 between two or more fMRI time series, has come to the forefront of research 11 efforts in the field of neuroimaging. Studies have revealed information about 12 the functional connections between different brain regions and local networks 13 and have provided significant insights into the organization of functional com-14 munication in the brain. Brain networks can be created by performing a FC 15 analysis, where the strength of the relationship between nodes is assessed 16 using the functional magnetic resonance imaging (fMRI) time series associ-17 ated with each node. The nodes can consist of individual voxels, [2, 3, 4], 18 pre-specified regions of interest (ROIs) [5], or sets of regions estimated using 19 independent component analysis (ICA) [6]. The relationship between nodes 20 can be quantified using a variety of metrics, and we present the networks in 21 this paper using both Pea...