Objective: To examine the cross-sectional and inter-temporal validity of the Household Food Insecurity Access Scale (HFIAS) for rural households in Burundi. Design: Longitudinal survey about food security and agricultural production, individually administered by trained interviewers in June 2007 and 2012. Setting: Ngozi, north of Burundi. Subjects: Three hundred and fourteen household heads were interviewed. Results: Tobit models showed that the HFIAS was significantly correlated with objective measures of food security, in this case total annual food production (P < 0·01), livestock keeping (P < 0·01) and coffee production (P < 0·01) in both 2007 and 2012. This confirms that the HFIAS is cross-sectionally valid and corroborates the findings of previous studies. However, while total food production decreased by more than 25 % in terms of energy between 2007 and 2012, households reported an improvement in their perceived food security over the same period, with the HFIAS decreasing from 13·9 to 10·8 (P < 0·001). This finding questions the inter-temporal validity of the HFIAS. It may be partly explained through response shifts, in which households assess their own food security status in comparison to that of their peers. Conclusions: The evidence from our study suggests that the HFIAS is crosssectionally valid, but may not be inter-temporally valid, and should not be used as a single indicator to study temporal trends in food security.