Lifelogging memory appliance for people with episodic memory impairment

Abstract

Lifelogging technologies have the potential to provide memory cues for people who struggle with episodic memory impairment (EMI). These memory cues enable the recollection of significant experiences, which is important for people with EMI to regain a sense of normalcy in their lives. However, lifelogging technologies often collect an overwhelmingly large amount of data to review. The best memory cues need to be extracted and presented in a way that best supports episodic recollection. We describe the design of a new lifelogging system that captures photos, ambient audio, and location information and leverages both automated content/context analysis and the expertise of family caregivers to facilitate the extraction and annotation of a salient summary consisting of good cues from the lifelog. The system presents the selected cues for review in a way that maximizes the opportunities for the person with EMI to think deeply about these cues to trigger memory recollection on his own without burdening the caregiver. We compare our system with another review system that requires the caregiver to repeatedly guide the review process. Our self-guided system resulted in better memory retention and imposed a smaller burden on the caregiver whereas the caregiver-guided approach provided more opportunities for caregiver interaction.

SenseCam: A new tool for memory rehabilitation?

“…Comparison with a written diary showed a significant improvement in the number of details provided when using SenseCam. These results confirm that the use of SenseCam in these patients is associated with an improvement in autobiographical memory (results corroborated by Lee and Dey [52] and by Crete-Nishihata et al [53]). …”

“…SenseCam should not only have a beneficial effect on the amount of information retrieved, but should also improve the quality of the information recalled, with more detail, and a richer experience for past events. An improvement in episodic memory with SenseCam has been demonstrated both in healthy populations -a group of 3 men [50], a group of 18 adults [28], a group of 14 young adults [51] -and in populations presenting with memory disorders -3 adults with AD [52], 5 adults with AD or mild cognitive impairment [53], 6 adults with AD [1].…”

SpeakerSense: Energy Efficient Unobtrusive Speaker Identification on Mobile Phones

“…Several previous prototypes have studied capturing audio to support retrospective memory assistance [e.g., 1,5,17] and have explored speaker identification using additional sensors [12] or multiple phones [7]. However, two key challenges must be addressed before continuous audio sensing for speaker identification is practical:…”

“…Kapur [4]'s survey of memory aids describes the wide variety in common use which span both technological (e.g., reminders on watch alarms, mobile phones or pillboxes) and non-technological (e.g., wall calendars, notebooks) solutions, and highlights the potential for advances in technology on mobile phones, cameras and location detection devices to provide further memory assistance. Research using the SenseCam [e.g., 2,5] has shown the potential of lifelogging to help people with memory impairments retrospectively review captured information to assist them in recalling events. Looking specifically at retrospective aids based on audio logging, Vemuri et al developed iRemember [17], a memory retrieval prototype which recorded and transcribed everyday conversations so they could be later searched.…”

The Cognitive Effects of Wearable Cameras in Mild Alzheimer Disease – An Experimental Study

“…Lee and Dey [22] tested three patients with mild AD. These patients wore the SenseCam for two weeks but only for personally significant events (as considered by the patient or its caregiver).…”