Neurocognitive mechanisms of real‐world autobiographical ...rissmanlab.psych.ucla.edu/rissmanlab/Publications... · autobiographical memory retrieval: insights from studies using

Ann. N.Y. Acad. Sci. ISSN 0077-8923

ANNALS OF THE NEW YORK ACADEMY OF SCIENCESIssue: The Year in Cognitive NeuroscienceREVIEW

Neurocognitive mechanisms of real-worldautobiographical memory retrieval: insights from studiesusing wearable camera technology

Tiffany E. Chow1 and Jesse Rissman1,2,3,4

1Department of Psychology, 2Department of Psychiatry and Biobehavioral Sciences, 3Brain Research Institute,and 4Integrative Center for Learning and Memory, University of California Los Angeles, Los Angeles, California

Address for correspondence: Jesse Rissman, Department of Psychology, University of California Los Angeles, 1285 FranzHall, Box 951563, Los Angeles, CA 90095-1563. [email protected].

In recent years, investigation into the cognitive and neural mechanisms of autobiographical memory has been aided bythe use of experimental paradigms incorporating wearable camera technology. By effortlessly capturing first-personimages of one’s life events, these cameras provide a rich set of naturalistic stimuli that can later be used to triggerthe recall of specific episodes. Here, we chronicle the development and progression of such studies in behavioraland neuroimaging examinations of both clinical and nonclinical adult populations. Experiments examining theeffects of periodic review of first-person images of life events have documented enhancements of autobiographicalmemory retrieval. Such benefits are most pronounced in patients with memory impairments, but there is mountingevidence that cognitively healthy individuals may benefit as well. Findings from functional magnetic resonanceimaging experiments using wearable camera stimuli as retrieval probes have produced results that, although largelyconsistent with the broader episodic memory literature, have significantly extended prior findings concerning theunderlying mnemonic processes and the neural representation of autobiographical information. Taken together,wearable camera technology provides a unique opportunity for studies of autobiographical memory to more closelyapproximate real-world conditions, thus offering enhanced ecological validity and opening up new avenues forexperimental work.

Keywords: episodic memory; recollection; life-logging; SenseCam; fMRI

Introduction

The ability to recollect detailed information aboutpast events is a hallmark of episodic memory.1

The vast majority of behavioral and neuroimagingstudies of episodic retrieval have used laboratory-encoded stimuli, such as words or pictures, asmemory probes. While such stimuli provideresearchers with tight experimental control over theperceptual qualities, exposure duration, and reten-tion interval of the events being probed, laboratorystimuli lack the richness of most real-world expe-riences. When events are encoded in a naturalisticsetting, it is more likely that the details will havepersonal relevance, including information aboutthe visuospatial context (event location), temporalcontext (timing of the event along with its relation

to other life occurrences), cognitive context (whatone was thinking about and/or trying to accomplishat the time), social context (who one was with), andemotional context (how one was feeling). Thus, itis perhaps unsurprising that neuroimaging studiescomparing the profile of brain activity during theretrieval of stimuli learned in a laboratory contextto that associated with the retrieval of autobio-graphical memories (i.e., memories for one’s ownlife events) typically find marked differences. Forinstance, autobiographical memory retrieval evokesmuch greater activation of default mode networkregions implicated in introspective cognition andself-referential processing—such as the medial pre-frontal cortex (mPFC)—as well as medial temporallobe (MTL) regions associated with recollection of

doi: 10.1111/nyas.13353202 Ann. N.Y. Acad. Sci. 1396 (2017) 202–221 C⃝ 2017 New York Academy of Sciences.

Chow & Rissman Wearable cameras and autobiographical memory

visuospatial contextual details, such as the hippo-campus and parahippocampal cortex.2–4 Indeed,a meta-analysis of functional magnetic resonanceimaging (fMRI) studies of episodic retrieval revealedonly limited anatomical overlap in the neural cor-relates associated with the retrieval of laboratory-encoded and autobiographical memories.5

Moreover, performance on standard laboratory-based memory tasks can be largely unrelated toone’s autobiographical retrieval abilities, as demon-strated by individuals with either “highly superiorautobiographical memory” or “severely deficientautobiographical memory.”6–9 Dissociations likethese have led some to propose that retrievingautobiographical event knowledge is fundamentallydifferent from other forms of episodic retrieval.3,10

Efforts to understand the neural mechanisms ofreal-world autobiographical memory retrieval haveutilized a variety of experimental techniques toevoke recall. These include prospective methods,which document life events as they occur and thusallow for increased experimental control.11 Suchstudies have benefited from the use of naturalisticstimuli, particularly photographs, to probe partic-ipants’ memories.2,12 While photographs can serveas effective retrieval cues that allow individuals torecollect experiences, the use of handheld cam-eras in autobiographical memory retrieval researchpresents potential methodological concerns as well.This is primarily due to participant involvementin the act of documenting personal events, whichmay result in modifications or biases in the resul-tant memories.13 This same limitation applies tostudies in which written diary entries14,15 or voicerecordings16,17 are used to log daily experiences.However, recent technological advancements havefacilitated the development of camera-based mem-ory paradigms that avoid the need for participants’explicit input. Namely, studies have begun to incor-porate the use of wearable digital camera devicesto automatically capture images of the wearer’slife events, which can later be used as probes toassess behavioral and neural processes related to theretrieval of these real-world autobiographical mem-ories. This novel, nonintrusive approach providesobjective measures of autobiographical details andoccurrences while increasing the ecological validityof experimental tasks.

The goal of this review is to summarize and eval-uate the growing set of behavioral and fMRI studies

published in peer-reviewed journals that have incor-porated wearable cameras as a tool to assess memo-ries encoded in naturalistic contexts. This includesdetailing how such camera devices have been used inexperimental paradigms on clinical and nonclinicaladult populations, with an emphasis on what thiswork has revealed about the mechanisms of autobi-ographical memory retrieval.

Wearable digital cameras

The first notable wearable camera device to beadopted by memory researchers was the SenseCamfrom Microsoft Research Cambridge (http://research.microsoft.com/en-us/um/cambridge/projects/sensecam), developed in 2003 as a tool forkeeping a visual record of one’s life experienceswithout the need for user intervention.18 TheSenseCam is most commonly worn on a lanyardaround the neck and automatically takes rela-tively low-resolution (0.3 megapixel), wide-anglephotographs from the wearer’s perspective every30 s (although the user can configure this intervalto be shorter or longer). Moreover, the camerawill capture additional photographs when itselectronic sensors detect salient variations in thewearer’s external environment, including changesin ambient temperature, light intensity and color,infrared (to detect body heat), and acceleration.The SenseCam was designed to operate for longperiods of time without recharging or uploadingphotographs to the computer for review andstorage. These characteristics allow the SenseCamto unobtrusively capture a large number of time-stamped images of its wearer’s life events, providinga wealth of content that researchers can use inautobiographical memory experiments.

A commercial version of the SenseCam, marketedas the Vicon Revue R⃝, became available in 2010owing to increasing public interest in life-loggingdevices,19 and the technology was later licensed toOMG Life, who released a higher-resolution andglobal positioning system (GPS)-enabled wearablecamera product in 2012 called the Autographer R⃝

(Fig. 1). While memory researchers have bene-fitted from these newer iterations of the originalSenseCam,20–22 these products have struggled toachieve commercial success, and, as of 2016, all man-ufacturing and sales operations have ceased. Givenrecent technological advances, the market for life-logging devices has since shifted toward wearable

203Ann. N.Y. Acad. Sci. 1396 (2017) 202–221 C⃝ 2017 New York Academy of Sciences.

http://research.microsoft.com/en-us/um/cambridge/projects/sensecam



Wearable cameras and autobiographical memory Chow & Rissman

Figure 1. The SenseCam wearable camera device and its commercial successors, the Vicon Revue and the Autographer, withexample photographs from each product. Images adapted from Microsoft Research, Vicon Motion Systems Ltd., and OMG Life.SenseCam photographs provided courtesy of Peggy St. Jacques; Vicon Revue and Autographer photographs provided by the authors.

video cameras, such as the GoPro HERO R⃝, Nar-rative Clip R⃝, MeCam HD R⃝, iON SnapCam R⃝, andSnapchat Spectacles R⃝. To our knowledge, no sci-entific studies of human memory have yet incor-porated these latest video-enabled devices, but weanticipate that they will soon become a valuableresearch tool. Additionally, several memory studieshave used necklace-mounted smartphones to docu-ment the lives of research participants.23,24 Althoughthe overall number of published memory studiesutilizing wearable camera technology is still quitelimited, the SenseCam and its immediate succes-sor, the Vicon Revue, remain the most prevalentdevices in the literature. The majority of studies haveused this technology for behavioral experiments,particularly in clinical contexts, but an increasingnumber of fMRI studies examining real-world auto-biographical event recall in healthy individuals haveemerged in recent years.

Research in clinical contexts

External memory aids can be effective tools forassisting individuals with memory impairments.25

Although patients with memory deficits can experi-ence difficulties with the retrieval of personal mem-ories, there are few external memory aids intendedto bolster memory for such autobiographical events.Wearable digital camera devices offer a promisingmethod to help compensate for mnemonic difficul-ties owing to their automaticity in capturing photo-graphs of one’s day-to-day activities.26 A numberof experiments, many conducted as case studies onindividual patients, have evaluated the SenseCam’sefficacy in supporting autobiographical memoryretrieval.

The first such study to appear in a peer-reviewedjournal was a behavioral experiment by Berry et al.26

on a 63-year-old patient with limbic encephali-tis. This patient’s bilateral hippocampal lesions,although relatively mild, resulted in difficultyretrieving both recent and remote autobiographicalevents. The researchers sought to evaluate whetherthe patient’s ability to recall details about her lifeexperiences could be improved by having her wear aSenseCam and periodically review the photographicrecord of any notable (i.e., nonroutine) events. Of

204 Ann. N.Y. Acad. Sci. 1396 (2017) 202–221 C⃝ 2017 New York Academy of Sciences.


particular interest was whether SenseCam-basedrehearsal could outperform a more traditional writ-ten diary–based approach; these two life-loggingmethods were employed sequentially, each for atleast a month. Throughout the study, the patient’shusband periodically tested her ability to recall thedetails of recent life events, with each recall testfollowed by an opportunity to review the Sense-Cam photographs or diary entries that recordedthese events. Although these two forms of life-logging are impossible to equate on all attributes,an effort was made to match the review proce-dure and manner of testing. Relative to the diaryentries, rehearsal of SenseCam photographs wasassociated with substantial improvements in thepatient’s ability to recall the recorded events, evenover long durations of time (e.g., 3 months) with-out the patient reviewing photographs betweentesting sessions. Moreover, her memory for eventssignificantly increased with successive viewings ofSenseCam photographs, but no such progressivebenefit was observed in the diary condition. Despitea number of methodological shortcomings, thisproof-of-concept case study provided support forthe notion that the photographs captured by wear-able cameras might be particularly efficacious ascues for triggering recall of autobiographical eventdetails and bolstering the long-term retention ofthese memories. When fMRI data were later col-lected from this same patient,27 greater activity wasobserved across a network of brain regions typicallyassociated with autobiographical retrieval whenthe patient reported recognition of photographsof an event that she had previously rehearsedusing the SenseCam reviewing procedure relativeto recognition of SenseCam photographs for anevent that had been exclusively rehearsed usingthe written diary procedure. Although such resultscannot prove that SenseCam photographs helpedthis memory-impaired patient recollect her actuallife events as originally experienced, rather thanremembering the repeatedly viewed photographsof the events, these encouraging demonstrations ofmnemonic benefits and heightened retrieval-relatedbrain activity motivated a series of follow-up inves-tigations.

Similarly encouraging results were obtained inanother research team’s case study of an amnesicpatient with a large right-lateralized MTL lesioncaused by herpes simplex viral encephalitis.28 Sense-

Cam cues were found to promote the recollec-tion of significantly more contextual details forautobiographical events relative to cues derivedfrom written diary entries. Importantly, these mem-ory improvements were observed even when theSenseCam photographs were only used as cues forprompting episodic recall and not also used asopportunities for rehearsal. This suggests that thebeneficial effects can extend beyond the strength-ening of autobiographical memory traces throughrepeated study and retrieval practice. SenseCam-induced memory improvements were also appar-ent in a contemporaneous case study of a patientwith mild cognitive impairment.29 While this studyprovided the patient with opportunities to reviewthe photographs captured by her camera (or, inthe control condition, to review diary entries writ-ten by her husband) during the first 2 weeks, theadvantages of the SenseCam procedure were wellapparent even after 6 months had elapsed since herlast event review session, with a twofold increasein the number of event features recalled. Relatedly,a study of six older patients diagnosed with mild-to-moderate Alzheimer’s disease found that reviewof events through SenseCam images, in compari-son with a written diary, resulted in the majority ofpatients being able to recall more event details inboth the short term (2 weeks after the event) andlong term (1 and 3 months afterward)30 (Fig. 2).It is notable that all of the aforementioned studiesreported that SenseCam photographs led patientsto recall event details that were not themselvesapparent in the images. This suggests thatthese automatically captured first-person snap-shots might be particularly effective at triggeringmnemonic pattern-completion processes,31,32 per-haps by harnessing the functional contribution ofany intact portions of the patients’ hippocampi tobring associated event details back to mind. Indeed,Loveday and Conway28 reported that their amnesicpatient would occasionally experience a “Proustianmoment”—a powerful flood of recollected details—when encountering her SenseCam photographs.

By virtue of enhancing patients’ ability to remem-ber events from their daily lives, use of the SenseCammay potentially bestow additional quality-of-lifebenefits. For example, rehearsal of events usingSenseCam photographs resulted in diminishedanxiety and stress as well as increased confidencefor a patient with mild cognitive impairment.29



Figure 2. A comparison of autobiographical recall in patients with Alzheimer’s disease when using different forms of externalmemory aids to review events. (A) Multiple experiments, particularly clinical ones, have included the process of revisiting eventphotographs captured by wearable camera devices. This study, in particular, allowed participants to review images using a softwareprogram called the SenseCam Viewer (Microsoft Research Cambridge). (B) Over the course of several months, participants withAlzheimer’s disease were tested on their recall for experienced events. Their performance, in terms of mean recall percentage,is shown for the SenseCam condition, written diary condition, and the baseline condition (in which no review of the eventswas conducted). Memory for events rehearsed with the SenseCam review method steadily improved across successive viewingsand outperformed diary-based rehearsal, with lasting improvements even after several months had elapsed since the last reviewopportunity. Figure adapted, with permission, from Ref. 30.



Relatedly, the SenseCam can be used within thecontext of psychotherapy for emotional events:for a patient with memory deficits and an anxietydisorder following acquired brain injury, the Sense-Cam was superior in evoking autobiographicalmemory retrieval, including the specific recall ofanxiety-producing events and internal state infor-mation critical for cognitive–behavioral therapeuticintervention.33 SenseCam review also decreasedapathy and increased sense of self in an older patientwith moderate Alzheimer’s disease.34,35 Similarly, apatient with memory impairment stemming fromKorsakoff ’s syndrome demonstrated better recallfor events captured and reviewed with the Sense-Cam, along with improved subjective ratings ofidentity.21 In a larger study of 51 patients with mildAlzheimer’s disease who were randomly assigned toone of three cognitive training programs, includinga written diary and a SenseCam intervention, theSenseCam group showed significantly improvedfunctional capacity and reduced depressivesymptomology when measures were comparedbefore the program and 1 week afterward.36

However, these beneficial effects were transientand decreased when measured 6 months later,suggesting that continued SenseCam use might benecessary to maintain these subjective quality-of-life enhancements. In comparison with the Sense-Cam, anecdotal reports from patients and caregiverssuggest that the written diary method was not asrewarding or effective and could even cause stressor tension with its use.26,29,30,36

It is important to consider what qualities ofthe photographs captured by wearable camerasmake them so effective at cuing episodic recalland strengthening the later accessibility of eventdetails. One advantage of photographs over verbaldiary entries is the fact that pictorial stimuli areknown to be associated with better memory thanverbal stimuli.37,38 Even if the people, objects, orlandmarks depicted within a given photograph areinsufficient to elicit recall of the specific episode,the high degree of perceptual correspondencebetween a first-person perspective photograph andthe visuospatial context in which the event wasencoded may facilitate recollection. Ample researchhas shown that mental reinstatement of a context,typically through the use of visual imagery, aids inthe recovery of information that had been acquiredin that context.39,40 By providing potent visual cues

to promote context visualization, photographs mayaccelerate the initial phase of the mental time travelprocess that is considered to be the hallmark ofautobiographical recollection.1,41,42 Furthermore,camera-based studies typically present participantswith multiple images depicting the temporalunfolding of an event, which provides additionalcontextual information and increases the likelihoodof there being sufficient cues for retrieval43 whileeasing the demands on the executive system toengage in self-initiated episodic search processes.Indeed, it has been suggested that the viewing ofbrief, ordered sequences of photographs capturedby wearable cameras may roughly approximate thetime-compressed and fragmentary characteristicsof actual endogenously retrieved autobiographicalmemories.28 That said, a recent study that probedparticipants’ memories with sequences of Sense-Cam photographs depicting events unfolding ineither their original forward order or in a randomorder found only a small advantage in recallfor the forward-order condition.44 This couldsuggest that the overall amount of detail containedwithin the set of images is more consequential thanthe temporal dynamics conveyed in the sequence.

When SenseCam photographs are periodicallyshown to memory-impaired patients to help themremember recent events, the accessibility of thesememory traces may be progressively strengthenedthrough the well-documented memory-enhancingeffects of spaced retrieval practice.45–47 It is also pos-sible, if not likely, that the details of the event mem-ories will be altered to some degree by each viewingof the photographs. Reminder cues are thought toreturn stored memories to a labile state in which theyare briefly amenable to updating—and distortion—before reconsolidation mechanisms act to stabilizethe trace.48 Although some efforts have been madeto understand the mechanisms and long-term con-sequences of memory reactivation and updating inwearable camera paradigms,49–51 more work willbe needed to evaluate the contributions of retrievalpractice and reconsolidation in memory-impairedpatients using photographic review procedures asan external memory aid.

One significant limitation of studies comparingSenseCam-based review to diary-based review is theinherent difficulty of equating the event-logging andreview procedures. Diary entries can differ wildly incomposition, ranging from basic outlines or notes



of important details26,30 to more expansive entriesrecording event information in addition to asso-ciated emotions and thoughts.29,36 While camerasare worn by the patient, diary entries are typi-cally (although not always28,36) written by some-one else—such as the patient’s spouse26,27,29,30 or theexperimenter30—owing to concerns that memory-impaired patients would be unable to accuratelylog their daily events or that their efforts to do sowould potentially alter their memories and bias theresults. In an apparent trade-off between ease ofimplementation and precise experimental control,many experimenters have opted to involve patients’spouses in reviewing or testing procedures. 26–30 Onestudy even had the experimenter assist some patientswhile other patients were assisted by their spouses.28

Future camera studies incorporating diary reviewas a comparison condition should require that thesame individual, preferably an experimenter, recorddiary entries to ensure consistency and should alsolimit spousal involvement in testing procedures toprevent potential subjectivity. Furthermore, mostof the case studies reviewed above have probedpatients’ memories with a relatively limited num-ber of life events. It would be useful for futurestudies to record and test a larger number ofunique events to better understand wearable cam-eras’ potential to cue retrieval for a broader range ofmemories.

In summary, studies examining the consequencesof SenseCam use in memory-impaired patients havereported promising benefits for the accessibility andvividness of memories for personal events, oftenwith concomitant improvements in subjective well-being. However, given that many of the results werederived from case studies on individual patients withheterogeneous memory disorders, caution is war-ranted in evaluating the robustness and generaliz-ability of these effects. It is our hope that, as wearablecameras become more widely adopted as a tool forpatient rehabilitation, psychologists and clinicianswill continue to collaborate on larger-scale studiesaimed at evaluating the factors that maximally affectthe efficacy of this approach.

Behavioral research in nonclinicalpopulations

Although wearable camera studies have demon-strated marked improvements in autobiographicalrecall for memory-impaired patients, an impor-

tant question is whether this technology wouldalso offer benefits to cognitively healthy individ-uals. Relatively few studies have examined unim-paired participants, but those that have done sohave largely reported positive outcomes. One earlystudy assessed whether young adults would showenhanced long-term retention of events that theyrehearsed using an end-of-day SenseCam photo-graph review procedure. 52 Although substantial for-getting occurred across the 8-week interval of theexperiment, participants’ memories for reviewedevents were more accurate in comparison with non-reviewed events, even when no explicit instructionswere given to memorize the images.

Another study compared SenseCam and diaryreview protocols in groups of healthy younger andolder adults and found that the SenseCam methodenhanced autobiographical memory performancein both age groups.53 Intriguingly, this studyalso found that the SenseCam condition wasassociated with broader enrichment of participants’cognitive function, as assessed by a battery ofneuropsychological tests. The largest effects wereobserved for both memory and executive functiontasks, including measures of semantic, verbal,and working memory. Participants’ subjectivereports indicated that reviewing photographs notonly cued more memories than reviewing diaryentries, but also produced a better sense of reliving.This is in line with previous SenseCam studiesof clinical populations.26,27,29 To explain the per-formance gains on neuropsychological measures,the researchers speculated that SenseCam-basedrehearsal may serve as a short-term cognitivestimulant, potentially by virtue of the photographsbeing interesting and pleasurable to look at, which,in turn, can heighten alertness.53 While potentiallypromising, these findings of generalized cognitiveenhancement should be replicated to confirmwhether the benefits of SenseCam use are as far-reaching as these researchers have claimed. A morerecent study also compared the effects of SenseCamuse in younger and older adults to examine thebenefits of SenseCam images as retrieval cues.44 Rel-ative to cuing memories with participant-generatedevent titles, cuing with SenseCam photographsled to improved recall (including of details notapparent in the images) in both age groups, with nosignificant effects of aging. The apparent lack of agedifferences in these two studies is surprising, given



other work showing age-related changes in behav-ioral performance and neural engagement duringautobiographical recall.54 Further exploration willbe helpful to evaluate what experimental designconsiderations might affect the degree to whichyounger versus older adults’ memories benefitfrom photographic rehearsal and cuing. Indeed, thedata indicating memory improvements in youngeradults are somewhat mixed. Although others havereported a similar benefit of SenseCam-basedmemory cuing over verbal cuing in youngeradults,55 one large study examining the effects ofSenseCam review procedures—versus diary reviewor no review—found no improvements in memoryrecall when participants were tested 1 week later.56

Given the methodological differences in thesevarious experiments, more research is needed todelineate the boundary conditions that determinethe utility of wearable cameras as a memory aid forcognitively healthy individuals.

Neuroimaging research in nonclinicalpopulations

The integration of wearable cameras into fMRI stud-ies has helped to elucidate the neural mechanismsunderlying autobiographical memory retrieval byincorporating naturalistic stimuli to assess partici-pants’ memories for real-world events. After brieflyreviewing the neuroimaging literature on autobio-graphical memory, we will discuss the insights thathave emerged from the seven fMRI experimentspublished to date that have used wearable cam-eras photographs as memory probes in cognitivelyhealthy adults. With the exception of the case studyby Berry et al.27 discussed above, wearable camerashave yet to be incorporated into neuroimaging stud-ies of memory-impaired patients or other clinicalpopulations.

Autobiographical memory retrieval involvesrecruiting a predominately left-lateralized networkof distributed brain regions.11,57 These includeMTL areas such as the hippocampus and parahip-pocampal cortex, which are critically importantfor recollection processes, as well as regions ofthe temporoparietal junction, lateral temporalcortex, and posterior parietal cortex.11,57 Medialregions of the prefrontal cortex (PFC) contributeto the representation of one’s self as an agent inthe memory, as well as the broader schema of theevent, while more lateral PFC regions mediate

episodic search processes and the selection andmaintenance of search results.11,57–59 Additionally,occipital regions, the precuneus, and the amygdalacontribute to the retrieval of mnemonic represen-tations through processes involving visual imageryand emotion.5 The posterior cingulate cortex andretrosplenial cortex are also thought to supportrecollection by facilitating mental reconstructionof the visuospatial reference frame.57,60,61 This setof regions has been consistently associated withautobiographical memory retrieval throughoutthe neuroimaging literature,3,5,11,57–59 althoughsome studies have suggested more bilateral62,63 orright-lateralized involvement.64,65

Neuroimaging studies have used photographicstimuli derived from the SenseCam to assess theneural correlates of recollection and familiarityduring autobiographical recall of real-world events.Milton et al.66,67 evaluated these processes as afunction of memory remoteness. The researchersfirst studied recent memories,66 where partic-ipants were scanned approximately 36 h afterphotograph acquisition. A modified remember/know paradigm was used during the fMRI scansession to assess recall as participants were shownSenseCam images generated from their ownlives as well as the lives of other participants.Recollected and familiar events evoked activity inoverlapping brain regions previously associatedwith autobiographical retrieval, including theposterior cingulate, right inferior parietal lobe,and right dorsolateral PFC. However, recollectionelicited greater activity in the right posterior andanterior parahippocampal gyrus as well as themPFC, whereas familiarity elicited greater activityin the right ventrolateral PFC and bilateral cingulategyrus. These findings are broadly consistent withprevious studies of recollection and familiarity.68,69

Moreover, the activity of the right hippocampusand posterior parahippocampal gyrus increasedparametrically as participants’ retrieval experi-ences increased from weakly familiar to stronglyrecollected. It is unclear whether these effectsshould be attributed to quantitative differences inmemory strength or qualitative differences in thesubjective attributes of retrieval (e.g., the degree ofcontextual reinstatement). However, the fact thatthese regions’ activity increased between weak rec-ollection and strong recollection trials supports thenotion that recollection may not be an all-or-none



phenomenon, but rather may operate as a contin-uously varying or graded retrieval process.70,71

Milton et al.67 then scanned the same participants5 months after last wearing the SenseCam in theoriginal study66 while they performed the samerecognition memory task in order to evaluate theneural mechanisms of recollection and familiarityfor remote autobiographical memories. Com-pared with the previous 36-h retention interval,photographs depicting events that had transpiredapproximately 5 months earlier showed decreasedneural activation related to both recollection andfamiliarity in the right hippocampus and parahip-pocampal gyrus. Indeed, for these remote mem-ories, recollection-related MTL activity no longerexceeded that observed during familiarity-basedresponses or correctly rejected novel images. Consis-tent with the standard consolidation model, whichpredicts reduced MTL involvement and increasedneocortical involvement as memories becomemore temporally remote,72 the researchers onlyfound recollection-related activity for 5-month-oldmemories in neocortical regions, such as the mPFC.However, it should be noted that the second fMRIexperiment from Milton et al.67 had a relativelysmall sample (n = 10), limiting their experimentalpower. Additionally, owing to the small number ofunique events captured by the cameras, they optedto use the same stimuli for both the short-delay andlong-delay scanning sessions, so the neural repre-sentations of the remote memories may have beensomewhat altered by this retrieval practice. Fortu-nately, more work is underway to investigate theeffects of both temporal remoteness and retrievalpractice on the neural correlates of real-worldmemories.73

St. Jacques and her collaborators have alsoused wearable cameras to conduct a set of fMRIstudies assessing the neural mechanisms of autobi-ographical memory retrieval for real-world events.Their first study74 investigated how the processesassociated with mentally projecting oneself intospecific events from one’s past differ from thosesupporting the simulation of another individ-ual’s perspective. Participants wore the Sense-Cam while concurrently keeping a written recordof daily activities. One week after last wearing theSenseCam, participants underwent fMRI scanningand were presented with photographs from theirown lives as well as from the lives of other individ-

uals and instructed to either retrieve the depictedevents or comprehend the depicted event fromanother individual’s perspective. Overall, SenseCamphotographs allowed participants to strongly re-experience their personal past as well as understandanother individual’s perspective. Not surprisingly,projection into one’s own past evoked greateractivity in areas previously implicated in autobio-graphical memory retrieval, including the bilateralventrolateral PFC, left hippocampus, posterior mid-line regions, and lateral temporal regions. However,an interesting dissociation was observed within themPFC, such that projection into one’s own pastpreferentially recruited a ventral component of themPFC, whereas projection into someone else’s pastpreferentially recruited a more dorsal component ofthe mPFC.59,74 Task-related functional connectivityfurther established the different contributionsof dorsal and ventral mPFC regions: the ventralmPFC showed greater connectivity with regions ofthe hippocampus and precuneus associated withepisodic retrieval memory processes, whereas thedorsal mPFC demonstrated greater connectivitywith areas of the frontoparietal network associatedwith control processes. These results provided novelevidence that ventral and dorsal mPFC regionssupport dissociable forms of self-projection.

Data collected from the aforementioned studywere also used to examine putative gender differ-ences in autobiographical recall evoked by visualversus verbal retrieval cues.55 During the fMRIscanning session, memories were cued by eithera sequence of SenseCam photographs (dynamicvisual cue) or a short textual description (verbalcue). Men demonstrated greater neural activityduring the reliving of memories elicited by thevisual cues, relative to the verbal ones, in regionsassociated with autobiographical memory, includ-ing the left hippocampus, left interior frontal gyrus,right occipital cortex, and retrosplenial cortex. Incomparison, women were equally sensitive to bothtypes of cues, such that neural activity did notdiffer significantly in response to reliving promptedby verbal or visual stimuli. These results couldhave important implications for studies usingcamera-based life-logging procedures to bolsterautobiographical retrieval in patient populations,as males and females may experience differencesin the relative efficacy of photographs versus diaryentries as memory prompts.



Figure 3. Common and distinct neural areas associated with subsequent true and false autobiographical memories. (A) Theexperimental design included three phases. Session 1 involved the encoding task, where participants engaged in a museum tour.Session 2 involved reactivation during the fMRI scan session, where participants were presented with images from museum stopsthey had visited (“targets”) and prompted to make ratings of their sense of reliving (“partial trial”). A subset of trials in thesecond session (“full trial”) also presented an image from an alternate museum tour (“lure”) and prompted participants to ratethe relatedness of the two images. Session 3 involved the recognition memory test: participants were presented with targets andlures that were either from the second session or were completely novel (“baseline”) and were prompted to indicate whether theimages contained a museum stop that had been visited. (B) Subsequent true memories (target images that were later recognized)and subsequent false memories (lure images that were later reported as recognized) were associated with activation in severalcommon brain areas, including the left parahippocampal cortex, bilateral retrosplenial cortex, and bilateral posterior inferiorparietal cortex. (C) However, different brain regions were associated with reactivation quality for subsequent hits and false alarms.For memories with high reliving ratings during target presentation relative to lure presentation, subsequent hits showed greaterreactivation-related activity in regions like the rostral mPFC and posterior cingulate cortex, while subsequent false alarms showedgreater reactivation-related activation in the right hippocampus and ventral mPFC. (D) Behavioral results from the recognitionmemory test demonstrated increased rates for hits and false alarms for reactivated images relative to baseline ones. (E) Behavioralresults indicated that the quality of memory reactivation differed on the basis of recognition memory performance. Mean relivingratings were greater for hits (responding “yes” to a target) relative to misses (responding “no” to a target). Mean reliving ratingswere also greater for false alarms (responding “yes” to a lure) in comparison to correct rejections (responding “no” to a lure). Figureadapted, with permission, from Ref. 50.

In another cleverly designed fMRI experimentincorporating wearable cameras, St. Jacques et al.50

investigated how the neural mechanisms associatedwith the cued reactivation of event memories cancontribute not only to the subsequent strength-ening of these memories but also, under certaincircumstances, to their distortion. Participants intheir study were given Vicon Revue cameras towear as they completed a self-guided museum tour(Fig. 3). Two days later, participants underwentfMRI scanning while they were presented withphotographs from their own cameras to triggermemory reactivation for events they had experi-enced during their museum tour, with a subset ofthese images followed by a new lure photographderived from an alternative version of the museumtour that participants had not actually experienced.Then, 2 days after the scan, participants completeda recognition memory task where they werepresented with photographs of reactivated targets

and lures that had been previously encountered inaddition to novel photographs of targets and lures.Not surprisingly, events that had been reactivatedreceived a boost in subsequent memory. However,participants also reported increased recognitionof photographs depicting event elements that theyhad not actually experienced in real life but whichhad become falsely woven into their memories ofreal events through the presentation of lure imagesduring the reactivation session. This reactivation-induced memory distortion was consistent withthe findings of the researchers’ previous behavioralexperiment,51 setting the stage for their investiga-tion into the neural correlates of this robust andputatively adaptive75 quirk of episodic memory.

When these researchers examined the fMRI activ-ity associated with photographs that participantswould subsequently claim to remember, theyfound a number of regions that showed increasesin activation (relative to subsequently forgotten



events), regardless of whether these memorieswere true or false. These regions—which includedthe bilateral posterior inferior parietal cortex, leftposterior parahippocampal cortex, and bilateralretrosplenial cortex—also showed sensitivity to thedegree of reliving reported by participants duringthe scanning session. Further examination of theseregions’ responses during lure trials revealed thatthe lure photographs that went on to later be cor-rectly rejected by participants (indicating that theywere not falsely integrated into the original memorytrace) were associated with low activity levels at thetime the lure appeared on the screen, whereas thelure photographs that went on to be falsely remem-bered as real experiences were associated with sus-tained involvement of these areas during both initialreactivation and lure presentation. In addition tofinding that these regions’ activity predicted bothtrue and false subsequent memories, the analysesalso identified regions that were uniquely associatedwith true or false subsequent memories. For trialswith high reliving ratings, comparing target presen-tation relative to lure presentation demonstratedthat subsequently true memories evoked greateractivity in the posterior cingulate and rostromedialPFC, whereas subsequently false memories evokedgreater activity in the ventrolateral PFC, ventralmPFC, lateral temporal cortex, and right anteriorhippocampus. Taken together, these findings help toclarify the neural processes at work when revisitingphotographs of a past event, showcasing how someof the same regions involved in strengtheningthe representation of a true memory can also con-tribute toward the creation of a (at least partially)false one because of the inherent malleability ofmemory traces immediately following reactivation.

On the basis of the success of this experimen-tal paradigm, it was later adapted in a behavioralstudy to assess differences in reactivation processesbetween healthy younger and older adults.49 Con-sistent with prior findings,50,51 reactivation qualityaffected subsequent memory such that photographsthat evoked greater reliving ratings during the reac-tivation phase were more likely to lead to subsequenthits or, in the case of lure stimuli, subsequent falsealarms during the recognition phase. Furthermore,in line with the broader literature on age-relatedincreases in the frequency of false remembering,76

older adults exhibited significantly more false alarmsthan younger adults. But despite this overall increase

in false memories, older adults showed a smallerimpact of reactivation on subsequent recognitionperformance. Accordingly, aging appears to dimin-ish the ease with which episodic memories can beupdated. Although this property likely has negativeconsequences in many circumstances in which it isdesirable to update one’s memory on the basis ofnew information, it also has the somewhat coun-terintuitive positive consequence of making thememories of older adults less vulnerable toreactivation-induced distortions. These data thushelp to advance our understanding of how the crit-ical process of memory updating changes over thelife span.

In another innovative investigation into the brainmechanisms that support memory for real-worldevents, Nielson et al.24 gave their participantscustomized neck strap–mounted smartphones torecord photographs, along with correspondingGPS coordinates, of their experiences over a periodof roughly one month (Fig. 4). The participantsthen underwent fMRI scanning, during whicheach was presented with individual photographsfrom their smartphone’s camera and instructedto mentally relive each experience. The resultingfMRI data were analyzed using multivoxel patternanalysis (MVPA), a methodological techniquethat differs from traditional univariate analyses byassessing the spatial pattern of brain activation,rather than the peak of activation, which can allowfor greater sensitivity.77–79 Using a variant of MVPAknown as representational similarity analysis, thefMRI activity patterns from individual trials werecompared with one another and their dissimilarity(i.e., “neural distance”) was computed. Whenthe researchers attempted to relate the neuraldistance between pairs of events to the spatialdistance between them (i.e., how much geographicdistance separated the locations where the probephotographs were captured), they found thatactivity patterns within the left anterior hippocam-pus could be used to predict spatial distancesbetween events, ranging from 100 m to 30 km.Strikingly, this same region was found to also carryinformation about the temporal distance betweenevents, such that events that took place furtherapart in time (e.g., 1 month) showed greater neuraldistance than events that occurred closer togetherin time (e.g., 15 h). These results demonstratelateralization in the hippocampal computations



Figure 4. The spatial and temporal distance between autobiographical events scales with the dissimilarity of neural activitypatterns (“neural distance”) in the left anterior hippocampus. (A) A heat map representing locations in Columbus, Ohio whereparticipants’ images were captured by wearable GPS-enabled smartphones. (B) Regions of interest in the medial temporal lobe:anterior hippocampus (red), intermediate hippocampus (yellow), posterior hippocampus (blue), and parahippocampal cortex(green). (C) Select event locations for a single participant, where each red marker indicates a photograph that was presented duringthe scan session. The time and corresponding location of four sample photographs are included, along with the associated heatmaps of the single-trial activation parameter estimates in the right and left hippocampus. (D and E) When the effects of otherfactors were eliminated from the model, neural distance within the left anterior hippocampus was correlated with both spatialdistance (D) and temporal distance (E). Each blue marker indicates a pair of photographs shown to participants, with the blacklines representing the estimated neural distance from each participant’s regression results and the red lines indicating the averagedestimated neural distance across all participants. Figure adapted, with permission, from Ref. 24.

supporting the recall of autobiographical eventdetails, with the left anterior hippocampus playinga particularly important role in representing andintegrating the spatiotemporal characteristics ofpersonal episodes. This study nicely illustratesthe potential for wearable camera technology toprovide valuable information, such as geographicaland temporal tagging of real-world experiencesover many weeks, which could not be easilyascertained through other methods. This in turnallows for an enriched understanding of how thespatial and temporal features of event knowledgeare represented in the brain.

Most recently, Rissman et al.20 examined thedegree to which an individual’s level of memory forpersonally experienced events can be decoded onthe basis of distributed fMRI activity patterns mea-sured in response to wearable camera photographs(Fig. 5). After wearing a Vicon Revue camera for3 weeks, participants were scanned while viewingbrief sequences of photographs depicting eventsfrom their own lives or from other participants’lives. Participants indicated their subjective retrievalexperience with one of eight response options,which included varying levels of novelty, familiar-ity, and recollection. Using MVPA methodology, theneural activation patterns associated with individualtrials were used to train a logistic regression classifier

algorithm, which learned the distributed patternsof activity most capable of distinguishing betweeneach of the subjective retrieval outcomes. The clas-sifiers were then used to predict the mnemonicstate of trials on which the model had not beentrained. The results revealed extraordinarily accu-rate classification (>90% correct) of whether eachprobed memory was from one’s own life or some-one else’s life. Classifiers could also decode morenuanced information about the subjective qualitiesof one’s remembrance, such as whether the pho-tographs evoked a strong or moderate sense of rec-ollection, familiarity, or novelty. These neural signa-tures of autobiographical retrieval were found to bestable across retention intervals of up to 1 month, aswell as highly consistent across participants. Assess-ment of the classifier-based “importance maps” pro-vided insights into which brain regions provideddiagnostic signals for each mnemonic classificationscheme. For instance, when classifying hits versuscorrect rejections, an extensive set of lateral fron-toparietal regions were highly predictive of partici-pants’ own events, whereas activity in visual regions,such as occipital and inferior temporal areas, tendedto be predictive of novel photographs from someoneelse’s life. When classifying between trials where par-ticipants reported recollection of contextual detailsversus trials associated with only familiarity-based



Figure 5. Decoding neural signatures of autobiographical event retrieval. (A) On each trial of the fMRI session, participantsviewed a sequence of four photographs depicting the temporal unfolding of an event and then made a judgment indicating theirlevel of memory for the event. (B) For each participant, experimental trials included photographic sequences from their own life,across the span of the 3 weeks they wore their camera device, as well as sequences from the lives of three other participants. (C)The response options for participants during the scan session range from reporting strong recollection of the depicted event toexpressing high confidence that the event was not from one’s own life. (D) Maps of classifier importance values, associated withfour different binary classification analyses, averaged across participants. Warm colors depict voxels where increased activationbiased the classifier to predict that a trial was associated with the condition listed in orange print, whereas cool colors depict voxelswhere increased activation biased the classifier to predict that a trial was associated with the condition listed in blue print. For eachclassification, decoding performance is reported as the mean area under the receiver operating characteristic curve (AUC). Rec,recollection; Fam, familiarity. Figure adapted, with permission, from Ref. 20.

recognition, regions most diagnostic of recollectionincluded the hippocampus and parahippocampalcortex, as well as medial frontal areas and parietalregions, such as the retrosplenial cortex and pos-terior cingulate cortex, along with the left angu-lar gyrus. These results build upon earlier efforts

to decode memory-retrieval states associated withlaboratory-encoded visual stimuli80 by extendingsuch effects to real-world events and showcasethe ability of fMRI to differentiate between subtlegradations in the strength and subjective quality ofone’s memory.



Taken together, wearable cameras have beenutilized by memory researchers to capture pho-tographs of real-world experiences that can later

be presented in the fMRI scanner to probe variousaspects of autobiographical memory (the mainfindings of these fMRI studies are summarized in

Table 1. Neuroimaging research in nonclinical populations

Experiment Camera protocol Memory protocol Main findings

Milton et al.65 ! 15 healthy participants(18–25 years old)! 2-day SenseCam capture ofdaily events

! fMRI scan occurred!36 h after camera wasworn.! Modified remember/knowparadigm used to testrecognition memory inresponse to presentations ofparticipants’ own photos orphotos from otherindividuals.

! Recollection elicited greateractivity in the mPFC and rightparahippocampal gyrus.! Familiarity elicited greateractivity in the rightventrolateral PFC and bilateralcingulate gyrus.! Regions including the righthippocampus, rightparahippocampal gyrus, andmPFC were parametricallymodulated by the subjectivestrength of recollection.

Milton et al.67 ! 10 healthy participants(18–25 years old)! 2-day SenseCam capture ofdaily events

! fMRI scan occurred!5 months after camera wasworn.! Extension of Milton et al.66

using the same memoryprotocol with identicalphotos presented in bothscan sessions.

! Relative to recently encodedmemories, remote memoriesshowed decreased recollectionand familiarity-related activityin the right hippocampus andparahippocampal gyrus.! Neocortical regions, includingthe mPFC, continued to berecruited during retrieval ofremote memories.

St. Jacques et al.74 ! 23 healthy participants(18–35 years old)! 6-day SenseCam capture ofdaily events

! fMRI scan occurred!1 week after the last daythe camera was worn.! Each trial presented adynamic sequence of 40photos depicting an eventfrom the participant’s life orsomeone else’s life.! Participants were instructedto mentally projectthemselves into each event,rating either reliving fortheir own life events orunderstanding for other’slife events.

! Self-projection preferentiallyengaged the ventral mPFC,while projection into another’sperspective preferentiallyengaged the dorsal mPFC.! The ventral mPFC showedgreater task-related functionalconnectivity with regions ofthe MTL network associatedwith memory processes.! The dorsal mPFCdemonstrated greatertask-related functionalconnectivity with areas of thefrontoparietal networkassociated with controlprocesses.

St. Jacques et al.55 ! 23 healthy participants(18–35 years old)! 6-day SenseCam capture ofdaily events

! fMRI data collected at thesame time as St. Jacqueset al.74! Trials comprised eitherphoto sequences or verbalretrieval cues describingevents from the participants’lives.! Participants were giveninstructions to recall eachevent and rate their reliving.

! Women were sensitive to bothvisual and verbal cues, with nosignificantly different activity.! Men were more sensitive tovisual cues, which evokedgreater activity in areasassociated withautobiographical memoryretrieval, including the lefthippocampus.

Continued



Table 1. Continued

Experiment Camera protocol Memory protocol Main findings

St. Jacques et al.50 ! 35 healthy participants(18–30 years old);26 included in the fMRIanalysis! !4- to 5-h Vicon Revuecapture of a self-guidedmuseum tour

! The experiment included 3sessions with 48 h betweeneach: (1) museum tour whilewearing camera, (2) fMRIscan session (reactivationphase), and(3) recognition test.! During fMRI scan,participants’ camera photoswere used to cue recall ofmuseum tour stops andparticipants rated theirreliving.! On some fMRI trials, after aparticipant’s own photo wasshown, a lure photo from analternate version of themuseum tour was presented,and the participant judgedhow related the depictedexhibit was to the one intheir own photo.

! Reactivated events increasedboth true and false subsequentmemories, depending onreactivation quality.! Common regions associatedwith true and false subsequentmemories included the leftposterior parahippocampalcortex, bilateral posteriorparietal cortex, andretrosplenial cortex.! Subsequently true memorieswere associated with greateractivation in regions such asthe rostromedial PFC, whilesubsequently false memorieswere associated with greateractivation in areas includingthe ventrolateral PFC, ventralmPFC, and righthippocampus.

Nielson et al.24 ! 9 healthy female participants(19–26 years old)! Participants woresmartphones for !1 monthto record daily events as wellas their time and GPScoordinates

! fMRI scan took place 1-3weeks after thecamera-wearing periodconcluded.! Photos from participant’scamera were presented oneat a time to cue retrieval,with participants indicatingwhether they recalled thedepicted event, and howvividly.

! Representational similarityanalysis searched for MTLareas where the “neuraldistance” between pairs ofevents was related to thespatial or temporal distancesbetween pairs.! The left anterior hippocampuswas found to represent recalledautobiographical events’spatial features for distancesranging from 100 m to 30 km.! The left anterior hippocampuswas also found to represent thetemporal features of recalledautobiographical events fortimes ranging from 15 h to1 month.

Rissman et al.20 ! 16 healthy participants(18–22 years old)! 3-week Vicon Revue captureof daily events

! fMRI scan occurred 6–9 daysafter the camera-wearingphase concluded.! Participants were presentedwith sequences of fourphotos depicting eventsfrom their own life or fromother participants’ lives.! Participants used one ofeight responseoptions—which includedlevels of recollection,familiarity, and novelty—toindicate their retrievalexperience.

! Participants’ subjectiveretrieval experience could bereliably decoded from fMRIactivity patterns.! The neural signaturesassociated withautobiographical retrievalwere highly consistent acrossparticipants and were stableup to a 1-month retentioninterval.! Regions most diagnostic ofrecollection (vs. familiarity)included the hippocampus,parahippocampal cortex, leftangular gyrus, medial frontalareas, and parietal regions.



Table 1). The results have largely corroborated thefield’s prior characterization of the neural substratesof autobiographical recall, as derived from studiesusing laboratory-based techniques for probingparticipants’ memories for past events. In this sense,rather than upending our understanding of thecore cortical and MTL brain systems that supportevent retrieval, camera-based fMRI paradigms havehelped confirm that these mechanisms can general-ize to the retrieval of real-world memories encodedin naturalistic settings. That said, this emerging liter-ature contains a number of novel findings, includingthe dissociation between ventral and dorsal mPFCcontributions to the reliving of a personally expe-rienced event memory versus projecting oneselfinto an event experienced by someone else,74 thegraded nature of neural representations of episodicrecollection and familiarity,20,66,67 hippocampal lat-eralization in the representation of spatiotemporalinformation associated with personal events,24 thedifferential sensitivity of men and women to verbalversus visual retrieval cues,55 and the mechanisms ofreactivation-induced distortion of real-world eventmemories.50 Although there are certainly circum-stances in which the enhanced experimental controlover exposure duration, attentional allocation, andevent content provided by laboratory stimuli canoutweigh the enhanced ecological validity providedby wearable camera stimuli, we believe that thefMRI studies reviewed above have provided animportant first step toward showcasing the viabilityof more naturalistic paradigms for catalogingpeople’s day-to-day experiences and characterizingthe brain processes evoked during their retrieval.

Discussion

Wearable camera technology has not only been usedto enhance individuals’ memories, but has also beeninstrumental as a tool for studying the cognitiveand neural processes that support autobiographi-cal memory. The integration of wearable cameratechnology into behavioral and fMRI experimentspermits more ecologically valid assessments of auto-biographical memory retrieval by providing detail-rich, personally relevant cues that evoke specificexperiences. As such, wearable camera photographsmay better capture the complex phenomeno-logical properties of real-world memories thanlaboratory-based stimuli. Like other prospectiveexperimental methods for logging one’s day-to-day

experiences, these nonintrusive camera devicesallow for some degree of experimental control,but critically avoid other techniques’ potential forinterfering with the encoding process.11 Althoughintegration of wearable camera technology withneuroimaging approaches to assess healthy adultpopulations has only occurred recently, these tech-niques have been effectively used to evaluate thecontributions of various cortical and MTL regionsto the recollection of events from one’s personalpast. While several other review articles on wear-able cameras have recently appeared, they predom-inantly focus on the devices’ rehabilitative appli-cations and do not comprehensively cover extantneuroimaging experiments.81–83 The combinationof wearable camera technology and neuroimagingmethods may prove to be a powerful approach thathelps further elucidate the complexities of autobio-graphical recall for real-world events.

Despite the many promising findings highlightedin our review, this still-small body of research suffersfrom a number of limitations that will be impor-tant to address as the field moves forward. Many ofthe clinical studies investigating the use of wearablecameras as a therapeutic tool for bolstering retentionof autobiographical memories in memory-impairedpatients have derived their results from single casesor very small cohorts. Given the heterogeneity ofthese patients, as well as of the experimental proce-dures of individual studies, it is hard to specify whichtypes of patients will be most amenable to the ben-efits of wearable cameras and which protocol forphotographic review will be most effective. Theselimitations could begin to be addressed by larger-scale clinical trials featuring more careful controlover the procedures for selecting photographs forpatients to review, as well as the structure and timingof the memory rehearsal and testing sessions. Effortsshould also be made to address the demographicdisparity in the wearable camera literature. Themajority of clinical experiments have focused onolder adults with memory impairments, whereasnonclinical experiments have primarily assessedhealthy, younger adults. In order to evaluate the gen-eralizability of extant findings, it would be helpfulto know whether memory-impaired younger adultscould benefit from wearable cameras, and morebehavioral and neuroimaging studies should bedone using wearable cameras in cognitively healthyolder adults.



On the basis of our own experiences, as well asthose described in other studies, there are manypractical challenges inherent in the implementationof camera-based experimental paradigms. One dif-ficulty is participant compliance: even with carefulinstructions, participants may not wear their cam-eras in the “on” mode for long enough to generatea sufficient number of photographs or to captureenough unique events. Even if cameras were wornas instructed, it is possible to capture repetitive andgeneric daily events that may not be memorable orpersonally relevant.20,23,66 Given the variability oflife experiences across participants, or even vari-ability within participants across days, it can behard to adopt universally applicable guidelines forselecting photographs for use as memory probes.Furthermore, in some experimental paradigms,participants’ cameras can generate several thou-sand photographs,20,52,84 so combing through theseimages in search of optimal stimuli can be an incred-ibly labor-intensive process. Other issues pertainto image quality. Despite ongoing improvementsin wearable camera technology, it can be diffi-cult to capture photographs under low-light condi-tions 23,24,26,44 or during periods of movement.24,66

One study estimated that 93% of their smartphoneimages were unusable owing to image quality orrepetitiveness,23 but even uneventful photographsmay be able to cue memory retrieval.85 It is alsohard to know whether a camera wearer was pay-ing attention to his or her surroundings at the timethat a given photograph was recorded; a seeminglyinteresting event could have been captured by acamera while its wearer was looking elsewhere orconsumed by unrelated thoughts. Although theseissues may make camera-based studies more chal-lenging to implement than other memory experi-ments, the advantages provided by such paradigmsin facilitating the study of real-world autobiograph-ical memories—for which the details can be verifiedby photographs, timestamps, and sometimes evenGPS coordinates—may outweigh the obstacles.

As video-enabled wearable camera devicesachieve increasingly greater storage capacity andbattery life, researchers should explore what addedutility video might provide for clinical applicationsand cognitive neuroscience research studies. Fur-thermore, if audio recordings are also collected, thenthis combination may provide additional contex-tual details (e.g., recognizable voices, interpersonal

dialogue, environmental sounds) to aid retrieval.Indeed, audiovisual stimuli typically evoke bet-ter recognition memory performance than eithermodality individually.86 However, legal issues andprivacy concerns pertaining to the surreptitiousrecording of conversations may ultimately limit theviability of audio and audiovisual life-logging tech-nology. Future research efforts should remain mind-ful of such user-experience considerations.87,88

Since the literature regarding camera-basedinvestigations of autobiographical memory retrievalis still nascent, the field is ripe with underexploredresearch questions that could be approached withthis technology. For instance, it could be infor-mative to deploy wearable cameras to examinehow people’s memories may be affected throughsocial interactions with other individuals, includ-ing the effects of photo sharing (e.g., through socialmedia applications) on memory accuracy and reten-tion. Studies could also provide cameras to groupsof individuals who experience the same eventsfrom different vantage points. The resulting pho-tographs could provide a unique opportunity toassess the viewpoint specificity of real-world mem-ories. This research direction may be of particu-lar interest in applied settings concerned with thedetection of autobiographical memories for specificpast experiences.20,89–92 In sum, although wearablecamera technology has already been productivelyused to further our understanding of autobiograph-ical memory retrieval—and in some circumstances,to rehabilitate its deficiencies—we hope that thesedevices will provide many exciting opportunities forfuture research into the recall of real-world events.

Acknowledgments

The authors would like to thank Andrew Westphalfor his helpful comments and suggestions on thismanuscript. Funding for this work was providedin part by a National Science Foundation GraduateResearch Fellowship Program award to T.E.C. and aHellman Fellows Fund award to J.R.

Competing interests

The authors have no competing interests to report.

References

1. Tulving, E. 2002. Episodic memory: from mind to brain.Annu. Rev. Psychol. 53: 1–25.



2. Cabeza, R., S.E. Prince, S.M. Daselaar, et al. 2004. Brainactivity during episodic retrieval of autobiographical andlaboratory events: an fMRI study using a novel photoparadigm. J. Cogn. Neurosci. 16: 1583–1594.

3. Chen, H.Y., A.W. Gilmore, S.M. Nelson, et al. 2017. Are theremultiple kinds of episodic memory? An fMRI investigationcomparing autobiographical and recognition memory tasks.J. Neurosci. 37: 2764–2775.

4. Kim, H. 2012. A dual-subsystem model of the brain’s defaultnetwork: self-referential processing, memory retrieval pro-cesses, and autobiographical memory retrieval. Neuroimage61: 966–977.

5. McDermott, K.B., K.K. Szpunar & S.E. Christ. 2009.Laboratory-based and autobiographical retrieval tasks differsubstantially in their neural substrates. Neuropsychologia 47:2290–2298.

6. LePort, A.K., A.T. Mattfeld, H. Dickinson-Anson, et al. 2012.Behavioral and neuroanatomical investigation of highlysuperior autobiographical memory (HSAM). Neurobiol.Learn. Mem. 98: 78–92.

7. Patihis, L., S.J. Frenda, A.K. LePort, et al. 2013. False mem-ories in highly superior autobiographical memory individ-uals. Proc. Natl. Acad. Sci. U.S.A. 110: 20947–20952.

8. Palombo, D.J., C. Alain, H. Soderlund, et al. 2015. Severelydeficient autobiographical memory (SDAM) in healthyadults: a new mnemonic syndrome. Neuropsychologia 72:105–118.

9. LePort, A.K., S.M. Stark, J.L. McGaugh, et al. 2017. A cogni-tive assessment of highly superior autobiographical memory.Memory 25: 276–288.

10. Roediger, H.L. & K.B. McDermott. 2013. Two types of eventmemory. Proc. Natl. Acad. Sci. U.S.A. 110: 20856–20857.

11. Cabeza, R. & P. St Jacques. 2007. Functional neuroimagingof autobiographical memory. Trends Cogn. Sci. 11: 219–227.

12. St Jacques, P., D.C. Rubin, K.S. LaBar, et al. 2008. The shortand long of it: neural correlates of temporal-order memoryfor autobiographical events. J. Cogn. Neurosci. 20: 1327–1341.

13. Henkel, L.A. 2014. Point-and-shoot memories the influenceof taking photos on memory for a museum tour. Psychol.Sci. 25: 396–402.

14. Barclay, C.R. & H.M. Wellman. 1986. Accuracies andinaccuracies in autobiographical memories. J. Mem. Lang.25: 93–103.

15. Burt, C.D., S. Kemp & M.A. Conway. 2003. Themes, events,and episodes in autobiographical memory. Mem. Cogn. 31:317–325.

16. Levine, B., G.R. Turner, D. Tisserand, et al. 2004. The func-tional neuroanatomy of episodic and semantic autobio-graphical remembering: a prospective functional MRI study.J. Cogn. Neurosci. 16: 1633–1646.

17. Svoboda, E. & B. Levine. 2009. The effects of rehearsal on thefunctional neuroanatomy of episodic autobiographical andsemantic remembering: a functional magnetic resonanceimaging study. J. Neurosci. 29: 3073–3082.

18. Hodges, S., L. Williams, E. Berry, et al. 2006. SenseCam: aretrospective memory aid. In Proceedings of the 8th Interna-tional Conference of Ubiquitous Computing (UbiComp 2006).P. Dourish & A. Friday, Eds.: 177–193. Springer.

19. Bell, G. & J. Gemmell. 2009. Total Recall: How the E-MemoryRevolution Will Change Everything. New York: Dutton.

20. Rissman, J., T.E. Chow, N. Reggente, et al. 2016. Decod-ing fMRI signatures of real-world autobiographical memoryretrieval. J. Cogn. Neurosci. 28: 604–620.

21. Svanberg, J. & J.J. Evans. 2014. Impact of SenseCam on mem-ory, identity and mood in Korsakoff ’s syndrome: a singlecase experimental design study. Neuropsychol. Rehab. 24:400–418.

22. Chow, T.E., A.J. Westphal, L.A. Storozuk, et al. 2014. Multi-voxel pattern classification differentiates personally expe-rienced event memories from second-hand event knowl-edge. Program No. 172.13. In Society for Neuroscience AnnualMeeting, Washington, DC.

23. De Leo, G., E. Brivio & S.W. Sautter. 2011. Supporting auto-biographical memory in patients with Alzheimer’s diseaseusing smart phones. Appl. Neuropsychol. 18: 69–76.

24. Nielson, D.M., T.A. Smith, V. Sreekumar, et al. 2015. Humanhippocampus represents space and time during retrievalof real-world memories. Proc. Natl. Acad. Sci. U.S.A. 112:11078–11083.

25. Kapur, N., E.L. Glisky & B.A. Wilson. 2004. Technologicalmemory aids for people with memory deficits. Neuropsychol.Rehabil. 14: 41–60.

26. Berry, E., N. Kapur, L. Williams, et al. 2007. The use of a wear-able camera, SenseCam, as a pictorial diary to improve auto-biographical memory in a patient with limbic encephali-tis: a preliminary report. Neuropsychol. Rehabil. 17: 582–601.

27. Berry, E., A. Hampshire, J. Rowe, et al. 2009. The neuralbasis of effective memory therapy in a patient with limbicencephalitis. J. Neurol. Neurosurg. Psychiatry 80: 1202–1205.

28. Loveday, C. & M.A. Conway. 2011. Using SenseCam with anamnesic patient: accessing inaccessible everyday memories.Memory 19: 697–704.

29. Browne, G., E. Berry, N. Kapur, et al. 2011. SenseCamimproves memory for recent events and quality of life ina patient with memory retrieval difficulties. Memory 19:713–722.

30. Woodberry, E., G. Browne, S. Hodges, et al. 2015. The useof a wearable camera improves autobiographical memory inpatients with Alzheimer’s disease. Memory 23: 340–349.

31. Horner, A.J., J.A. Bisby, D. Bush, et al. 2015. Evidencefor holistic episodic recollection via hippocampal patterncompletion. Nat. Commun. 6: 7462.

32. Liu, K.Y., R.L. Gould, M.C. Coulson, et al. 2016. Tests ofpattern separation and pattern completion in humans—asystematic review. Hippocampus 26: 705–717.

33. Brindley, R., A. Bateman & F. Gracey. 2011. Explorationof use of SenseCam to support autobiographical memoryretrieval within a cognitive–behavioural therapeutic inter-vention following acquired brain injury. Memory 19: 745–757.

34. Crete-Nishihata, M., R.M. Baecker, M. Massimi, et al. 2012.Reconstructing the past: personal memory technologies arenot just personal and not just for memory. Hum. Comput.Interact. 27: 92–123.

35. Massimi, M., E. Berry, G. Browne, et al. 2008. An exploratorycase study of the impact of ambient biographical displays on



identity in a patient with Alzheimer’s disease. Neuropsychol.Rehabil. 18: 742–765.

36. Silva, A.R., M.S. Pinho, L. Macedo, et al. 2017. It is not onlymemory: effects of sensecam on improving well-being inpatients with mild Alzheimer disease. Int. Psychogeriatr. 29:741–754.

37. Snodgrass, J.G., R. Volvovitz & E.R. Walfish. 1972. Recog-nition memory for words, pictures, and words + pictures.Psychon. Sci. 27: 345–347.

38. Maisto, A.A. & D.E. Queen. 1992. Memory for pictorialinformation and the picture superiority effect. Educ. Geron-tol. 18: 213–223.

39. Smith, S.M. & E. Vela. 2001. Environmental context-dependent memory: a review and meta-analysis. Psychon.Bull. Rev. 8: 203–220.

40. Smith, S.M., J.D. Handy, G. Angello, et al. 2014. Effectsof similarity on environmental context cueing. Memory 22:493–508.

41. Brewer, W.F. 1996. “What is recollective memory?” InRemembering our Past: Studies in Autobiographical Memory.D.C. Rubin, Ed.: 19–66. Cambridge: Cambridge UniversityPress.

42. Williams, J., H. Healy & N. Ellis. 1999. The effect of image-ability and predicability of cues in autobiographical mem-ory. Q. J. Exp. Psychol. A 52: 555–579.

43. Barnard, P.J., F.C. Murphy, M.T. Carthery-Goulart, et al.2011. Exploring the basis and boundary conditions ofSenseCam-facilitated recollection. Memory 19: 758–767.

44. Mair, A., M. Poirier & M.A. Conway. 2017. Supporting olderand younger adults’ memory for recent everyday events:a prospective sampling study using SenseCam. Conscious.Cogn. 49: 190–202.

45. Roediger, H.L., 3rd & A.C. Butler. 2011. The critical role ofretrieval practice in long-term retention. Trends Cogn. Sci.15: 20–27.

46. Soderstrom, N.C., T.K. Kerr & R.A. Bjork. 2016. The criticalimportance of retrieval–and spacing–for learning. Psychol.Sci. 27: 223–230.

47. Sekeres, M.J., K. Bonasia, M. St-Laurent, et al. 2016.Recovering and preventing loss of detailed memory: differ-ential rates of forgetting for detail types in episodic memory.Learn. Mem. 23: 72–82.

48. Schwabe, L., K. Nader & J.C. Pruessner. 2014. Reconsoli-dation of human memory: brain mechanisms and clinicalrelevance. Biol. Psychiatry 76: 274–280.

49. St. Jacques, P.L., D. Montgomery & D.L. Schacter. 2015.Modifying memory for a museum tour in older adults:reactivation-related updating that enhances and distortsmemory is reduced in ageing. Memory 23: 876–887.

50. St. Jacques, P.L., C. Olm & D.L. Schacter. 2013. Neural mech-anisms of reactivation-induced updating that enhance anddistort memory. Proc. Natl. Acad. Sci. U.S.A. 110: 19671–19678.

51. St. Jacques, P.L. & D.L. Schacter. 2013. Modifying memoryselectively enhancing and updating personal memories fora museum tour by reactivating them. Psychol. Sci. 24: 537–543.

52. Finley, J.R., W.F. Brewer & A.S. Benjamin. 2011. The effectsof end-of-day picture review and a sensor-based picture cap-

ture procedure on autobiographical memory using Sense-Cam. Memory 19: 796–807.

53. Silva, A.R., S. Pinho, L.M. Macedo, et al. 2013. Benefits ofsensecam review on neuropsychological test performance.Am. J. Prev. Med. 44: 302–307.

54. St. Jacques, P.L., D.C. Rubin & R. Cabeza. 2012. Age-relatedeffects on the neural correlates of autobiographical memoryretrieval. Neurobiol. Aging 33: 1298–1310.

55. St. Jacques, P.L., M.A. Conway & R. Cabeza. 2011. Genderdifferences in autobiographical memory for everyday events:retrieval elicited by SenseCam images versus verbal cues.Memory 19: 723–732.

56. Seamon, J.G., T.N. Moskowitz, A.E. Swan, et al. 2014. Sense-Cam reminiscence and action recall in memory-unimpairedpeople. Memory 22: 861–866.

57. Svoboda, E., M.C. McKinnon & B. Levine. 2006. The func-tional neuroanatomy of autobiographical memory: a meta-analysis. Neuropsychologia 44: 2189–2208.

58. Gilboa, A. 2004. Autobiographical and episodic memory—one and the same? Evidence from prefrontal activation inneuroimaging studies. Neuropsychologia 42: 1336–1349.

59. Gusnard, D.A., E. Akbudak, G.L. Shulman, et al.2001. Medial prefrontal cortex and self-referential mentalactivity: relation to a default mode of brain function. Proc.Natl. Acad. Sci. U.S.A. 98: 4259–4264.

60. Marchette, S.A., L.K. Vass, J. Ryan, et al. 2014. Anchoringthe neural compass: coding of local spatial reference framesin human medial parietal lobe. Nat. Neurosci. 17: 1598–1606.

61. Vann, S.D., J.P. Aggleton & E.A. Maguire. 2009. What doesthe retrosplenial cortex do? Nat. Rev. Neurosci. 10: 792–802.

62. Graham, K.S., A.C. Lee, M. Brett, et al. 2003. The neural basisof autobiographical and semantic memory: new evidencefrom three PET studies. Cogn. Affect. Behav. Neurosci. 3:234–254.

63. Vandekerckhove, M.M., H.J. Markowitsch, M. Mertens,et al. 2005. Bi-hemispheric engagement in the retrieval ofautobiographical episodes. Behav. Neurol. 16: 203–210.

64. Fink, G.R., H.J. Markowitsch, M. Reinkemeier, et al. 1996.Cerebral representation of one’s own past: neural networksinvolved in autobiographical memory. J. Neurosci. 16: 4275–4282.

65. Markowitsch, H.J., A. Thiel, M. Reinkemeier, et al. 2000.Right amygdalar and temporofrontal activation during auto-biographic, but not during fictitious memory retrieval.Behav. Neurol. 12: 181–190.

66. Milton, F., N. Muhlert, C. Butler, et al. 2011. The neuralcorrelates of everyday recognition memory. Brain Cogn. 76:369–381.

67. Milton, F., N. Muhlert, C. Butler, et al. 2011. An fMRI studyof long-term everyday memory using SenseCam. Memory19: 733–744.

68. Yonelinas, A.P., L.J. Otten, K.N. Shaw, et al. 2005. Separatingthe brain regions involved in recollection and familiarity inrecognition memory. J. Neurosci. 25: 3002–3008.

69. Diana, R.A., A.P. Yonelinas & C. Ranganath. 2007. Imagingrecollection and familiarity in the medial temporal lobe: athree-component model. Trends Cogn. Sci. 11: 379–386.



70. Mickes, L., P.E. Wais & J.T. Wixted. 2009. Recollection is acontinuous process: implications for dual-process theoriesof recognition memory. Psychol. Sci. 20: 509–515.

71. Slotnick, S.D. 2013. The nature of recollection in behaviorand the brain. Neuroreport 24: 663–670.

72. Squire, L.R., L. Genzel, J.T. Wixted, et al. 2015. Mem-ory consolidation. Cold Spring Harb. Perspect. Biol. 7:a021766.

73. Uncapher, M., J.T. Boyd-Meredith, J. Rissman, et al. 2014.Decoding the age and rehearsal history of real-world mem-ories. In Cognitive Neuroscience Society Annual Meeting,Boston, MA.

74. St. Jacques, P.L., M.A. Conway, M.W. Lowder, et al. 2011.Watching my mind unfold versus yours: an fMRI study usinga novel camera technology to examine neural differencesin self-projection of self versus other perspectives. J. Cogn.Neurosci. 23: 1275–1284.

75. Schacter, D.L., S.A. Guerin & P.L. St. Jacques. 2011. Memorydistortion: an adaptive perspective. Trends Cogn. Sci. 15:467–474.

76. Devitt, A.L. & D.L. Schacter. 2016. False memories with age:neural and cognitive underpinnings. Neuropsychologia 91:346–359.

77. Norman, K.A., S.M. Polyn, G.J. Detre, et al. 2006. Beyondmind-reading: multi-voxel pattern analysis of fMRI data.Trends Cogn. Sci. 10: 424–430.

78. Tong, F. & M.S. Pratte. 2012. Decoding patterns of humanbrain activity. Annu. Rev. Psychol. 63: 483–509.

79. Rissman, J. & A.D. Wagner. 2012. Distributed representa-tions in memory: insights from functional brain imaging.Annu. Rev. Psychol. 63: 101–128.

80. Rissman, J., H.T. Greely & A.D. Wagner. 2010. Detectingindividual memories through the neural decoding of mem-ory states and past experience. Proc. Natl. Acad. Sci. U.S.A.107: 9849–9854.

81. Alle, M.C., L. Manning, J. Potheegadoo, et al. 2017. Wear-able cameras are useful tools to investigate and remediateautobiographical memory impairment: a systematicPRISMA review. Neuropsychol. Rev. 27: 81–99.

82. Dubourg, L., A. Silva, C. Fitamen, et al. 2016. SenseCam: anew tool for memory rehabilitation? Rev. Neurol. 172: 735–747.

83. Silva, A.R., M.S. Pinho, L. Macedo, et al. 2016. A criti-cal review of the effects of wearable cameras on memory.Neuropsychol. Rehab. DOI: 10.1080/09602011.2015.1128450

84. Kelly, P., S.J. Marshall, H. Badland, et al. 2013. An ethi-cal framework for automated, wearable cameras in healthbehavior research. Am. J. Prev. Med. 44: 314–319.

85. Hodges, S., E. Berry & K. Wood. 2011. SenseCam: a wearablecamera that stimulates and rehabilitates autobiographicalmemory. Memory 19: 685–696.

86. Meyerhoff, H.S. & M. Huff. 2016. Semantic congruency butnot temporal synchrony enhances long-term memory per-formance for audio–visual scenes. Mem. Cogn. 44: 390–402.

87. Doherty, A.R., K. Pauly-Takacs, N. Caprani, et al. 2012.Experiences of aiding autobiographical memory using theSenseCam. Hum. Comput. Interact. 27: 151–174.

88. Harvey, M., M. Langheinrich & G. Ward. 2016. Remember-ing through lifelogging: a survey of human memory aug-mentation. Pervasive Mob. Comput. 27: 14–26.

89. Meegan, D.V. 2008. Neuroimaging techniques for memorydetection: scientific, ethical, and legal issues. Am. J. Bioeth.8: 9–20.

90. Bles, M. & J.D. Haynes. 2008. Detecting concealed informa-tion using brain-imaging technology. Neurocase 14: 82–92.

91. Meixner, J.B. & J.P. Rosenfeld. 2014. Detecting knowledgeof incidentally acquired, real-world memories using a P300-based concealed-information test. Psychol. Sci. 25: 1994–2005.

92. Agosta, S. & G. Sartori. 2013. The autobiographical IAT: areview. Front. Psychol. 4: 519.


https://doi.org/10.1080/09602011.2015.1128450

Documents

Neurocognitive mechanisms of real‐world autobiographical ...rissmanlab.psych.ucla.edu/rissmanlab/Publications... · autobiographical memory retrieval: insights from studies using