Effect of emotional valence on episodic memory stages as indexed by event-related potentials ()
Keywords: Emotion; Memory; Familiarity; Recollection; Old/New Effect; Event-Related Potentials
1. INTRODUCTION
The presence of an emotional context plays an important role in memory, affecting both encoding and retrieval processes [1,2]. One way to study such factors is to relate recognition memory to the presence of emotionally evocative stimuli. Emotional stimuli can be represented by a two-dimensional space with one axis defined by the valence (from unpleasant to pleasant) and the other axis by arousal (from calming to exciting) [3]. Rather than assuming independent discrete emotional states (i.e. happiness, anger, etc.), a dimensional view of emotion assumes these two primary dimensions encompass the spectrum of emotional behavior. These dimensions of emotional arousal and valence have been operationalized through studies of the recall of photographic images depicting complex scenes (landscapes, animals, crime scenes, erotica, mutilations, car accident, etc.). Bradley, Greenwald, Petry and Lang [4] found that photographic images rated as highly arousing were recalled more accurately than low-arousal images. High-arousal images, encoded earlier in the experiment, produced faster reaction times than their low-arousal counterparts, with no additional effects due to valence. However, unpleasant pictures (high and low-arousal) delayed reaction times for the new pictures not encoded earlier. This finding suggests that both dimensions of arousal and valence are salient during initial encoding.
Indeed, brain-imaging studies have supported the hypothesis that short-term memory is enhanced in the presence of unpleasant compared to neutral or pleasant pictorial scenes [5]. An adaptation mechanism may favor aversive images through enhancement of retrieval and consolidation in memory, perhaps via the mediating role of the amygdala or other networks sustaining recognition memory [2]. With Event-Related Potentials (ERP), it has been shown that ongoing emotional stimuli are evaluated at several points in the information processing stream and the affective content could interact with many processes within a few hundred milliseconds after stimulus onset [6-9].
In recognition memory paradigms which included non-emotional material, it has been consistently shown that items previously presented (old) elicit a larger parietal ERP amplitude than the items not (new) previously presented [10-12] generating three topographically distinct “old/new” effects. First, an early (300 - 500 ms) bilateral, frontal effect occurs when the access to perceptual and conceptual information related to test items is facilitated [10,11,13,14]. Secondly, a left parietal oldnew effect (500 - 1000 ms) shows larger amplitude for deeply than to superficially encoded items. This parietal deflection reflects conscious recollection [15-19]. Following the parietal deflection, a late old-new effect (1000 - 1400 ms) normally arises just after the correct identification of the stimulus. Recent investigations propose that this effect is sensitive to the extrinsic (i.e. background) context in face recognition [20] and to a post-retrieval monitoring mechanism [19,21].
In the field of emotional memory, Maratos, Allan & Rugg [22] investigated these three recognition stages in relation to emotionally negative and neutral words. They showed that the left parietal old-new effect for negative words was smaller than that elicited by neutral words. Maratos et al. suggested that emotionally valenced words influence recognition memory mainly because of their higher levels of semantic cohesion, which leads to a tendency for false recollection of new negative items, diminishing the parietal old-new effect. In a similar study, Windmann and Kutas [23] proposed that emotional words could induce a propensity to respond “old” to a negative rather than to a neutral item, whether on not the item is actually old or new. In their investigations, the old-new ERP difference for correctly recognized items, was largely unaffected by negative words. So, regardless of emotional valence, the ERP associated with old words was characterized by a widespread positivity relative to that for correctly rejected new items. However, ERPs to hits (old) and false alarms (new) revealed a valence effect. While neutral items showed a large old/new difference, negative items did not. Windmann et al. [23] proposed that the frontal cortex might be responsible for relaxing the retrieval criterion for negative words so as to ensure that emotional events are not as easily forgotten as neutral events.
Two complementary conclusions emerge from these two ERP investigations. First, episodic retrieval is affected by the negative emotional valence of a stimulus. Second, the emotional negative valence seems to affect specific stages of memory processing while leaving other processes relatively intact. However, as noted earlier, ERP studies are in the unique position to inform brain imaging studies contrasting pleasant and unpleasant valence with neutral images by monitoring relevant recognition memory stages.
The goals of the present study were twofold. The first aim was to investigate whether recognition performances are affected by the presentation of emotional images when controlled for valence and arousal level ratings. The second goal was to consider whether the three recognition memory stages, as indexed by ERPs, are affected equally by emotional valence and arousal. The comparison between equally arousing pleasant and unpleasant images would clarify the separate influence of emotional valence and emotional arousal on specific memory stages. Based on previous data obtained with similar emotional images [4,24], we hypothesized that both valence and arousal would have an impact on recognition performances. Electrophysiologically, we also predicted that unpleasant images would mainly affect conscious recollection and, thus, reduce the left parietal old-new effect [22].
2. METHOD
2.1. Participants
Twenty right-handed female participants with normal or corrected-to-normal vision were selected. The mean age of the group was 25 years (range = 19 - 48 years old) with an average schooling of 16 years. Participants were administered the Raven test of non verbal intelligence and the California Verbal Learning Test (French versionCVLT) in order to validate and ensure normal functioning in general intelligence and verbal memory. All participants scored within the normal range according to published norms. All participants were recruited by announcements in the local media and were screened initially by telephone for suitability in terms of geographical accessibility, motivation to attend and absence of psychiatric or medical history. Only female participants were included in order to homogenize variance of our sample, since earlier studies demonstrated gender differences in affective processing and reported greater reactivity in women to aversive than pleasant pictures [24, 25].
2.2. Experimental Setting
On arrival at the laboratory, participants read and signed an informed consent form and received psychometric testing (CVLT and Raven test). EEG recordings were made in a dimly lit room where the participant was seated in an adjustable chair in front of the computer monitor. The recording room constituted a separate corner of a larger room in which the experimenters, amplifiers and computers were located. One experimenter gave instructions, while another experimenter was assigned to control the experiment with constant visual monitoring of the participant by means of a video camera. The nylon electrode cap (Electro Cap International), electro-oculogram and mastoid references were installed within 30 minutes. A one-minute resting baseline was recorded at the beginning of the experiment to facilitate laboratory adaptation.
2.2.1. Stimuli Selection
The emotional materials were photographic images from the International Affective Picture Systems (IAPS: Center for the study of emotion and attention [CSEA-NIMH], 1998), a standardized collection of images gathered from a wide variety of emotional and semantic categories.
A total of 150 photographic images were chosen and classified into three groups, based on the arousal and valence estimation from the IAPS normalization (50 unpleasants, 50 neutrals and 50 pleasants). The stimuli for the study phase included a total of 75 images. For the test phase, the lists included the 75 images of the study phase (old), plus 75 images that had not been presented (new) before. The images selected were classified into three basic categories, based on the IAPS female ratings of valence [unpleasant = 1 - 3; neutral = 4 - 6; pleasant = 7 - 9]. These 150 images (25 trials by 2 old/new response types by 3 valence categories—pleasant/unpleasant/neutral) were presented in different image orders to counterbalance effects due to sequence. In addition, for half of participants, the old/new order of presentation was inverted. Mean normative valence ratings from the IAPS were significantly different between valence estimation (F(5,144) = 353.02; p < 0.001). Multiple comparison post hoc tests (BONFERRONI) revealed that the valence was significantly different between neutral and unpleasant (p < 0.001), between neutral and pleasant (p < 0.001) and between unpleasant and pleasant (p < 0.001). These images also contained significant differences on arousal ratings (F(5,144) = 18.11, p < 0.001) and post hoc tests revealed that arousal ratings were significantly different between neutral and unpleasant (p < 0.001), between neutral and pleasant (p < 0.001), but not between pleasant and unpleasant images (p = 0.63). Finally, there were no significant differences between old and new categories across valence or arousal values (all p’s > 0.36). In each emotional category, the images contained the same basic attributes (scenes including humans, animals, inanimate objects or landscapes) across old and new category in order to preserve coherence across recall conditions.
The images were presented one at a time on a 17 SVGA monitor (Viewsonic), for a fixed duration of 4000 ms, at a distance of 90 cm calculated from the nose to the center of the computer screen with a 5 degree angle. They were presented at a resolution of 640 × 480 pixels in 256 colors without distortion between image presentations. The inter-trial interval (ITI) was fixed at 2000 ms during which a red and white checkerboard image appeared (IAPS #7182). This red and white checkerboard image informed the participant to fixate on a point between picture presentations and reduce the eye movements. This procedure also helped to reduce the after image effect, which occurred during presentation of a white blank background in our previous pilots.
2.2.2. Experimental Procedure
The experimental session began with a study phase during which the participants were instructed first to fix their gaze on a red and white checkerboard screen while waiting for the next images to appear. At that point, participants were told that a series of image presentations would be presented and that they should attend to each picture the entire time it appeared on the screen without giving any response. A short retention interval of 10 minutes was allowed between the study and the test phase. In the test phase, images were projected for the same duration and ITI as for the study phase. The participants were instructed to detect the images that were presented (old) during the study phase by a button press and also to identify the images that were not present (new) during the study phase by pressing another button. The reaction times were obtained with a three button device placed in front of the subject. They were instructed to emphasize both speed and accuracy in their responses. The emotional evaluation based on the Self Assessment Manikin (SAM) was administered after the ERP experimentation and the participants rated, by a paper and pencil response, each of the 150 images presented in a booklet. Previous brain imaging studies using emotional photographic images have shown that task instructions, prompting preparation for the processing of the evocative images, are susceptible to affect neural activity [26]. So, for both study and test phases, participants were not informed about the emotional value of the images beforehand in order to minimize emotional expectancy before the task and the paper and pencil was done post-test to keep the emotional nature of the task implicit during the experimentation.
2.3. EEG Recordings and ERP Extraction
The EEG was recorded from 26 tin electrodes mounted in an elastic nylon cap (Electro-Cap International Inc.) only during recall (test phase). The scalp electrodes were placed according to the guidelines for standard electrode position by the American EEG Society [27] at F7, F8, F3, F4, Fz, FC3, FC4, T7, C1, C2, C3, C4, Cz, T8, TP7, CP3, CP4, TP8, P7, P3, Pz, P4, P8, O1, Oz and O2. All electrodes were referenced to linked mastoids and their impedances were kept below 5 KW. The Electro-oculograms (EOG) was recorded using four 9-mm tin external bi-polar electrodes for horizontal and vertical movements. For the horizontal EOG, electrodes were placed at the outer canthus of each eye and for the vertical EOG at infra and supra-orbital points at the left eye, aligned with the pupil looking straight. A bioelectric analog amplifier model ISS3-32BA (SAI-InstEP) amplified the EEG signals (EOG gain = ±10,000 and EEG gain = ±20,000) with a band-pass between .01 and 30 Hz. The EEG was recorded continuously at a sampling rate of 250 Hz and averaged offline in a time-window beginning at 100 ms before and until 1900 ms after picture onset. The EOG artifact contained in the EEG were corrected with a dynamic multiple regression in the frequency domain [28]. The regressions were applied using the horizontal and the vertical EOG activity subsequently. After EOG corrections, all remaining epochs with a voltage exceeding ±100 uV and clippings due to saturation or blocking of the amplifiers were eliminated automatically during the averaging procedure. On average, 2.5 trials per condition were rejected, after EOG corrections, because of the remaining artifacts (range = 0 - 5 trials). To exclude a possible residual effect of the EOG on the EEG and ERPs, ANOVAs, applied to the number of artifact rejected, failed to show any significant effect across response type and emotional valence conditions (all p’s over .30). A second ANOVA applied on the two EOGs separately also failed to reach any statistical significance according to response type or valence (all p’s over .10). A minimum amount of 16 trials free of both errors (false alarms and misses) and artifacts were included in the ERP averaging, which is comparable to the criteria used in similar ERP experiments [22,23,29-31]. Finally, four time windows were defined as the early (300 - 500 ms), middle (500 - 700 ms, 700 - 1000 ms) and late latencies (1000 - 1400 ms). Our experimental hypotheses were tested using the mean amplitudes of the ERP detected within the temporal windows as defined in previous recognition memory research [15,32].
2.4. Statistical Analyses
The behavioral data comprised the subjective affective ratings, median Reaction Times (RT), old/new discrimination accuracy (Pr), hits, False Alarms (FA) and response bias (Br). The Pr was computed by the following subtraction: Pr = Hit-FA where “Hit” represented the probability of “old response’’ to an old item and “FA” (False Alarm), the probability of “old response” to a new item. The Br was calculated on the basis of the FA and Pr as: Br = FA/(1 − Pr) according to the two-high threshold theory (Snodgrass & Corwin, 1988). Mean ERP amplitudes for each time-window were collapsed across electrode sites to form the anteriority (anterior vs posterior) and the hemispheric (left/right) factors. The anterior electrodes were composed of F7, F3, FT7, FC3, T3, and C3 for the left hemisphere and F8, F4, FT8, FC4, T4, and C4 for the right hemisphere. The posterior electrodes included TP7, CP3, T5, P3 and O1 for the left hemisphere and TP8, CP4, T6, P4 and O2 for the right hemisphere. The behavioral and ERP data were submitted to a repeated measure ANOVA (SPSS-Windows ver 10.0). The behavioral data was submitted to an ANOVA with RESPONSE TYPE (with two levels; hits/correct rejection), and EMOTIONAL VALENCE (with three levels; pleasant/unpleasant/neutral). The analysis of the ERP data contained two additional within-subject factors related to cortical regions of ANTERIORITY (with two levels; anterior/posterior) and HEMISPHERE (with two levels; left/right). Analyses were carried out separately on each of the four temporal windows and additional post hoc tests were computed for multiple comparisons. There were insufficient trials to average EEG signals related to errors (misses and FAs). In all analyses the significance level was set at 5% (two-tailed) with Greenhouse-Geisser corrections for degrees of freedom where necessary.
3. RESULTS
3.1. Behavioral Data
3.1.1. Valence and Arousal Ratings
An ANOVA on the valence ratings showed significant differences between the three categories (F(2,38) = 310.42; p < 0.001). Multiple comparisons (Bonferroni) post hoc tests showed that the mean valence rating was significantly different between neutral and unpleasant (mean difference = 2.39; p < 0.001), between neutral and pleasant (mean difference = 2.14; p < 0.001) and between pleasant and unpleasant (mean difference = 4.45; p < 0.001). Thus, our participants’ valence rating scores were consistent with the IAPS’s standard scores where differences between IAPS and our results were 0.05 for the pleasant, 0.02 for the neutral and 0.33 for the unpleasant (see Table 1).
An ANOVA on the arousal ratings also showed significant differences between the three categories (F(2,38) = 177.30; p < 0.001). Multiple comparisons (Bonferroni) post hoc tests showed that the mean arousal rating was significantly different between neutral and unpleasant (mean difference = 3.09; p < 0.001), between neutral and pleasant (mean difference = 1.17; p < 0.001) and between pleasant and unpleasant (mean difference = 1.92; p < 0.001). Our participants’ arousal ratings were consistent with the IAPS’s standard scores for the pleasant (difference = 0.2) and neutral (difference = 0.2), but not for the unpleasant where they gave higher arousal values than the IAPS (difference = 1.2) (see Table 1).
3.1.2. Discrimination Accuracy, Response Bias and Reaction Times
Hits and correct rejection rates for the unpleasant, pleasant and neutral items along with discrimination and response bias indices are shown in Table 2. The Old/new discrimination accuracy (Pr) was affected by emotional valence (F(2,38) = 6.64, p < 0.01). Multiple comparisons (BONFERRONI) revealed that the Pr was smaller to pleasant than neutral (p < 0.05) conditions and also when contrasting pleasant with unpleasant (p < 0.05), but no significant difference emerged between unpleasant and neutral images. The response bias (Br) consistently showed a significant emotional valence effect (F(2,38) = 10.49, p < 0.001). Multiple comparisons (BONFERRONI) revealed that the Br was higher for pleasant than for neutral (p < 0.001) images and for unpleasant than for neutral (p < 0.005) images, but no significant difference emerged between unpleasant and pleasant images.
Reaction Times (RTs) data are shown in Table 2. An ANOVA applied to median RTs revealed a main effect of response type (F(1,19) = 10.30, p < 0.01), emotional valence (F(1,19) = 10.46, p < 0.001) and an interaction between these factors (F(2,38) = 5.99, p < 0.01). Multiple comparisons (BONFERRONI) revealed that hits (old) were identified significantly faster (955 ms) than correctly rejected (1018 ms) images. RTs to unpleasant images were delayed (1007 ms) in comparison with neutral (985 ms) and pleasant (968 ms) images. Further paired comparisons revealed that the response type effect was significantly larger in both unpleasant (t(19) = −4.48, p <
Table 1. Comparison of the mean “valence” and “activation” between IAPS female norms (Center for the study of emotion and attention [CSEA-NIMH], 1998) and our participants evaluations.
0.001) and pleasant (t(19) = −2.24, p < 0.05) in comparison with the neutral (t(19) = −1.88, p = 0.07).
3.2. Electrophysiological Data
To assess amplitude differences between conditions, ANOVAs were carried out for each latency window separately. Figure 1 shows the grand average of the ERPs (n = 20) for correctly recognized old and new images for all electrode locations and conditions. Table 3 presents all significant interactions for the four ERP intervals during the test.
3.2.1. Early Old/New Effect (300 - 500 ms)
As can be seen in Table 3 and Figure 2(a), analysis of the early latency period (300 - 500 ms) revealed a significant main effect of response category and anteriority. A significant response type by anteriority by valence interaction was also found. In order to further examine the nature of these interactions, two subsidiary ANOVA was carried out separately on anterior and posterior regions. The first analysis, applied to the anterior region, revealed a response type main effect (F(1,19) = 25.18, p < 0.001) expressed as a more negative amplitude to new (−2.80 uV) than old (−1.16 uV) images. The emotional valence factor failed to reach significance at the anterior region. The second analysis, applied to the posterior region, revealed a significant main effect of response type (F(1,19) = 9.55, p < 0.01), hemisphere (F(1,19) = 7.45, p < 0.05) and an interaction between these factors (F(1,19) = 8.61, p < 0.01). This interaction was expressed by a larger response type effect over the left (t(19) = 3.53, p < 0.005) than over the right (t(19) = 2.27, p < 0.05) hemisphere. An interaction between response types and emotional valence was also present (F(2,38) = 3.96, p < 0.05). Further ANOVAs, computed separately for each valence class, showed that the effect of response type was significant only for the pleasant category (pleasant: (F(1,19) = 16.09, p < 0.001; unpleasant: (F(1,19) = 0.001, p = 0.97; neutral: (F(1,19) = 1.19, p = 0.29).
3.2.2. Left Parietal Old/New Effect (500 - 700 ms)
This latency window (500 and 700 ms) is depicted in
Table 2. Probabilities of correct responses to old (hits) and new (correct rejection) items, Reaction Times, and measures of old and new discrimination, false alarms and response bias.
(a)
(b)
(c)
Figure 1. (a) Grand average of ERP waveforms elicited by correctly classified old (bold) and new (dotted) unpleasant images; (b) Grand average of ERP waveforms elicited by correctly classified old (bold) and new (dotted) pleasant images; (c) Grand average of ERP waveforms elicited by correctly classified old (bold) and new (dotted) neutral images.
Table 3. Summary of ANOVAs results during the four latency windows. Significant interaction effects are reporteda.
Figure 2(b) for all conditions. A significant main effect of response type and anteriority along with several interactions involving the hemisphere and the valence factors was shown (Table 3). In order to assess the reliability of the old-new response type effect, elicited by each class of emotional valence, separate ANOVAs were applied to emotional categories separately. The analysis revealed a main effect of response type in each emotional condition (unpleasant: F(1,19) = 5.19, p < 0.05; pleasant: F(1,19) = 33.08, p < 0.001; neutral: F(1,19) = 18.32, p < 0.001). An interaction between response types and anteriority was also present for the pleasant (F(1,19) = 33.08, p < 0.001) and neutral category (F(1,19) = 4.39, p < 0.05), indicating that the old-new effect in the pleasant and neutral category was maximal at posterior regions. A separate ANOVA focusing on the posterior region revealed a significant interaction between response type, valence and hemisphere (F(1.98,37.71) = 8.06, p < 0.001). Separate ANOVAs, calculated for left and right posterior regions, revealed a main effect of response type in both hemispheres (left: F(1,19) = 35.53, p < 0.01; right: F(1,19) = 30.29, p < 0.01) while the emotional valence main effect