Background: Subjective well-being is a central concept of positive psychology, and is directly coupled with a high level of positive affect and a low level of negative affect. Positive affect is associated with enthusiasm, activity, hope and inspiration, while negative affect is associated with emotions such as anger, contempt, guilt, shame, fear, anxiety, depression, and stress. Physical activity is crucial for both physical and mental health and is positively associated with well-being. Gender and social factors (e.g., parenthood or partnership) have complex relations with well-being and affect. In the present study we aimed to 1) examine the association between leisure-time physical activity and affect and 2) investigate whether or not social factors interact with this association. Method: The study included information from 155 Swedish university students: 64 men (mean age 23 years) and 91 women (mean age 27 years). Students were asked to estimate their usual engagement in physical activity during their leisure-time by responding to the Godin Leisure-Time Exercise Questionnaire. They also reported the level of positive and negative emotions experienced during past weeks by completing the Positive Affect and Negative Affect Schedule instrument. Results: In the Swedish student population leisure-time physical activity correlated only with positive and not with negative affect. Students’ gender, age or whether or not they had children did not influence this association. However, this correlation differed significantly between those who lived with a partner and those who were single. Conclusion: Leisure-time physical activity is positively correlated with an overall subjective well-being, and this correlation is modified by the social factor of partnership.