Comparison of Fitness Tracking Using Three Different Smartwatches during Free Activities in Daily Life ()
1. Introduction
Women of reproductive age experience psychosomatic modulation associated with the menstrual cycle, and feel stress on a daily basis due to this psychosomatic modulation. Premenstrual syndrome (PMS) is one of the accompanying symptoms associated with menstruation. Symptoms of PMS include mental symptoms such as irritability, depression, anxiety, and difficulty concentrating, as well as physical symptoms such as abdominal pain and headache [1] . PMS has been reported to result in decreased ability to perform tasks [2] and decreased performance in schoolwork and exercise [3] [4] [5] . Cases in which the mental symptoms of PMS are stronger and affect daily life or interpersonal relationships are referred to as premenstrual dysphoric disorder (PMDD) [1] [6] . Therefore, women must conduct monthly stress monitoring and management of physical and mental symptoms that coincide with the menstrual cycle.
Measuring autonomic nerve activity is one method of stress monitoring [7] . Autonomic nerve activity is assessed based on heart rate variability. This heart rate variability is easily changed by physical actions such as breathing and standing, as well as mental load and sleep [8] . Menstrual cycles and psychological stress have also been reported to be associated with autonomic nervous activity [9] . It has been reported that women with PMS or PMDD have more altered autonomic activity during the luteal phase than in the follicular phase [10] [11] . Furthermore, women with PMS have been reported to have increased heart rate during sleep and decreased high frequency, which represents parasympathetic nervous system activity, during the luteal phase [12] [13] . Therefore, autonomic nerve activity is thought to be an indicator for capturing psychosomatic modulation in the menstrual cycle.
Wearable devices incorporating photoplethysmography (PPG) technology have been developed in recent years. PPG can detect changes in blood volume in tissue microvascular beds and measure cardiovascular pulse waves [14] . Previous research has defined the pulse wave as measured by PPG as the heart rate [15] . Autonomic nerve activity can be easily measured using PPG technology, as it enables the monitoring of heart rate, blood pressure, and oxygen saturation simply by wearing a device on the earlobe or fingertip without needing to attach a large device such as an electrocardiogram to the body. Smartwatches are an example of wearable devices equipped with PPG, as well as accelerometers and triaxial sensors. As a result of these sensors, smartwatches act not only as timekeeping devices, but also fitness trackers for information such as steps taken, energy expended, heart rate, sleep patterns, and stress levels [16] [17] . Smartwatches are convenient and allow for continuous monitoring in daily life. Furthermore, due to the real-time feedback they provide, they are useful as tools for health management [18] [19] [20] . Based on the above, smartwatches are expected to be developed as devices that allow users to support their own health management [16] .
There are many studies on the reliability and validity of fitness tracking in smartwatches using actual laboratory measurements [15] [17] [21] [22] . These previous studies have shown that smartwatch heart rate measurements have high accuracy. Meanwhile, fitness tracking, which contributes to health management, also includes calculations of stress levels, number of steps, and distance traveled [16] [23] . These fitness tracking measurement and calculation methods are ambiguous [24] , so they are thought to differ by model. Furthermore, smartwatches are used in daily life. Therefore, it is necessary to compare fitness trackers during free activities in daily life and verify their accuracy. Moreover, to the best of our knowledge, no studies have compared multiple smartwatch models worn simultaneously during free activities in daily life. Smartwatches are also promising devices for assessing and managing women’s menstrual symptoms in daily life. However, few studies have examined the validity of fitness trackers in sexually mature women.
The purpose of this research is to compare the heart rate, stress level, sleep time, number of steps, and distance traveled that are calculated from the fitness tracking in several different smartwatches during free activities in daily life at women of reproductive age.
2. Methods
2.1. Participants
The participant sample size was calculated by G*power. The significance level α was set to 0.05, the power 1 − β was set to 0.8, and the effect size d was set to 0.5. As a result, 27 participants were required.
The participants were healthy women in their 20 s to 30 s. The inclusion criteria were having a BMI of higher than 18.5 and less than 25, and being able to answer questions in Japanese. A BMI of 18.5 to 25 is a normal body mass. Approximately 70% of his 20 - 30 s women are within the BMI range of 18.5 - 25 [25] . The exclusion criteria were having a smoking habit, having an underlying disease, and using drugs that influence autonomic nerve activity. Stress level is calculated from heart rate variability. Smoking, including e-cigarettes, affects autonomic nervous system activity [26] . Autonomic activity is known to be affected by oral medications such as antidepressants [27] . The participants were selected by checking with the physicians enrolled in the study group regarding their oral medications.
2.2. Recruitment Method
We recruited research participants through the research group’s website. A summary of the research was posted on the website. The participants who were interested in the research registered their e-mail addresses in the application form created using Google Forms. This application form also included a screening item that confirmed the participant’s selection criteria.
The researchers contacted the participants via the e-mail addresses collected through Google Forms. An overview of the research was explained at a later date to the participants face-to-face, and the participants signed a consent form when consenting to participate in the research. Surveys were conducted among those from whom a consent form was obtained.
2.3. Survey Procedure
The survey was conducted from December 2021 to June 2022.
The following three smartwatch models were used in this survey: Mi smartband 6 (Xiaomi Corporation Inc., China), vivosmart®4 (Garmin Ltd., USA), and Band 6 (Huawei Technologies Co., Ltd., China). The participants wore the three smartwatches simultaneously on their wrists for 48 hours continuously. They did not remove their smartwatches while bathing or sleeping. The smartwatch wearing method was left up to the discretion of the participant, so the smartwatch wearing order also differed by participant. Many of the participants wore the three smartwatches on the wrist of their non-dominant side. Some participants wore two smartwatches and one smartwatch on each wrist.
In order to take measurements in an environment representing everyday life, surveys were conducted while avoiding days of competitions or events. The participants were allowed to continue activities that they regularly participated in, such as extracurricular and club activities; their habitual exercises were not restricted. The participants recorded activities such as eating, exercising, and learning on an activity chart.
2.4. Equipment Used
2.4.1. Activity Chart
We created activity charts with hourly divisions from 0:00 to 24:00 so that the participants could fill in their activities by hour. The participants recorded the type of activity at the time of the activity, such as eating, exercising, or learning. We told the participants to record the activity on the activity chart immediately. For exercise, we turned on the activity tracker that automatically measures exercise on the smartwatch. We checked the exercise measured by the smartwatch against the exercise listed on the activity chart. In the study, no participants removed the smartwatch in 48 hours.
2.4.2. The Following Three Smartwatch Models Were Used
The smartwatch was used in this study because it can store data for up to one week. We synchronized smartwatches data after the study was completed.
1) Mi smartband 6
Mi smartband 6 weighs 12.8 g and measures 47.4 mm × 18.6 mm × 12.7 mm. The display size is 1.56 inches and is AMOLED. The battery life is up to 14 days. It is waterproof for daily life activities.
After the data were synchronized with Mi Fit, which is a dedicated application, the data were converted to a CSV file. Mi Fit has a section called Export Data. It is possible to extract activity data other than stress level. For stress level, we extracted what is displayed on the application.
2) vivosmart®4
vivosmart®4 weighs 16.5 g and measures 15 mm × 10.5 mm × 19.7 mm. The display is equipped with OLED. The battery life is up to 7 days. It is waterproof for daily life activities.
The data were synchronized with Garmin connect, which is a dedicated application.
Afterwards, the Fit file was downloaded from the web browser version of Garmin connect and converted to a CSV file. The activity data used in this study could be extracted by converting Fit files to CSV files.
3) Band 6
Band 6 weighs 18 g and measures 43 mm × 25.4 mm × 10.99 mm. The display size is 1.47 inches and is AMOLED. The battery life is up to 14 days. It is waterproof for daily life activities. The data were synchronized to HUAWEI Health, which is a dedicated application, and extracted.
It was not possible to extract the activity data directly from the CSV file. Therefore, for Band 6 data, we only used the data displayed on the HUAWEI Health application. Heart rate and stress level, the time and measured values could be checked on the application.
All extracted data were graphed and visually verified to ensure that the graphs were similar to the application.
2.5. Data Collection
For the participant attributes, we obtained the age, height, weight, and oral medications of the participants.
We extracted and analyzed five datasets from the data measured by each smartwatch: heart rate, stress level, number of steps, distance, and sleep time.
For stress level, number of steps, distance, and sleep time, the daily average values calculated by each smartwatch were used.
2.6. Analysis Method
For the comparison of the heart rates between the three smartwatches, we used Spearman’s rank correlation coefficient ρ. Next, we created Bland-Altman plots to visually examine the heart rate agreement between the smartwatches. We calculated the mean difference and lower to upper limits of agreement for the heart rate. The lower to upper limits of agreement were calculated by mean ± 1.96 standard deviation (SD).
In order to compare the fitness trackers of each smartwatch, we calculated the mean absolute percentage error (MAPE). In this research, we targeted the five activity data of rest heart rate, stress level, sleep, steps, and distance, which could be measured and calculated by the three smartwatches. The mean error rate was determined as the absolute value of the true value minus the measured value. The true value for MAPE was the average of the data calculated by the three smartwatches. For the sleep time, bed time and wake time were entered in the activity chart, so the sleep time obtained from the activity table was taken as the true value. The absolute value obtained by subtracting the data calculated by the smartwatch from this true value was divided by the true value and multiplied by 100 in order to calculate the MAPE [17] .
Statistical analysis was conducted using IBM statistics SPSS ver. 29, with the significance level set to 5%.
2.7. Ethical Considerations
This research was approved by the ethics committee of Tsukuba of University Medical School (approval number 1684). The research was explained to the subjects, and after written consent was obtained from them, we conducted the survey.
3. Results
3.1. Basic Attributes of Research Participants
There were 25 applications for research participation. Of these, two declined to participate in the research, so 23 people were set as analysis participants.
The average age of the participants was 22.9 (±2.4) years. The average BMI was 20.8 (±2.1). Among the participants, five were taking oral contraceptives, and two were taking anti-allergy drugs.
3.2. Comparison of Heart Rate during Free Activities in Daily Life
We calculated the daily heart rate measured by the three smartwatches and compared them using Spearman’s rank correlation coefficient ρ.
A very strong correlation was observed between Mi smartband 6 and vivosmart®4 (ρ = 0.828, p < 0.001) “Figure 1(a)”. A strong correlation was observed between vivosmart®4 and Band 6 (ρ = 0.684, p < 0.001) “Figure 1(b)”. Similarly, a strong correlation was observed between Mi smartband 6 and Band 6 (ρ = 0.675, p < 0.001) “Figure 1(c)”.
Next, Table 1 and Figures 2(a)-(c) show the mean difference and lower and upper limits of agreement of the Bland-Altman plots that compare each smartwatch for each pair of models. The mean difference between Mi smartband 6 and vivosmart®4 was the lowest at −1.74. This indicates that the heart rate error was small.
3.3. Comparison of Smartwatch Fitness Trackers
Figure 3 shows changes in heart rate and stress level calculated by each of a participant’s (ID-20) smartwatches for 24 hours. The time not calculated by the smartwatch was left blank.
Table 2 shows the daily resting heart rate, sleep time, stress level, number of steps, distance traveled, and standard deviation for the fitness trackers calculated
![]()
Table 1. Bland-Altman analysis of heart rates measured by three smartwatches.
Lower to upper limits of agreement: mean difference ±1.96SD; SD: standard deviation.
![]()
Table 2. MAPE for fitness tracking calculated by three smartwatches.
MAPE: Mean absolute percentage of error; SD: standard deviation; rest HR: rest heartrate.
![]()
(a)![]()
(b)![]()
(c)
Figure 1. (a) Heart rate correlation between Mi smartband 6 and vivosmart®4. Spearman’s rank correlation coefficient (ρ = 0.828, p < 0.001); (b) heart rate correlation between vivosmart®4 and Band 6. Spearman’s rank correlation coefficient (ρ = 0.684, p < 0.001); (c) heart rate correlation between Mi smartband 6 and Band 6vivosmart®4. Spearman’s rank correlation coefficient (ρ = 0.675, p < 0.001).
by the three smartwatches. For the sleep time, vivosmart®4 had a high value at 6.58%. This was because vivosmart®4 had a high sleep time error compared to the other two models. Next, for the resting heart rate, Band 6 had the lowest MAPE at 4.77%. This shows that the error with the average resting heart rate calculated by the three smartwatch models was small. For the stress level, Mi smartband 6 had a mean error of 55.22%. This shows that Mi smartband 6 had a high stress level error compared to those in vivosmart®4 and Band 6. For the number of steps, Mi smartband 6 had an error of 17.80%, which was a small
![]()
(a)![]()
(b)![]()
(c)
Figure 2. (a) Bland-Altoman plot: Mi smartband 6 vs. vivosmart®4; (b) Bland-Altoman plot: vivosmart®4 vs. Band 6; (c) Bland-Altoman plot: Mi smartband 6 vs. Band 6.
![]()
![]()
![]()
Figure 3. Heartrate and stress level calculated by each smartwatch.
value among the three smartwatch models. Similarly, the distance traveled was 18.25% for Mi smartband 6, which had a small error compared to the two other models.
4. Discussion
In this study, we compared and examine fitness trackers measured by several smartwatches under free activities in daily life.
4.1. Basic Attributes of Research Participants
The average BMI for women in their 20s is 21 in Japan [25] . The BMI can be said to be representative of Japanese women. Five participants were taking oral contraceptives. Two participants had hay fever. In Japan, cedar and cypress pollen increases in spring. The survey was conducted from December to June, which is winter and early summer. Therefore, the participants were taking anti-allergic medication internally. These oral medications are common. The participants of this study are considered to be similar to the general population of Japanese women.
4.2. Association between Heart Rate Calculated by Each Smartwatch in Daily Life
The participants wore the smartwatches continuously for 48 hours, and we compared the heart rates measured by the smartwatches during their free activities. Mi smartband 6 and vivosmart®4 were comparatively able to calculate the heart rate every minute. Meanwhile, Band 6 calculated the heart rate once every 5 - 10 minutes. Looking at the correlations between the heart rates calculated by each smartwatch at the time the heart rate was measured, we found a very strong correlation between Mi smartband 6 and vivosmart®4. When visually looking at changes in the heart rate, vivosmart®4 and Mi smartband 6 had sections where the heart rates were not in agreement. This was the same result as in a previous study that compared heart rate accuracy in laboratory environments. In that study, the device manufactured by Polar was used as a gold standard, and it was shown that devices manufactured by Xiaomi and Garmin had a generally accurate PPG [15] . Meanwhile, the agreement between vivosmart®4 and Band 6 tended to be high in the sections where the Mi smartband 6 and vivosmart®4 were not in agreement. According to a previous study that compared various models, the device manufactured by Xiaomi had the lowest heart rate accuracy [17] . In previous research, the Mi smartband 2 was used, and the model used in the present research was an improved version of this. Therefore, it is thought that the PPG accuracy improved compared to the device used in previous research.
From the correlation coefficient, vivosmart®4 and Band 6 had a slightly higher heart rate agreement (ρ = 0.684) than Mi smartband 6 and Band 6 (ρ = 0.675). It can also be seen from the Bland-Altman plots that the mean difference was lowest between Mi smartband 6 and vivosmart®4. Each of the Bland-Altman plots showed high agreement between Band 6, Mi smartband 6, and vivosmart®4. It can be said from these results that the heart rate measurement method under free movement was consistent. This is similar to previous research that compared various device models [15] [22] . Therefore, the smartwatches used in the present study are able to measure heart rate at the same level even under free movement in daily life.
4.3. Comparison of Fitness Tracking under Free Activities in Daily Life
In this research, we calculated the MAPE, using the average value of the three smartwatch models as the gold standard. Results showed that the Band 6 manufactured by Huawei was the closest to the average of the three models. However, Huawei has a low agreement of continuous heart rate, and vivosmart®4 tended to underestimate heart rate more than Mi smartband 6, so it is thought that Band 6 was the average value of smartband 6 and vivosmart®4. In other fitness tracking indices, the stress level value in the Xiaomi device was the biggest outlier. When confirming the changes in daily activity, heart rate, and stress level, vivosmart®4 calculated the stress level most often, and the stress level was calculated when the heart rate was calculated. The heart rate of Mi smartband 6 was measured to the same extent as vivosmart®4, but the stress level was calculated irregularly. Band 6 calculated the stress level when the heart rate could be measured continuously. Previous research has also not reported consistent results on the validity of stress levels, and the algorithms in these devices for calculating stress levels are unclear [23] . A goal in stress monitoring using wearable devices is the generalization of stress models using various forms of machine learning [28] . There have also been studies on psychological and physical stresses [29] [30] , but further research on integrating these stress models is needed. In recent years, there have been not only linear analyses of autonomic nerve activity such as time domain analysis and frequency domain analysis, but also changes in nonlinear heart rate variability measurements [31] . This nonlinear heart rate variability has been studied using accelerometer and gyroscope signals [32] . These sensors are also installed in smartwatches, and it is assumed that future measurements of stress conditions will be from heart rate variability using these sensors. Examining the validity of stress levels requires capturing the autonomic nerve activity from linear and nonlinear analyses for a specified stress load and formulating a model equation that can calculate stress.
Research that examined the fitness tracking of sleep, rest heart rate, steps, distance, calories, and sleep in smartwatches manufactured by Huawei and Xiaomi reported that the MAPE of Xiaomi-manufactured devices was small [17] . In the present research, we showed that Mi smartband 6 had values closest to the average, with the exception of the stress level values. This is thought to be due to the fact that the Xiaomi-manufactured device took the average of data that calculated the fitness tracking in the Garmin- and Huawei-manufactured devices. It has been reported that the heart rate accuracy during cycling of Garmin-manufactured devices was lower than that of Xiaomi-manufactured devices [15] . Meanwhile, research that examined accuracy using the Mi band 4 manufactured by Xiaomi showed that the accuracy of the number of steps or heart rate under free activities was relatively high, and their usability in daily life has been proven [33] .
Presently, various smartwatches have been developed, and they have an influence on fitness tracking, such as data accuracy and user behavior [19] [24] . Wearers need to select a smartwatch that suits their living environment.
4.4. Limitations and Future Issues of This Study
This study has several limitations. The first is that the target population is not a representative population. In this study, we recruited research participants through the research group’s website. The participants were students at the university where the research group is located. Therefore, the target population is limited. In order to ensure a diverse and representative sample, it will be effective in the future to use methods such as social media and advertising [34] . In the future, our research should examine ways to collect samples using these methods.
The second is that this research focused on women but did not consider their menstrual cycles. It has been reported that autonomic nerve activity changes according to the menstrual cycle [4] [9] ; therefore, in the future, we will need to create a research protocol that considers the menstrual cycle.
The third is that we were unable to examine fitness tracking other than heart rate with sufficient accuracy. We did not set an exact gold standard, since we calculated MAPE from the true value obtained from the fitness tracking of the three smartwatches and its error. As such, there is a need to set and examine the gold standard for sleep time, and to use pedometers and distance measurements for the number of steps and distance accuracy. In the future, there is a need to verify the accuracy and validity of physical activity using the accelerometers and triaxial sensors installed in smartwatches.
5. Conclusions
In this study, we compared the fitness tracking of different smartwatches under free movement in daily life. Results showed that there was high agreement in the measured values of heart rate calculated by several different smartwatches. Therefore, it is suggested that the smartwatches used in this study were able to accurately measure the heart rate under free movement in daily life. The calculation method for the stress level differed in the fitness tracking of the three smartwatches, and vivosmart®4 had calculated the stress level the most often. Furthermore, Band 6 calculated the stress level when the heart rate could be measured continuously. The error was similar between vivosmart®4 and Band 6; however, the Mi smartband 6 calculated the least amount of stress level data. The stress level in the Mi smartband 6 had a large error compared to the other two models.
Smartwatches are equipped with not only PPG but also various sensors such as triaxial sensors and accelerometers. In the future, there is a need to examine the accuracy, validity, and effectiveness of fitness tracking calculated by these sensors for each model based on specific physical activities.
Acknowledgements
This research was funded by JST SPRING grants (JPMJSP2124), and Grant-in-Aid for Scientific Research (C) (20K10574) from the Japan Society for the Promotion of Science (JSPS).