How I analyzed the data from my FitBit to improve my overall health.
Yash Soni / November 10, 2018
12 min read
Physical activity trackers have become a multimillion-dollar product category. I have had my fair share of fancy trackers, starting early from Nike Fuelband and then MI activity band. Personally, I couldn’t adopt any of them very well and eventually, they all ended up being a fancy digital watch which needed a re-charge every few days in order for them to tell the correct time.
And then, a few months back, I was gifted a Fitbit Versa by a friend of mine. Something clicked. Why?
A lot had happened since I gave up the idea of health bands being effective. When I received this gift, I was at a stage where I was glued to a chair and my laptop. As a result, I had a severe problem of back pain and postural imbalance. The wrongdoing of many years could not be fixed in just days or weeks. Fighting an arduous battle against sitting, which is now considered by many as the new ‘smoking’, I became mindful about my posture and the idea of maintaining an active lifestyle.
So, I thought of giving Fitbit a clean slate and tried it for a week. The Fitbit app did a pretty good job of capturing and showcasing the data in a consumable format.
It had been two weeks since my break up with my mechanical watch. Intrigued by the sheer variety of data points Fitbit could capture, an urge began to see what lies hidden behind the data.
I soon began an experiment to see how well I could keep up with the goals that I had defined. I also wanted to know if there were any extraneous factors which influenced these goals. Finally, I wanted to uncover any hidden and interesting findings along the way. I was particularly interested to discover:
How active are my days? Do I spend a considerable amount of time being sedentary?
How does this data vary on weekdays vs weekends?
What factors contribute to the highest calorie burn?
Which exercises are the best and easiest way to achieve my daily goals?
Have I been following a steady sleep schedule? What factors influence it?
Understand the sleep stages and find out what it takes to get a better deep sleep.
What is the impact of a Netflix binge on weekend sleep?
Train a simple Machine Learning model to see if there is a hidden pattern to attain better sleep.
I couldn’t find the answers to these from the standard Fitbit app. I needed the raw data.
Getting the Data
The first task was to figure out ways to extract the data from my device. Scanning through the developer pages, I found that they have a provision of Web API to access the user data. Examining these APIs, I was shocked to see the sheer amount of data that is being captured and saved every minute. Steps covered, calories burned, sleep stages and even heart rate/minute for any-given-day is being recorded!
Sometimes, the seductive lure of knowing about our general wellness makes us forget what personal information we end up sharing. Reading through their privacy policies, I found that Fitbit has put in additional checks to keep the data safe. Anyways this requires a separate post of its own, so without digressing from our main goal, let’s continue.
I registered my app and got the necessary client side credentials to begin the data grabbing. After going through the necessary authorization steps, I collected and merged my daily Activity, Sleep and Heart rate data and dumped it in an Excel file. After some data cleansing, the dataset was ready!
PS. The entire code can be found here along with the Jupyter notebook.
PPS. Disclaimer: This data analysis is based on a very limited set of data points and will be difficult to generalise to the masses. Please consider it as a fun read!
Fitbit has a wide array of data points to measure daily activity levels. Steps, Calories, and floors are some of the standard measures. It also tracks how many minutes I spend daily being moderately, slightly and very active.
Not fussing on the daily calorie burn, I had kept a goal of attaining 8000 steps each day on my Fitbit device. The graphs below suggested I am averaging about ~7800 steps per day which is quite close to my goal. There are some studies which suggest hitting 10000 steps per day is ideal and that shall be the next target.
Tuesdays to Saturdays were the days where I averaged about 40 minutes of *very active minutes — *which simply translates to active exercise. Fewer minutes on Sundays were due to purely due to laziness/recovery time. The drop in active minutes on Mondays proves that I am falling for the Monday Blues and guess it’s time to fix that. 💪🏼
Analysing the amount of calories burned per minute for various Activities shows some interesting findings. Though there is a lot of similar data available on internet, it is very difficult to generalise these numbers for everyone. Since a lot of this depends on the fitness levels, demographics, skillset and most importantly how much I enjoy doing some specific exercises.
It is interesting to see that running helps me burn almost 12 calories per minute. The math is simple: to compensate for a beer, a 10 minute run is what I need. 🏃🏻+ 🕑 = 🍺
Tennis 🎾 — the favourite activity from the lot — takes the second spot. That’s again a win-win scenario! It will be interesting to see if this number changes as I improve my skills.
Swimming numbers were not shocking to me because I am still struggling to keep up with my continuous lap count. And after spending some time in pool, the exercise turns to a leisure activity.
Point to note here is that calories burned should not be the only metric on which these activities can be graded. But, this happens to be the only metric which I can currently measure via Fitbit.
Lastly, it is useful to see how the various data points correlate with each other. Plotting a correlation heat-map helps uncover some findings.
The calories burned is strongly related to amount of steps and active minutes. Minutes sedentary has a negative correlation with Weekdays which implies that I spend more time slacking off on weekends.
Sleep is essential to helping maintain mood, memory, and cognitive performance and there is no running from it. We spend about a third of our life in sleeping. That’s a staggering 26 years spent sleeping in bed! While metabolism generally slows down, all major organs and regulatory systems continue to function. Hence, it becomes important to get the most out of our sleep.
Reading more on this, I found that there are some standard ways which can help achieve a good night sleep.
Avoid caffeine later in the day
Sleep in a cool and dark room
Getting at-least 7–9 hours of sleep. There are some studies which say that even in 5 hours you can achieve the most out of your sleep.
Over the course of this experiment, I tried to follow the above steps to bind myself to a strict sleep schedule. It was time to validate them.
From the graphs below, I found that I was averaging a sleep of 7 hours without much deviation in the numbers. Though I was able to hit bed before 11, the wakeup timings still ranged from 5:30–7:00 AM.
Though the average duration was somewhat similar, the overall sleep quality was not the same. On some days, I was very active even by attaining 6 hours of sleep, while there have been many instances when even after sleeping late, I wasn’t feeling fresh. I found the answer by analysing the mystic sleep cycles.
While we are asleep, our body typically goes through several sleep cycles, alternating between the following stages:
**LIGHT SLEEP: **This stage typically begins within minutes of falling asleep. Breathing and heart rate typically decrease slightly during this stage. Light sleep promotes mental and physical recovery.
**DEEP SLEEP: **Deep sleep typically occurs in the first few hours of sleep. Breathing becomes slower and muscles relax while heart rate usually becomes more regular. When we wake up feeling refreshed in the morning, it is likely that we have experienced solid periods of deep sleep. Deep sleep promotes physical recovery and aspects of memory and learning.
**REM SLEEP: **REM sleep is an active period of sleep marked by intense brain activity. The first phase of REM sleep typically occurs after an initial stage of deep sleep. Breathing is more rapid, irregular and shallow. The eyes move rapidly in various directions, hence the name Rapid Eye Movement — REM Sleep. This is the stage where we generally see dreams in our sleep. REM sleep has been shown to play an important role in mood regulation, learning, and memory.
The plot below shows that on an average, my body spends just about 17% in Deep sleep, 19% REM and the rest in either light or being slightly awake. The date-time plot of Light and Deep sleep shows that these numbers vary a lot.
If we plot the correlation of different sleep stages, we see that time spent in bed is highly correlated with Light sleep but there is no strong correlation with Deep sleep.
This essentially means that just by sleeping more doesn’t always guarantee a good deep sleep. I guess this helps validate the important learning about sleep:
It is the quality of sleep that matters and not the quantity 🔗
Following a strict sleep schedule on weekdays is easy, but weekends are a different ballgame altogether.
The boxplot above shows Saturdays are the most affected, where the time in bed ranges from 5–9 hours. The Netflix Binge and weekend parties are some of the vices which have affected this routine. On the contrary, the smaller boxplot on Sundays depicts me gearing up for the Monday mornings. It’s interesting to see how these subconscious body behaviours are clearly exposed in these plots.
Finally, I wanted to see if my daily activities had any effect on my sleep. Though I didn’t have much data for the Machine Learning model, an initial run showed some interesting results. It predicted that being active in the day and getting to bed before 11PM has some positive contributions to the final deep sleep minutes.
Though it is too soon to validate it, I will be repeating this again once I have some more sleep data and additional features to improve the accuracy of the model. The details can be found in this Jupyter Notebook.
Was it all worth it?
This experiment has been a rewarding experience. I identified some interesting ways in which my body responds to external stimuli. It is like a machine where tweaking certain knobs can help achieve different results.
Next up in line I plan to set up some improved activity goals. I will also be trying out some biohacks to see if these have any positive effects on my sleep quality. I am also considering developing a Fitbit alarm app which only wakes me up once I have acquired sufficient amount of quality sleep (not sure if this already exists?).
Finally, I do not intend to label this as an *experiment *anymore. The daily routine which felt forceful in the beginning has become a habit now. In the past, I’d come across many articles that were studies of the importance of waking up early as well, and finally I embraced it first-hand. This Medium article, which is in fact my first one, is one of the many byproducts of this new found habit.