Collecting Data I Do Not Yet Know How To Use

If we ask questions – as Socrates warns us we must – eventually our ability to answer them becomes limited by the data we have available. If we wait until we have the question to start collecting data, it will take us some amount of time to get baseline data and then even more to measure change during experiments. We won’t have our answer for some time – weeks, months or even decades. We might never get an accurate answer.

This is why we must record data that we do not yet know how we will use. Some day we will have a question that these data can answer. This is why we must bear the burden of recording and storing information. Some day an important question will be quickly answered because of the hard work we put in now.

Here is a story to highlight one recent, though not all that important, example in which I experienced this.


In 2015 I started tracking how frequently I participate in various hobby activities as a way of measuring the enjoyability and balance of the life I have created, and am creating for myself.

It is more precise, and thus less overwhelming, for me to make the statement “I feel happiest when I surf at least 12 times a year but I’ve only surfed 8 so far” than to deal with some ambiguous emotional statement about “not felling like I surf enough anymore”. The former is actionable – creating the opportunity for four surfing sessions is fully possible by booking a week vacation on whichever coast/island is getting waves in the next month. Problem solved.

The decision I periodically consider is whether 12 times a year is really cutting it and if not, what would be required to increase that number. The two biggest things I hypothesize are limiting my frequency of surfing are accessibility and other priorities – mainly the fact that I have a job and three kids.

I am easily able to test how limiting other priorities are by looking at windows I have now for other hobbies that could in theory be substituted.

The former is more challenging to test – how much does accessibility really impact frequency?

How much did I really surf when I lived two blocks from the beach & how does it compare to my target of 12 times per year? Was it really every day, 365 times per year, like I sometimes hyperbolically state? Or was it actually more like 2-3 times a week – for 100-150 total surfing sessions per year? Or was it really 2-3 times per week during the good half of the year and very few during the bad part for a total closer to 50?

Would greater accessibility affect things by a factor of two, or an order of magnitude?

How can I answer that question? I could try to recall how much we used to surf – and how that changed as I moved further away. But my memory isn’t precise and is biased to remember things I enjoy. I am frequently reminded the fallibility of recall. I could try to track data on the surfing population and compare frequency to accessibility – but would laborious and I wouldn’t have my answer for months.

Fortunately, I solved the problem for myself a long time ago. I logged data that is useful here.

I’ve been tracking things about myself for over a decade. The largest set is from a longitudinal project in which I track every minute of my time during a sample week, once per quarter. These data go back to when I was still in college, which means it captures a wide range of life – from student, to gap year, to young professional – from bachelor, to husband, to father – from Pittsburgh, to Newport Beach. to San Francisco.

I took a look at two sample weeks during my Newport Beach years when I was a two minute walk from checking the conditions. I surfed on six days of the week for a total of nine hours during the first week and four days for a total of four hours the second week. By averaging those I ended up at five days for 6.5 hours. Assuming those weeks are an accurate sample (which I trust with decent certainty due to my methodology) that would put me in the 200-250 a year range. 15 – 20 times more than my current target.

Now some of that has to due with difference in lifestyle – but I can normalize for that by looking at how frequently I run now compared to periods closer to those sample weeks. That math accounts for a factor of two to three.

So we are left with the conclusion that moving into a house two blocks from the beach would increase the amount I surf on a yearly basis by a factor of 5 – 10. Not insignificant.

This ultimately leaves me with value questions. How important is surfing to me compared to other hobbies? How important are hobbies to me compared to other priorities?

But what I am not left with is ambiguity about the effect of the change – which means I can focus on those important questions and approach them from a solid base of facts rather than emotion.


Thanks to data I recorded previously for one purpose, I was able to quickly answer a new question with relative accuracy.

That is why we track things, even things we aren’t quite sure how we will use. At some point in the future we might be better equipped to use them. Tools, methodologies and questions that arise in the future are what will give value to our task of recording data today.

As I looked back at the data I had so preciously recorded and saved, my only regret is that I do not have more. More frequent samples, more details & more types of things recorded. This is what gives me the drive to track all the things I track now – of which the list is growing.

The Dread of Collecting Data

The thing I dread the third most is collecting data. Usually by manually logging it in some spreadsheet. It isn’t uncommon for me to log things in three to five different systems on a given day. The period when I’m experimenting on myself and have to track things is always such a chore and I am so relieved when an experiment ends.

The thing I dread the second most is having to analyze the data. I usually get into flow once I start, but these days, with windows of time to focus being limited, a file of raw data brings more dread than joy.

The thing I dread the most, however, is not having data when I make important decisions. Without data, not only will you be inaccurate more often, but you will not know how inaccurate you are. Data helps us be more accurate and also helps keep our gut in check my reminding us how often we aren’t correct. That is why I press through the things I dread second and third most.

2017 Focus: Quarter Year Update

With the start of a new year, I take the time to set my focus for the coming year. I believe that by doing less, I can achieve results that are exponentially greater than the sum of the results from split focus.

I detailed my 2017 focus here (read that first if you want more context). Here is how I’m progressing.

2017 Theme: Sabbath Year

2017 Challenge: Define & Launch A Sabbath Year

The launch of my sabbath year is slowly becoming a real thing. I’ve taken the first steps to put it into action and begun to lay out logistics.

After years of keeping the topic limited mostly to family & close friends, I recently gave my work concrete notice. I had roughly suggested the idea previously to my manager and one of the founders, but there was no firm timeline or next steps associated. There are still a lot of details to figure out, but the fact that I will be taking a leave from the workforce is looking fairly certain at this point.

As of now, it looks like the start date will be mid-year – sometime between now and August. This means I need to accelerate my planning a bit more as I’ve been somewhat light on that to date. I’ve not come in any danger of going over the seven hour a week planning limiter that I set for myself.

Along with planning logistics, I have done some rough thinking on the overall philosophy of a year of rest. Funny enough, more than a handful of people in my close circles are going through similar processes of taking rest or time away from their primary labor. Though most of those are shorter, in the few month range, their philosophies often share common motivations so we’ve been able to have some great discussion.

2017 Habit: Time Outside With Family

This year I renamed this particular focus item and changed a few things about it. So far I am extremely happy with the result of that process change. It has encouraged the right behaviors, repeatedly, without any strange side effects or end-of-year rushing.

To give one concrete example, I start my week on Monday, so by Sunday I am usually aware of how much time I have remaining to hit my weekly goal and trying to find extra minutes where I can. Walking to church takes about 20 minutes while driving takes about 10 – I typically prefer my mental state after walking but often time pressure & the weather push me towards driving. Lately the extra 40 minutes of outside time has helped tip the scales the other way though and I’ve been walking, even if it is cold or a bit rainy. To me that is one example of a decision I wanted to influence this year.

One process change I might consider in the future, however, is a phased ramp up where the goal is slightly easier at first and progressively gets harder during the year. For this year particularly I suspect Q3 is going to be really easy to hit 7 hours per week on account of the nice weather, long days & my break from a day job. I wouldn’t be surprised if a few weeks hit the 30 hour range.

As for results, of the 13 weeks in the first quarter I’ve managed to hit the seven hour goal 7 times.

This might seem unimpressive, but it was actually quite hard. Seattle winters are cold, wet & dark – none of which make being outside with toddlers easy and all of which increase the time overhead of getting outside (putting on jackets, shoes, gloves, hats, etc.) On top of that I am at work during the day during the week which means there are only a few hours where I am home and the kids are awake and those are usually hours where I’m doing wake up or bed time routines.

I realized how aggressive the seven hour goal was during a recent vacation to warm weather. My total for the week was only 16 hours and I had nothing to do but be outside. A lot of time is eaten up by nap times, meals (minus a few picnics) and driving to sights we wanted to see. I also took some me time to surf which doesn’t add to the total.

So far I’ve found that if I can get ~2 hours logged by Friday I can usually fit the other 5 in over the weekend.

To get some interesting data, along with recording time, I’m keeping track of what activity we took part in and which family member I was with.

We can see that our primary activity is walking – we do a lot of that around our neighborhood on account of the low overhead. A few blocks at toddler pace takes about a half hour, which is usually all I can fit in after dinner.

Most of my time outdoors is spent with the boys, or as an entire family. It is not very often that I have all three kids by myself and am outdoors on account of taking all three out is really hard. I have not once been outside with just my baby girl – I’ll need to change that in Q2. I have been outside with just Suzanne a few times, but I guess I didn’t log those – another thing to correct.

I will add that on top of that time shown in this table, I also spent another 31 hour outdoors without any family members. This includes runs by myself, surfing and activities with non-family members. I’m excited that less than 1/3 of my total outdoor time was solo this year, discouraging that was part of the motivation behind this goal. (I don’t have great baseline data on this from last year unfortunately.)

Finally, here is a collage that includes pictures of a few of my favorite outdoor activities from this year so far: New Years polar bear swim, bundled up, running, gardening, picnicing and playing in the snow.

2017 Exemplar: Eric Liddell

I started my study on Liddell by watching the movie that most people know him from, Chariots of Fire. It had been a long time since I’d seen it and this time I had something particular to pay attention to. I really enjoyed the movie.

I am now reading a biography about Liddell that came out recently – For the Glory. This seems to be the most in depth one written to date, so the timing is fortunate.

What is obvious about him so far is his work ethic and concern for others. Neither of which was my focus for this year, but perhaps I’ll learn more about his sabbath and rest in time.