Innovation Session: Evaluating My System for Gathering Data on Myself

Last December I started measuring a few things about myself every day. Now, four months in, I’d like to take a look at how it has gone and what that data has shown me so I can improve upon the system.

Success of the System

Over 102 days I completed the survey 80 times. Based on that I would deem the method a success. Any system that is able to remind me to do something and succeed in getting me to do it ~80% of the time is doing pretty well in my book.

Pivoting my completion percent by the day of week gave me the following.

gathering-data-on-myself-completion-percent

 

The astute reader will notice that my completion rate was >1 on Wednesdays. I thought there might be a bug or double logging errors in my system. When I looked into it I realized that a few of those were actually my Tuesday records being logged sometime after midnight. I’ve been doing these innovation sessions which often keep me up past my usual bed time on Tuesdays.

Nonetheless, mid-week my success rate is much higher than on weekends. This is probably due in part to the fact that on weekends I am more likely to be out of cell range while camping. I am also often hanging out with people at 8:00PM when my reminder sounds and sometimes forget to do it when I get home. A snooze option might help with that.

What I Learned

When I started this project I had three goals, each with their own type of questions:

  1. Things I want to quantify so I can later try to correlate them
  2. Things I want to codify so I can later look back on them
  3. Things I want to ask so they will stay on my mind

I’d like to evaluate each category on its success.

Category One: Qantifiable Items

Looking at these I don’t see much of a trend. After some iteration I landed on measuring my health in six areas on a scale of 1-10. On their own the items aren’t very helpful because they aren’t associated with any sort of action items. Data without action isn’t very valuable.

In order to make these items valuable I will need to associate them with something else. For example I could associate my vocational health data with records of who I had meetings with that day to help me see who I enjoy working with and who I might avoid when possible.

If I can find a few fact items to associate with each of the six health items I might be able to make use of these but as they are, there isn’t much to see.

Category Two: Records of the Time

These were free text questions about how I succeeded and failed that day as well as what I was excited or worried about. Unfortunately free text processing is not my strong suit and so I am at a loss about what to do with these. I usually only wrote down a few words and so there isn’t much detail. Looking for the most common words returns a lot of generic words; got, was, had, etc.

In order to make this section more useful in the future it might be helpful for me to create some categories to choose from or to get more text recorded so I can better use it in the future. Overall I think the benefits from these questions actually fits better in the next section.

Category Three: Items to Keep in Mind

These were questions I asked myself because answering them would require me to reflect on my day. The questions were all along the lines of “how did you demonstrate X today?” The responses were free text and so I didn’t leave a response if I could’t think of anything for that day that answered the question. It turned out I recorded something only ~35% of the time.

Despite that, I think having these daily questions was actually a positive thing. Even on the 65% of days I did not have answers, I asked myself the question.  Skipping putting in an answer came with a desire to try harder tomorrow.

Conclusion

The system I set up to gather data is actually fairly effective, the questions I ask need improvement though. I would like to see more actionable items coming out of these daily surveys and so I will need to hit the drawing board again. When I sit down again I’d like to dig into some  aspects of my life that I have the power to change and that data can help influence my decisions on.

Innovation Session: Innovation Sessions

Last December – nearly five months ago – I posted my first innovation session. The idea was one that I had been floating just long enough to come up with a name (which I later found out is another name for a brainstorm).

I was driving home after a surf session one Saturday, a time in which my head is usually very idea filled thanks to endorphins and adrenaline, and I realized that I could practice innovating. It dawned on me that innovation is a mental response and like other responses, it could be trained and strengthened.

I am a strong believer in innovation. Not only do I enjoy it, but it is also the skill set that I will likely depend on to put food on my family’s table for the next 40 or so years. The magnitude of that realization led me to take action and create situations where I could practice innovating and strengthen my innovation response. The resulting sessions have looked like some combination of business plan exercises, hackathons, Google’s twenty percent time & writing a research paper.

What is innovation?

It is a mental response to facing a problem while equipped with limited resources. We do not innovate every time we face problems. When faced with common problems we often rely on common solutions. We innovate when face problems that there is not an adequate solution for within our constraints. We innovate because we are short on money, time, patience, material, labor, etc. This is, I believe, one reason large companies do not tend to innovate well – they aren’t constrained as dramatically by limited resources.

What are innovation sessions?

Now, five months and 13 innovation sessions later, I would like to define what an innovation session is. My main hope is that it will help me stay focused and say no to things that are not innovation sessions. But, I am also hopeful that others will be inspired to find ways to adapt this format and take part in their own innovation sessions.

Goal & Benefits

Innovation sessions have provided me with three benefits. They allow me to:

  1. train my innovation skills by introducing my mind to new and diverse problems, racking my brain for solutions and to exposing the resulting solutions to public criticism.
  2. explore ideas I have for products, visualizations or tools
  3. learn new skills and hone others that I do not use as regularly in my day job

The first benefit was my initial intent – the training of my innovation response. Like we subject our muscles to strain in the gym to make them stronger, I want to submit my problem solving skills to strain in order to strengthen them. I try to present myself with diverse problems to broaden my focus – thus preparing me better for more future situations.

I try to plan each week so that the result is some sort of delivered solution – whether that is a working product or spec. Ensuring I reach this point allows me to present the solution publicly and get feedback. This part of the loop is important – I’ve found that whether or not anyone reads a post or leaves a comment – the knowledge that it is public holds me to certain standard and results in a higher quality of work.

The second benefit is the exploration of an idea. Like many of you, I am constantly coming up with ideas for new things. Often I find myself thinking “wouldn’t it be cool if someone made a…” or “we should build an app that…”. When I have those thoughts I note them as ideas for future innovation sessions. Exploring these ideas is a mental release. The act of testing the idea for four hours often results in the conclusion that the idea wasn’t as cool as I thought – but I would rather find that out than spend years dwelling on it.

The third benefit is the increasing and maintaining of skills. There are tons of tools, programs and languages that I would like to explore but haven’t been able to set aside time for. As I think through projects I will often favor those where I might be able to learn to do something new. Lately I’ve been itching to play with the D3 data visualization language and so it would be a smart bet to guess that you will see animated charts on my blog in the next few weeks.

Additionally there are skills I have that I have to some degree but do not use regularly in my day job that I would like to maintain. I used to do a lot of video editing, photoshopping, internet marketing, programming and blogging but I don’t spend much time doing those as a product manager. I’ve made a conscious decision that these aren’t things I want to do full-time, but I believe they are still useful skills to have and would like to maintain them for future projects I work on. Innovation session are a great opportunity to keep them up to date.

Format

I set aside four hours every Tuesday night for my innovation sessions. I look at as 200 hours a year of fighting status quo.

The goal is to start and finish in four hours – but I sometimes cheat and do some research ahead of time or end up staying up until 2 am finishing something. I need to get better at that because the four hour window is on of the constraints I’ve put on myself to force innovation.

The night starts by defining the focus area and starting to research if anyone else has tried to tackle the problem. I love seeing other solutions as they often give me ideas I can build off of. Sometimes I find a solution so awesome that I don’t think I can improve it in four hours – in those cases I pick a new problem for the night.

The rest of the time is spent racking, hacking, building and writing. I like to write as I design or program to capture my train of thought. Running these processes in parallel allows me to explore a lot of paths quickly in the format that is quickest. I end up deleting and rewriting a lot which is a nice iterative process.

Throughout the session I tend to learn a lot about the subject or the tools, but it is after I ship is when most of the innovation learning takes place. As I get feedback from other people, find out about projects I hadn’t found in my research and think of new ideas myself I start to see paths I hadn’t thought of. In those aha moments where I see something better I must process why I limited myself to one solution and how I can better equip myself to explore more possibilities next time.

Concluding Thoughts

At their core innovation sessions are about solving problems. This leaves the doors open for a wide range of project possibilities. The session focus, blocked calendar time and week-to-week project switching create an environment that fosters a broad learning of skills and a deep learning of innovation.

I’m excited to see where my next projects take me and hopeful that others will be inspired. If you decide to try out innovation sessions, let me know how they go – maybe your first one can be to iterate on my format – I’m sure there are ways to improve it.

Examples

Here are some of my favorite innovation sessions so far if you feel like taking a look at some examples

Building a Better Surfing App With Data

Boardgames + Math: Pass The Pigs

Gathering Data on Myself

Best NFL Team

Innovation Session: Building A Better Surfing App With Data

Imagine this – its 6:00 AM on a Thrusday and you’re driving half an hour up the coast to surf a wave you never go to during the week. Why? Because your iPhone told you to. That killer session you had last summer, it looks like the swell is lining up to recreate it. So you grab your board and hit the road hoping to turn the stoke up to 11.

better-surfing-app-recommendations


The world of surf forecasting & reporting has evolved slowly over the last 50 years. While it has adapted to the world of websites and mobile apps – most are simply new skins on the broadcast weather radio reports surfers have relied on since 1967. They are channels for data. They tell you the swell height, period and direction and something about the wind. Even when they look amazing they are usually showing the same information.

They are not simple and intuitive nor are they predictive. We can do better.

surfing-apps

Tonight I’m going to dive into the world of surf forecasting through the lens of a data scientist and explore what a better solution would look like if Surfline, Magic Seaweed or Swellnet approached technology more like Google, Amazon or LinkedIn.

Why We Use Surf Reports

Surf forecasts & reports exist to help surfers make the most of limited resources – time & waves –  in order to maximize the desired outcome – stoke.

Outside of the small percentage of competitive surfers – most  are doing so because they enjoy it. The idea of optimizing for stoke isn’t new to surfers – check out this quora answer about who the best surfer is.

Despite some similarities, every surfer is unique in how they get stoked. Some surfers charge giant waves while others like a gentle roller. No forecast can accurately predict stoke without taking into account the wide range of preferences that exist.

So, an ideal surf forecast would know something about me and make it incredibly simple to transform my limited resources into maximum stoke.

Breaking Down The Factors

Most surf forecasts focus on a few pieces of data about waves & wind. These are incredibly important to surfing, but not the only factors in the equation. In the table below I outline a number of the factors a surfer takes into account when going to surf.surfing-factors

There are a lot of things to keep in mind when picking where, when and what to surf. Some things are controlled by the surfer – what board they ride – but others are up to nature – the swell and wind.

The items in green are what most surf forecasts report on, but that leaves the others for the surfer to decide. Each of these factors interacts with the others as well, resulting in a lot of permutations.

The swell direction dramatically affects which beaches will pick up the waves. Tide changes throughout the day will cause some spots to turn on and off. The time of day will affect the crowds at popular spots. A break choice will impact which board choice will work best.

The Two Paths For Surfers

Because of this complexity – most surfers begin to walk down one of two paths:

  1. The Path Of Complication: These surfers become encyclopedias of surf spot information. You’ll recognize them because they’ll start to say things like “its a 2′ dropping tide and 6′ 10second south swell – Newport will be drained out and besides, it is Saturday, we’ll never find parking, lets wait a few hours and try for a sunset session at Magnolia”. These surfers will eventually grow to love the raw data on StormSurf.com and will be able to carry on conversations with oceanographers.
  2. The Path Of Simplicity: These surfers start to run on auto pilot. As a result of information overload, they begin to simplify their process. They pick one or two breaks they tend to stick to. They have a go to surfboard. They surf at about the same time every day. These surfers will have their break dialed, but will miss out on a ton of stoke at other places and on other boards.

How Can We Do Better?

An intelligent surf forecast would make recommendations for me. It would take into account where I have surfed before, cross reference that with a database of historical swell conditions account for the board I used and optimize for stoke.

Much like how Amazon recommends products we might be interested in based on other products we have purchased – an intelligent surf forecast would recommend times and days for me to surf based on past sessions that I gave good ratings to.

Data Gathering

To do this – the app would blend user collected data with buoy data.

From the user we would want to record: date & time, duration, location, surfboard used & a session rating.

better-surfing-app

 This data would then be combined with swell data we have on record for that break. Because we have the location and time – we can easily do a look-up in our own records to find the tide, swell and wind information for that spot during the time that session took place.

Personalization

Now, you’ll notice for surfboard I entered ‘Jezebel’ – one of my boards. I am a big believer of the ride everything movement and I think this is a great opportunity to encourage diversity of quiver. Rather than just having the person select a type of surfboard – we can actually have them enter their quiver and then select the board they rode each time. Future recommendations would be able to refer to the board by what the surfer calls their board.

better-surfing-app-surfboard-detail

 Imagine being able to look at your quiver like this.

better-surfing-app-quiver

I bet a lot more people would longboard if they saw how stoked they were whenever they took out the log.

Recommendations

This is the intelligent part of the app. Once a data set was populated we would use a bit of simple aggregation to start making recommendations. It would take into account all of the factors and make a prediction about what attributes could combine to create a session with maximum possible stoke.

There are some simple operations like ranking all of the surf spots you’ve been to by the average score you give them.

better-surfing-app-surf-spots

There are also some more complex lookups. Lets look at a few scenarios a surfer might find themselves in.

Say there is a forecast showing a 5′ SW swell heading in over the weekend and I want to know what the best possible beach & board combination for the weekend is. The app would look at all of the past times when a 5′ SW swell was present. It would group those by location and board combinations, average the ratings and show me the top three. Rincon with my hybrid, Blacks with my thruster or 52nd Street with my booster board. This is awesome.

better-surfing-app-recommendations

We could also limit the parameters a bit. Lets say I am in Newport Beach and have two hours, but am not sure where to go surf. The app could get my location from GPS and limit the search to surf spots within a half hour drive. We could also limit the tide to the current tide. We then group like before and make a recommendation.

If I’m feeling longboardy – the app could filter by my sessions where I rode a longboard and show me the best spot to go. If I’m meeting friends at a particular break – the app can limit it to that spot and let me know which board to bring. If I know I want to surf my thruster today at Newport, the app can figure out which tide gives me the types of waves I like best. There are a lot of cool ways to use this data.

We can also get really specific. Imagine an alert that let you know you might be able to recreate that awesome session you favorited from a few years ago by bringing your hybrid to Upper Trestles at 2pm on Saturday. I would pay money for that.

Combining Data

From what we saw above – there is a lot we can do. The one limiting factor is how much data we have. If I surf regularly, the data set will grow, but the more breaks I go to and boards I ride, the more diluted the data becomes and the harder it is to find a match.

For the scenarios above we were exploring using my own data only – this is important because we are optimizing for my stoke and I might like something very different than another surfer. If, however, I’m on a surf trip the above scenarios aren’t going to help me very much. This is when we might want to use anonymous data from similar surfers.

We could build this data set using a network map of surfers with common ratings in similar scenarios. Lets say that I usually rate Blackies very high and the board I ride there is a longbaord. A network map would group me close to other surfers that rated blackies high. Maybe a few of them also like Malibu and a few other Malibu surfers like La Jolla Shores. Our app now knows that Blackies, Malibu and La Jolla Shores are similar based on surfers who have surfed more than one of them and rated them similarly. This opens up a lot of possibilities and expands the data set.

Conclusion

Surf forecast sites have focused on giving surfers data. We don’t want data, we want to go surfing. By taking advantage of mobile technology and some data know-how there is a lot of opportunity to build something amazing. Hopefully one of the current surf forecast companies will take note and start working on something similar, I would be the first one in line to start using it!