Doing the Most by Doing the Least
Contrary to popular belief, sometimes to do more, we must do less.
I’m an endurance athlete by nature. I love doing big volume training that leaves me too tired to do much but veg on the couch. Weekends with 5+ hour bike rides trying to squeeze whatever’s left in my legs out? Yes please. A common ideology in endurance sport is that big volume leads to big success. Which made it all the more surprising to hear a podcast interview with the 2023 Western States 100 Mile champion, Tom Evans, talking about training philosophy.
In the interview, Tom noted he and his coach had taken an opposite approach to training building into this race. “What’s the least amount you can do while getting the most from yourself?”. Compare that to the common approach of “What’s the most I can do without injuring myself?”. Sometimes more is more. Other times, less is more.
While that conversation was around ultramarathon training, the concept can and should be applied in the data science world.
Big data has become a trendy topic over the last couple of years. The concept isn’t new but with the continued development of technology, data is being produced at the highest rates ever. Consequently, data science & analytics became hot topics as a way to handle all of this data and derive “meaning” from it. The natural reaction seems to be trying to track and create as many metrics as possible, building dashboard after dashboard in the name of “personalized insights”.
Of course that approach has to led to our current problem. Data science was supposed to help discern the signal from the noise. Instead, we often find ourselves generating more noise.
How we got here is nuanced, but some parts of clear.
When I started as a data analyst, the biggest mistake I made was the idea that being able to implement complex ideas and approaches would make me better at my job. Having knowledge of these things is fantastic, but knowing when to use them is equally as important.
I wanted to build new metrics from the jump. My oversight came in determining the value of what these metrics might be. Rather than working directly with business owners to understand their biggest issues, I assumed my knowledge of data would allow me to determine the right ways to build out these new variables. Which brings up the issue of everything above: what value are you providing?
My issue was thinking that complexity + volume = value. More data with more metrics would give business owners more insight. Maybe! But the problem then pivots from lacking insights to lacking sight. Instead of needing more information, we now need less. A business owner doesn’t have the time (or need) to dig through columns upon columns, trying to identify what’s going on in their world. They need you, the analyst, to provide them with that information.
Picture two scenarios:
- You provide a business owner with a dashboard that has five different tables, each with ten different metrics. You’re giving them 50 (!!!) different fields to help them slice up their business and see what’s going on.
- You provide a business owner with a single table that contains 2–3 metrics. This is easy for the user to digest.
Which one is best?
In most scenarios, the second is likely superior. While the first technically provides more information, it creates a massive barrier-to-entry for the user. They have to invest a significant portion of their time determining which parts provide the most value. In the latter, it provides less overall information but (if done right) has a handful of metrics that provide the most value and is easily digestible for the user.
As an analyst, the first scenario gives the impression you’re doing more work. But the point of analytics isn’t how much work you can do, it’s how much value you can provide to the business. If the work you do isn’t adopted and used, what benefit is it?
All of this to say, there will always be times and places for complex work. AI, LLMs, and neural networks aren’t going anywhere. I’m too young to make any bold statements but if I was speaking to young analysts, I’d emphasize one thing.
Focus on why you’re doing what you’re doing, less on how you’re doing it.
