Reflecting On My First Year At Databricks
I recently finished my first year at Databricks, so I wanted to take a chance to reflect on the year; summarize what I’ve learned, see if the reasons I joined have proven valid, and check in on how I’m progressing on the goals I set for myself.
If you haven’t yet read the blog post on why I decided to join Databricks, you might benefit by referencing that.
How Has My First Year At Databricks Been?
I’m generally very happy here and think I made a good decision to join. I am truly living up to my LinkedIn tagline of “having fun doing hard things with great people”. I’ve had a lot of fun, but it has also been very challenging, in the way I enjoy – the way that stretches me to be better.
In some ways I got really lucky joining when I did, much like I got really lucky regarding the team I joined at Google and timing of that decision. If I keep getting lucky like this, people might start to suspect that I know what I’m doing – so I’ll make sure to plan a few more silly adventures for next year to help avoid that misunderstanding.
Continue reading to learn about:
- why I am so optimistic about the company
- my reasons for believing this is one of the best PM orgs in tech
- how Databricks is helping me learn to be a next-level product manager.
What Did I Learn This Year?
1. This Might Be the Best PM Team in Tech
I understand that it is hard for me to not be biased here, but let me state my case nonetheless. Our PM team is relatively small – <100 people in a company of ~6-7k employees. This only works because everyone is a bit overpowered for the job they are doing. Our average PM has the skillset seniority of someone who would be a PM Manager or Director at most other top tier tech companies.
Let’s use me as an example – at my last job, at Google, I was managing 6PMs and our group was responsible for the work of ~70 Google engineers. At Databricks I’ve spent the last year as an individual contributor, writing PRDs and leading the work of ~15-25 engineers (as we grew the team). Normally an IC PM writing a PRD for a few engineers might have a few PRDs under their belt, or sometimes not, everyone has their first PRD. In contrast I have dozens products I’ve shipped to GA and learned from and dozens more I’ve supervised. I’m just more experienced at the job and so when I write a PRD, there is less guessing and more knowledge from having done it before.
Why are Databricks PMs so senior? My understanding of the history of this is because we build a very technical product that is used by engineers, so many of the founders and early senior engineers effectively served as PMs for the first years. Eventually PMs joined, but to add any value, they had to be really good, otherwise the engineers building the product would know more about the engineers using it and ignore the PMs. So, the bar was set high. The bar has stayed high because we have such great engineers at Databricks – they don’t accept sloppy work or weak evidence.
What that means is we have a smaller number of PMs who are all very strong. Of the ~100 I am probably about the 10-15th most senior – essentially the most junior of the PM managers. When I meet with the PM leadership, I look at the faces on the screen and am super impressed. Each one of those people is someone I could learn a ton from, would be happy to have as a manager and who has shipped amazing products in their past – or led the org that shipped those products. Microsoft Commerce, Google ML/AI, Github Copilot, AWS EC2, YubiKey, Docusign, UiPath, etc. It feels a bit like the 1992 Dream Team, which I guess makes me Christian Laettner, the player in the spot that should have been taken by Shaq were it a year or two later.
2. Databricks Keeps Taking Big Swings
This one might be obvious from the outside too, but it stood out to me over the last year. Databricks is not a company content to grow its existing business, it keeps inventing new things and moving into new areas.
For example, at a company-wide meeting last February someone asked the CEO “what is something we aren’t doing much of that we should probably do more of in the next year?”. He mentioned LLMs were going to grow in importance. Sure enough, a few months later, we had released the open-sourced Dolly LLM, announced the Lakehouse IQ feature and acquired Mosaic AI, among other LLM related activity. That is a big billion+ dollar bet we’re taking on an emerging technology and I’m excited to see that we have the appetite to do that.
You can never know the future for sure but I remain convinced that Databricks has a great shot at being one of the technology companies that helps write the story. Taking these big swings at new trends, is one of the requirements to do so and I’m happy to see we’re doing it.
3. Our Execution is Intentionally Excellent
Taking big swings is necessary, but not sufficient to be great. Vision and strategy are nothing without execution. Remember, Microsoft swung at mobile a few times, but never connected and Theranos had a great vision for a test from a single drop of blood but it never worked. Databricks keeps impressing me with the way we can actually execute on our vision.
Some of that is because our leaders have been very intentional about creating a culture where execution thrives. There is a strong bias towards action and delivering value to customers and that is incredibly refreshing.
Out leadership team has also tried to combine the best aspects of companies like Google, Microsoft, Amazon, Facebook, Uber, etc. to form best practices for how we operate. This shows up in things like how we invest in employees, how we think of hiring and management, how meetings are run, how disagreements are settled, etc. For example, I’m often in meetings where I’ll read a written brief – like at Amazon – before discussing how we can apply Google-like engineering rigor to a problem and strategically position the product like Microsoft does so well. Sometimes mixing a bunch of best practices results in a garble of nonsense, but Databricks seems to have found a balance that is quite impressive.
4. Growing During the Work-from-Home Period of COVID was a Lucky Break
I hadn’t thought about this when I joined, but I now realize that the time period in which Databricks grew from ~500 to ~5k employees meant most of those employees were hired working-from-home. What that also means is the company hasn’t had to pay for a ton of unused office space, as so many other companies have. We certainly have offices, nice ones, but they are smaller since our norm has been ~1 day per week in the office, using shared desks that we book using an app. This creates a bit of an unfair advantage where the company is spending less per employee on office space, so it can afford to spend more on perks, and still maintain a lower overhead. That has the dual benefit of feeling better day-to-day and also creating positive momentum where our employee perks keep improving rather than getting clawed back – as they are at so many other tech companies. That makes it feel good to work here, which is motivation to work harder, which then results in more business impact.
5. Google’s Reputation Outside Isn’t What We Thought it Was Inside
When I joined Google I described how Google has a bit of a halo around it. I found that to be generally true during my time there but have noticed that halo exists mostly outside of tech. Inside the tech world, many people have worked at Google or know people who have and the reputation is a bit more realistic. It took some adjusting for me to realize that saying ‘at Google we used to do such-and-such’ doesn’t matter that much to most people.
That is probably fair. Google launched some amazing products in the 2000’s that changed the world – search, maps, Gmail, android, docs, YouTube, AdWords, etc. and even more research and best practices that powered those behind the scenes – map reduce, borg, data center design, SRE culture, etc. But what has Google done in the past 10 years? Photos, the 3rd place public cloud and incremental growth at ~20% per year. Sure the research is still strong – transformers, quantum compute, etc. – but it hasn’t yet turned that into the type of products I mentioned above, and certainly not at the pace we saw from 2004-2008. With that in mind, it is rational that most people are less interested in learning about best practices from the Googleplex than they once were.
I still think Google has amazing people working there, but there are also plenty of amazing people elsewhere, many of whom have done impressive things, and had less of a foundation to launch off of. I’ve learned to approach conversations with a little more humility, I now say things like “I’ve had some success doing such-and-such before”, and letting the point stand on its own merit, rather than the fact that it came from Google. I think many Googlers and Xooglers would benefit from taking to heart.
6. Recruiting is a Lot Harder Here
One area that I’ve learned is very different at Databricks compared to places I’ve worked before is recruiting.
At my last company, we had a steady inflow of pre-qualified candidates from the centralized interview process and so as a hiring manager, my job was mostly to make the role sound exciting and find a candidate that seemed like a good fit for the team. Prior to that I was at a startup, that was much smaller and where as a hiring manager, I had to generate interest in the company. Because of our small size, we were much more likely to take a bet on a candidate that seemed bright, even if they didn’t have much relevant experience.
Hiring at Databricks, in contrast, is a bit different. I’d summarize it by saying we’re trying to hire the best people in the world for the job. Bright individuals who have executed exceptionally well in relevant areas. We’re looking for fits that are so perfect it seems almost impossible to imagine a candidate that fits the bill. The hard part is that even when those people do exist, they are often the exact type of people that aren’t interested in a new job at the moment because they’re doing impactful things at another great company.
With that challenge comes growth. I’m learning new ways to generate excitement for the company and role, different methods to evaluate talent and hunger and finally, to be patient and strike when the timing is right. It will take me a while to experience this full process and see the results of it, I’m inclined to trust it based on the amazing team I see around me and the fact that this process helped create that team.
Was Joining Databricks A Good Decision?
Last year I discussed a few reasons that I decided to join Databricks – I briefly want to reflect on those to see if they were in fact accurate assessments and if they’ve stayed true or if my decision was built on faulty assumptions.
A number of people have read that blog post and reached out to say they appreciated my logic. Some of them even joined us at Databricks. I hope these updates will help anyone still on the fence.
- Great People That I can Learn From – absolutely, more on this below
- Growth Potential – again, true, and more on this below
- Simple Business Model – while the monetization can be a bit complex – otherwise they wouldn’t need me – the business model is incredibly simple. Data has value and we build tools that help unlock that value and then charge money for access to those tools.
- Technologist-led company – yes – I sometimes laugh at how nerdy the C-level reviews are. It’s a bunch of computer science PhDs using programming metaphors to tackle business problems and I love it.
- High Importance of My Project – more that expected with the way the economy has gone. The tightening macro environment pushed many companies to be more cost conscious, which increased their need for the tools my team builds. As a result, talks I’ve given at customer events have repeatedly been some of the highest attended, which is a nice proxy for importance.
- Low Risk – this continues to feel true about the company. Startups are a gamble – a $B business isn’t really – its more just a question of pace and duration of continued growth.
- Data at the heart – no misrepresentation here – this is what we’re all about
- Building for a championship – more than I thought, which has pros and cons (see above reflection on how hard recruiting is)
- Unified mission – absolutely. At my last company I feel like 50% of meetings had an undertone of “I need to do this so I can get a good Perf rating” and I have seen practically zero of that in a year at Databricks. Everyone is focused on one thing – helping our customers succeed, so our business can grow.
- Pre-IPO but Potential to IPO – we remain pre-IPO with the potential to do so – pay attention to our CEO for updates there, I don’t know anything that isn’t public information
- Founder-led – yes, the founders are very involved, sometimes at a surprisingly granular level. I’ve gotten to talk with just about all of them at some point. I also found out that Databricks has an unofficial “Founders Jr.” group of folks that have been here a very long time and who have a huge amount of influence. I’ve gotten to have dinner, drinks and convos with some of them and was pleasantly surprised by the caliber there. What this means is Databricks doesn’t have a single great mind behind it, there are at least 10+, which bodes really well for resiliency.
- Bias for action – in my first months at Databricks we launched a preview for a feature that I pitched to the CEO in my first month. I was awestruck because it would have taken me longer than that to get approval to start thinking about it at other companies.
- “It feels like Google 10-15 years ago” – I wasn’t at Google then, but I know a lot of people who were and I’ve read a lot of internal documents written in that period. The feeling at Databricks lines up with my understanding of what Google was like then – which was of course the period when Google was named “Best Place to Work” basically every year.
By my count that is thirteen for thirteen assessments that still seem true a year later. My due diligence appears to have been sufficient here.
Now, just to show this update wasn’t a sponsored post from our recruiting team – let me take a minute to reflect on some things I learned that are less-than-positive.
- Diversity is lacking – though inclusion is pretty strong in my opinion. We need to get better at having more diverse perspectives, backgrounds and people. This is something that is important to me and I hope to be able to influence it now that I’ve gotten through my onboarding period.
- Burnout is real – I had sort of heard this before joining, but was willing to take that risk as I usually do a decent job at setting boundaries. I haven’t suffered from this (yet) and have generally found I work about the same amount of hours I did at my last job, but that I get a lot more done in those hours, since there is less communication overhead. This sometimes means I’m more tired at the end of the day though, since meetings were often a time to mentally rest at my last job and I’m doing more writing and thinking here. I have seen other people suffer from burnout though, but thankfully from what I’ve seen most managers are reasonable and willing to help if an employee says something. There is a bit of an onus on the employee, unless they get a really observant manager though, so I might suggest if you’re going to join Databricks to have a burnout accountability partner within or outside of the company to help tell you when your are getting grumpy and tired and probably need to take a vacation. For what it is worth, I do see lots of vacation getting taken and everyone, up to the C-level is very respectful about giving folks space when their calendar says OOO.
- Collaboration is everywhere – The environment is the most collaborative I’ve ever worked in. This one isn’t so negative as much as a fact that might be polarizing, so I’m going to leave it in this section. Essentially I’ve worked in places where the culture was more ‘my feature, my decision’ and that isn’t the case at Databricks. I’ve found most of my projects involve collaboration from 2+ PMs as there is lots of overlap between horizontals and verticals. The good news is our PM team is amazing, so in my experience to day, that results in a better product, for a bit more communication overhead. I’m willing to make that tradeoff but folks should ask themselves if they are willing to before joining.
How I’m Doing At the Goals I Set For Myself When I Joined
I set a few goals for myself when I wrote about why I decided to join Databricks. Let’s check in on how those are gong one year in.
1. Learn a lot
One of my key reasons I ended up leaving my last company was to continue to stretch myself. I had learned a lot there but felt that I could learn even more by coming to a new company in a slightly different domain, with a different way of doing things, and at a different stage of growth. Learning is how I improve and set myself up to be able to do more tomorrow, so I am continually seeking more of it.
I expected to learn in some specific places – I wrote:
“I expect I’ll learn a lot about recent developments in the data/analytics space, the other two big cloud providers (Amazon and Microsoft), new ways to be a PM, and different parts of the billing stack.”
Those have all proven true.
- Recent data/analytics developments – I am much more aware of recent changes in the data and analytics space than I was a year ago. This is partially from my core responsibilities where I work with both a) various customers as they attempt to understand and optimize the cost of their usage of our products and b) Databricks product teams as they design and launch new products, helping them think through what the customer experience around billing will be. The other part of my awareness is that I have become a heavy user of the Databricks products. A big part of our company mission is to democratize data – to help every decision be data-forward – so it should be no surprise that our leadership expects us to use our product to analyze internal data to defend positions we present.
- Azure and AWS – I knew a ton about how GCP’s systems worked, having helped build that system, but my knowledge of specifics of the other clouds wasn’t nearly as deep. I’ve had to learn a lot more about details of the billing systems and products because of how closely integrated we are with them and how that interacted with my remit.
- How to be an excellent PM – I’ve found Databricks to be among the more rigorous places when it comes to defining the Product Mangement role. (The most rigorous I’ve worked at, but I’ve only worked at like 5 companies.) Specifically, where many places use PMs as putty to fill gaps, or focus on the art of PMing, Databricks seems to focus more on the science of PMing. This includes more of an emphasis on doing research, forming hypotheses, collecting data to validate or invalidate those and then sharing the knowledge with a broad group. Much like the scientific method is used to increase human understanding and improve our condition, but in this case the method is applied to business problems to increase customer understanding and improve our software development. In some ways this extra rigor feels like more work in the short term, but I’m being convinced that it results in more impact in the long run.
- Different parts of the billing stack – Every company draws team boundaries a bit differently, and I expected to find that about the billing org when I came to Databricks. In fact, I’ve since found out that the area I was the deepest in previously is an area my current team spends the least time on, so most of my effort has gone into other parts of the system where my knowledge was more periphery previously.
I also expected to be surprised by some of the things I would learn, which I touched on above in the section on what I’ve learned. I said I wanted to always be learning and at this point I still feel like there is more to learn than I have brainpower for, so I remain excited.
Action Item: Just keep learning, just keep learning (to the tune from Finding Nemo)
2. Build a great billing team
When I joined last year I said “if the team quality is no better than the average industry team, I haven’t really taken advantage of the opportunity I had here. I want to help build a team that feels special and has a reputation for greatness within the company and perhaps even outside of it.”
I meant that in the specific sense of the Billing PM team I will directly manage and in the broader sense, the overall Billing org that I’m one of the key leaders in.
It takes time to build that sort of a team and I’m overall happy with the progress over the last year.
As an overall team, we’ve grown a lot, taken on new scope and rebranded in an effort to reflect that change in mission. We started a few very big projects which are starting to payoff and we have a great team in place to start the next few big projects. A lot of work goes into forming a high performing team and I feel like we’ve made good progress on some of the important trust building steps that are necessary to get there.
As far as the Billing PM team, I’ve been our solo PM this past year, but now that I’m ramped up and have a strategy in place am starting to hire for the first new position. I expect will eventually be a decent sized and very strong team, that owns a very important set of features. Even with only one PM, we were able to get lots of good input from customers this year and I recently had the chance to present some new features we delivered to a few hundred people at our Data and AI summit (video here).
Action Item: Keep the Billing team on track towards greatness and make sure the PMs I hire help us get there
3. Build deep relationships with great people
This one is easier than I expected because it is a target rich environment. There are a lot of amazing people at Databricks who have worked on amazing things in their careers. They’re all pretty nice too.
I described last year how my approach to achieve this was to make sure I invest the extra energy to meet and connect with people – taking the chance to be outgoing. I’d say I lived up to that.
One week that stands out is our company kickoff, which was hosted in Las Vegas. I probably averaged 4-5 hours of sleep a night because I was taking advantage of every social event and adding more where possible. One day I spend all day in presentations/meetings followed by team dinner and then an optional company-wide 5v5 indoor soccer tournament until midnight, followed by waking up at 5am for a run with another group of people, coffee with someone else, and then more back to meetings. I was filling hallway walks with more convos and wearing silly outfits to help break the ice. It was a lot, but it was also a great way to connect with a lot of people fairly quickly.
It takes time to build deep relationships, so I wouldn’t by any means say I’m done here, but I think I’m heading in the right direction.
Action Item: Now that I know a lot of people, invest the energy to achieve depth and trust. I’ve found that being in the trenches together through difficult challenges is the best way to do that.
4. Get to be a part of an IPO
Databricks has not gone public, yet.
Part of my reason for coming here was wanting to experience the run up to that exciting moment. I previously said, “The more work I do pre-IPO, the better that celebration will feel.” Now that I’m a year in, I feel that I’ve done a few things that have contributed to our ongoing success, but it will really take another year or two for my projects to grow into their full impact.
We did recently raise a round of funding at a $43B valuation, though, which felt like a sneak preview of the excitement.
Action Item: Keep shipping features that help our customers and our company – the hard work now is what helps the celebration feel great later
5. Hit my FIRE number
This is essentially asking whether or not leaving Google to join Databricks was a good financial decision. I don’t have an expensive lifestyle – I drive a 10 year old Subaru and wear t-shirts I get free from work – so my financial goals at this point are mostly about saving for retirement.
It is too early to make a judgement here, since my investment horizon was always intended to be longer, but our recent fundraising round hints that I might have made a good choice.
Action Item: Same as above, keep working hard and help the company grow
6. Continual high growth
This is less of a goal and more of an expectation. I said that “I expect Databricks to remain in the top tier of cloud companies in terms of yearly growth.” Basically, I wanted to be on a championship team, and so I set this benchmark to help keep that top of mind.
That has continued to be the case. Our CEO, Ali Ghodsi, recently told Bloomberg that he believes we are the “fastest-growing software company” and we shared that we are growing at 50% year over year. This is during a time when many other companies are seeing their growth slow and valuations dip as macroeconomic forces put pressure on their revenue.
I wish I could take some credit for this, but efforts to impact growth trends usually take more than a year to have an effect, so our recent success is likely mostly due to work done before I joined Databricks. I can, however, be thankful that I made a decisions that seems to have been wise.
Action Item: Expect the best and do what I can to help it persist.
Conclusion
It’s been a great first year at Databricks. In many ways I feel like my first year was all about onboarding – learning how to do things in the specific ways they’re done at Databricks. Now that I’m ramped up on that and have a few features launches under my belt, I feel like I’ve shifted into high gear. I’m excited about the next year and being able to help have an impact on this company and our customers at such a critical time. I’ll check back in next fall to let you know how things go.
Awesome post! Really appreciate your candid insights. In your opinion, which of the unique elements of Databricks’ organizational structure and processes that you listed would change the most as a result of an IPO?
In a following post, it would be great to hear about how you are applying a product mindset to evolving monetization approaches.