Reflecting On Two Years At Databricks
I’ve been working at Databricks for two years now, so I wanted to take a chance to; reflect on my time here, summarize what I’ve learned, see if the reasons I joined have proven valid, and check in on how I’m progressing on the goals I set for myself.
This post my prior one on why I decided to join Databricks.
How Has My Second Year At Databricks Been?
I’ve generally remained happy here and with my decision to join Databricks.
My first year required me to slow down a bit in order to ramp up to the culture – Databricks is quite unique in how it works. My second year, in contrast, allowed me to move closer to what I’d consider full speed – delivering features, defending positions, hiring people and representing the company externally. This is the type of impact I expect of myself.
Going into my third year, I feel like I’ve earned my seat at the table and I’m now building out a team to multiply my impact and very excited about the potential.
I have been challenged – mostly in good ways that forced me to grow. I’ve been frustrated – mostly by things that come with rapid growth. I have had a lot of fun – mostly because I like doing really hard things.
What Did I Learn This Year?
1. Patterns Follow the Customer
One of the things I was cautious about when joining Databricks was trying to overfit the solutions my team had built at Google to the problem space we own here. It is possible that some decisions my team made at Google weren’t the best option or were only local maximums because of other context, like prior decisions, company processes, etc.
I’ve largely found this concern hasn’t been valid. Most of the time, when faced with decisions about what to build or how to build it, the decisions end up in a very similar place. Even if I go back to first principles and build up a case from scratch. Even if everyone here resists it. We often end up shipping a similar feature.
I think the reason for this is simple – people and companies are slower to change than technology allows for. Since I build software for people, mostly people that work for large companies, that means the customer needs are slow to change. The market research I did three years ago about how enterprises wan’t to see their cloud bill remains largely valid.
This breaks down in certain consumer applications, especially those targeting younger users, who are learning something for the first time. In that case the customer is really a whole new person that has a chance to get used to a new way of doing things as they’re learning from scratch. It also likely breaks down when there is a disruptive technology as the value of that new technology justifies the effort of the people and company to change.
There are durable places where the needs don’t change and disruption hasn’t happened. In those places you can run fast using a memorized trail. The crux is knowing when this stops being true.
What this meant practically is I was able to support a very large scope this past year. My ENG to PM ratio reached 50:1, which is sort of unheard of. It surprisingly kept working decently well for two reasons 1) my engineering counterparts are amazing and were willing to pick up a bunch of work a PM would normally do 2) I knew the answer before the test started. I could often sit down with an engineer for an hour, listen to the problem, match the CUJ and describe the likely shape of the solution, along with a few red flags to look out for. Validation was still needed, but those one hour sketches proved fairly accurate a number of times.
The practical way to utilize this learning would be to ask if a domain was one where the customer had expectations and was slow to change. I suspect there are more of those than most technologists care to admit. That likely also means change management is a more important skill than creativity in some areas.
2. This culture isn’t for everyone
I’ve seen a number of great PMs leave Databricks and I sympathize with them. The reasons they’ve shared with me are valid and I think a lot of it comes down to working style preference.
Some PMs like ownership and being able to operate with autonomy. That isn’t how Databricks works. It is a very collaborative environment where very senior people end up defending their opinions on things that would be within that person’s decision-making scope at other companies.
Is that ok? I think there is room for doing things different ways and the results here are important to consider. Would I love to be able to make decisions unilaterally? Sometimes it seems faster. Are the decisions my team ends up making better because of all of the dissection and discussion that goes into them? Yes.
Is it worth the effort? That is the thing about technology – it can scale so quickly that it is often worth the effort of getting it right. Consider Apple and the effort they put into design and polish. I suspect they spend twice as long working on things as some of their peers and they often don’t ship products that don’t meet their bar. Is it worth it? It appears so.
3. Recruiting is a Lot Harder Here
Last year I shared how difficult recruiting at Databricks was. I had only scratched the surface.
The hiring bar for PMs here in unreal. We are truly only looking for the best product managers in the world. I had over one thousand qualified PMs – people working at places like Amazon, Microsoft, Google, etc. – apply to a role I had posted before I was able to hire someone.
I reviewed every resume, talked to ~150 people, had ~25 go through interview loops, got 5 to offer consideration and finally hired someone last month. That is a 0.1% acceptance rate. For comparison, Harvard has a 3% acceptance rate and the Secret Service is around 1%.
I nearly lost hope at one point. At some point when you’re looking at 0 successes out of 999 candidates, you have to wonder if the task is possible. Thankfully a few of my peers were very encouraging along the way and reminded me that they felt the same way at one point.
One success later and I realize it was possible, just very difficult. I can deal with difficult.
Did I learn a lot? Yes, I did. As frustrated as I was at times, I would remind myself that the reason our company was such a great place to work is that we had a very high hiring bar. Clearly the process had worked to find great people. I’ve absorbed those lessons now and calibrated my radar to the company’s standards. Will I follow them forever? I’m not certain, but I’m glad to have internalized them nonetheless.
Was Joining Databricks A Good Decision?
When I joined Databricks I discussed the reasons that I decided to join – I briefly want to reflect on those to see if they were, in fact, accurate assessments or if my decision was built on faulty assumptions.
- Great People That I can Learn From – yes, I continue to spend a lot of time with people that teach me things. Some great minds in computer science and technology, other great business people. Databricks has brought together many of the best technologists and that means I have a lot of people who are financially motivated for my team to do well. That is the recipe for learning quickly.
- Growth Potential – yes, for the company, for my team and for me personally
- Simple Business Model – the business has expanded, but the model remains simple – customers pay us for tools that help them extract the value from their data.
- Technologist-led company – remains true
- High Importance of My Project – the company revenue has ~tripled since I joined, but my team has grown even faster. Why? It is an incredibly important area that was underinvested in previously. I was hired to help highlight the importance of it to leadership, set a vision and build a team that could execute on that vision. Thanks to support from partners around the company and an amazing team of engineers, we’ve been able to make great strides here.
- Low Risk – yes, I think the company might actually be lower risk than bigger tech companies who have gone through layoffs and refocus efforts. That is somewhat ironic as often a bigger company is considered less risky. I think that stopped being true at just about the moment I joined Databricks. Call me lucky.
- Data at the heart – remains true. The company messaging certainly includes more about AI now than when I joined, this is true of nearly all companies though. Our approach to AI remains data-centric though, which is a bit unique and something I enjoy.
- Building for a championship – continues
- Unified mission – yes, I can still give a 10 second pitch for the company. I meet with hundreds, if not thousands of Bricksters and it always feels like we’re fighting towards the same goal. I love that.
- Pre-IPO but Potential to IPO – we remain pre-IPO with the potential to do so. Will that ever change? I don’t have any answers and couldn’t share them even if I did.
- Founder-led – yes. In fact, our CEO recently weighed in on the whole ‘founder mode’ convo and took it to a place I hadn’t expected. He said that while we had seven founders, not all were involved day to day anymore and there were people who he considered ‘founder mode’ that weren’t here on day zero. That was encouraging to hear as I thought we had 5x redundancy in founder-leadership, but perhaps we have even more.
- Bias for action – I’ve actually started to hear customers share that we ship too many things. I do think we need to address what is at the heart of that, but it is funny to hear given the default state of a growing company is to slow down and the default state of customers is to want more features, faster.
- “It feels like Google 10-15 years ago” – while I was not at Google then, so it hard for me to directly compare things, my current experience at Databricks feels very similar to stories I’ve heard from Google from that time period. Mostly from people who have since left Google.
How I’m Doing At the Goals I Set For Myself When I Joined
I set a few goals for myself when I wrote about why I decided to join Databricks. Let’s check in on how those are gong one year in.
1. Learn a lot
When I joined Databricks I wrote:
“I expect I’ll learn a lot about recent developments in the data/analytics space, the other two big cloud providers (Amazon and Microsoft), new ways to be a PM, and different parts of the billing stack.”
Those have all proven true.
- Recent data/analytics developments – Because my team’s product is responsible for monetizing all of the features at Databricks and, is built on top of many of the features at Databricks, I get to spend a lot of time learning about Databricks. Does that mean I know about all of the developments in data and analytics? No. I don’t really know anything about what is going on with DuckDB. But I’ve learned a ton more about how to train LLMs, streaming data, recent visualization trends, vector databases, etc. So this one holds mostly true, limited only by Databricks’ surface area, which is constantly expanding.
- Azure and AWS – This continues to be true. In the last year I’ve had meetings on campus with Microsoft, AWS and Google. Being at Databricks is really a great place to learn about the broader cloud industry and not just the one you happen to work for.
- How to be an excellent PM – I wrote before that Databricks focuses on the science of PMing – doing research, forming hypotheses, collecting data to validate or invalidate those and then sharing the knowledge with a broad group to help define a solution. I’ve gone through this product development process many times in my career as a PM, but this year I was in a situation where I had to do it quickly, frequently and very well. I shipped about six PRDs this year, mostly due to the huge scope of my team. I would guess at most companies, the typical PM delivers closer to 1-3 PRDs a year, so getting so many reps in one year was a great chance to really practice things.
- Different parts of the billing stack – My scope has shifted in a few directions I didn’t really anticipate, but that I’ve been enjoying. Where as previously I focused on enabling the business to execute by building a robust and flexible system, I am more involved now on the observability and cost management side of things – helping customers understand the value they get from our platform. I still spend a decent bit of time thinking about higher level things, but the depth is in a new area, which is interesting. I think that will likely be valuable in the future as I spend more time setting strategy, now that I’ve had a chance to spend a few years focusing on each of these sub-components.
Action Item: Continue to look for new areas that are challenging and force growth
2. Build a great billing team
One of my goals in joining Databricks was to help build an excellent team.
Upon joining I said, “if the team quality is no better than the average industry team, I haven’t really taken advantage of the opportunity I had here. I want to help build a team that feels special and has a reputation for greatness within the company and perhaps even outside of it.”
I think the Money Team is getting there. This year we got to see some of the results of the hard work from the last few years and our internal reputation is fairly strong because of it.
Externally we’re starting to get some notice too. Two of our engineers published a great blog post about how we turned around our system reliability.
We launched some critical features that were promoted by our CEO and very well received by customers.
I’ve gotten a chance to present to a number of customers, partners and industry peers who have heard mention of the Money Team and are excited about what we’re doing.
I think we’re in a great place to continue delivering value and doing so as a team that feels really fun to be a part of.
Action Item: Hold the ship steady
3. Build deep relationships with great people
When I think about building professional relationships, I tend to think of three groups:
- People farther in their career than me who I can learn from and ask advice of
- People at roughly the same place as me who I can share stories with and grow alongside
- People earlier in their career whose energy lifts my spirits
As I’ve progressed in my career, group 1 has gotten a bit smaller and group 3 has gotten a lot bigger. With that, my mentality has shifted a bit. Early in my career I would try to identify leaders I respected and find excused to work alongside them, so I could learn something from them and earn their trust to work on future projects of increasing scope.
I still try to do that, but now, more of my time is spent working with engineers, Solution Architects, and product managers who are more junior than me. I try to be as good a mentor as the best I ever had. I spend a lot of time meeting with more junior PMs and helping build up their courage – reminding them they can do great things.
I didn’t predict when I joined Databricks that as much of my time would go to this, but upon reflection, I’m happy it is working out this way.
Action Item: Slow it down a bit and keep investing in the next generation of great PMs
4. Get to be a part of an IPO
Databricks has not gone public, yet…
Now that I’m two years in I feel like I’ve earned a drink to celebrate if and when an IPO happens, though.
I’ve been at the company longer than most people and I’ve contributed to a number of big decisions. I’ve put sweat and blood in. I’ve shipped features. I’ve talked to hundreds of customers. I’ve burned the midnight oil. I’m proud of that.
I recently had a moment I’m proud of that that I’ll share here. In a CEO review of another team’s new feature – a pretty strategic one – our CEO and founder, Ali, stopped to ask “Where is Greg? What does he think of this?”
I was on mute and off camera, but hopped in to share my perspective. This wasn’t the first time I’d shared my perspective at an Ali review, but it was the first time he’d asked for it specifically.
In some ways, I take it with a grain of salt – Ali hates it when everyone agrees and since I’m usually in the minority on various perspectives – perhaps he just wanted to use me to stir the pot. In other ways, it feels like a positive signal – in a meeting full of long time employees, founders and VPs, our CEO asked for my perspective – maybe I’ve earned some trust around here.
Action Item: Be patient – keep doing the hard work
5. Hit my FIRE number
It is too early to make a judgement call on this, but I think things are trending well. I’ve vested more than half of the stock of my initial four year grant and I got a healthy refresher earlier this year, which means my vesting next year will be even more than I had predicted. There are lots of things that can change a company’s valuation, and thus the value of stock, so I remain cautiously optimistic that once my stock is liquid, I’ll end up in a better place financially than I would have as a Group Product Manager or Director at Google.
Action Item: Same as above, keep working hard and help the company grow
6. Continual high growth
This is less of a goal and more of an expectation. I said that “I expect Databricks to remain in the top tier of cloud companies in terms of yearly growth.”
This remains the case. We are growing quickly and show no signs of taking our foot off of the gas. Sometimes I wish we would, but other days I remind myself this is the speed I like.
Action Item: Expect the best and do what I can to help it persist.