Avoiding enterprise “funny money” 🔮
The virtues of subjective-but-informed-decisions, hard truths of founder-led sales, and where to start with enterprise deals.
Welcome to the 92nd edition of The SaaS Baton.
A fortnightly newsletter that brings you hand-curated pieces of advice drawn from the thoughtful founder-to-founder exchanges and interviews taking place on Relay (curated with 💛 at Chargebee for Startups) and the interwebz. So, stay tuned!
(Apologies for the slight disruption in scheduling over the past couple editions. We’re back at it. And you can now expect to hear from us every other Thursday.)
In this edition, you’ll find the following instructive and inspiring pickings:
#1: Sentry’s co-founder and CTO, David Cramer, ardently argues how A/B testing is often a deceptive cushion for good decision making and thus is simultaneously at odds with speed, clarity, and organizational learning.
#2: Plain’s co-founder and CEO, Simon Rohrbach, writes about the many tough-yet-enlivening realizations he has had while learning to love sales as a designer turned founder.
#3: Writer’s co-founder and CEO, May Habib, offers astute observations on how they’ve approached their enterprise commitment (for a generative AI product) from the outset — “succeed in enterprise, first, and the deals will come, second.”
Finding these discerning founder takes valuable? Please consider sharing this edition with an ever-curious teammate or a much-cherished SaaS friend. 🙂 New readers can sign up here.
#1: Optimizing for taste (over A/B tests) 🎷
(From: Sentry’s David Cramer) (Source: cramr)
There are many, many reasons I am opposed, but the one we should care about is how it [A/B testing] fosters a culture of decision paralysis. It fosters a culture of decision making without having an opinion, without having to put a stake in the ground. It fosters a culture where making a quick buck trumps a great product experience.
That goes wildly against our core values, how we built Sentry, and what we want Sentry to be. Pixels Matter, one of our core values, is centered around caring about the small details, and that by its very nature is subjective.
What details? Which ones matter? Those decisions all center on taste, and around someone making a subjective decision.
…
Small targets drive small outcomes.
…often tests are focused on small targets, on the tiniest of incremental change. This is by design! It’s to ensure you can identify what is actually driving success. Unfortunately now we’re back to small outcomes.
“Which text for the button will perform the best?” - well if only a few hundred people click that button in a month, the answer is it doesn’t fucking matter.
Additionally a few hundred clicks is not going to be enough data to achieve a statistically significant result. While I realize this anecdote is extreme, it’s representative of the general problem.
Opportunity Cost.
…
A good example of this in practice is Sentry Bundles. We launched the bundles to only new customers (this limits exposure and risk), and we’re able to measure the cohort of bundles vs prior cohorts of non-bundles. Are there other changes that might impact the results here? Sure, but that impact - from experience and raw data points - is minimal, especially because our period of observation is only 4 weeks long.
You could do this with A/B testing as well, but what would you gain? At best an accuracy improvement, but more likely the data would be insignificant (we only get a few thousand paid customers per month in total).
We could also A/B test different price points, but we’d rather have an opinion on what the right package and price point is, and prove if that’s going to work or not. If it doesn’t, we’ll take those learnings and iterate on a v2. Our focus is on narrative and customer experience, not min/max on the bottom line.
Experiments as a substitute for data.
…
Some months back we changed the New Project flow to create multiple projects instead of one. The thesis here was that customers have multiple projects, so we should prompt them to set them up all right away to improve expansion. If this triggers your spidey sense, it should!
One of the pieces of feedback we had when this idea was pitched is that users generally are not working on multiple applications at the same time, so prompting them to set up multiple applications would increase friction. This feature was implemented, it was run through a controlled A/B test, and that test suggested the multi-project creation was better. Was the feedback wrong then?
Months later we noticed an activation issue, and lo and behold we found there were a lot of accounts where they had many empty projects. Reasoning would suggest that yeah, obviously this would create issues, because instead of an account having one project, which is fairly easy to navigate and comprehend, accounts now had multiple projects, many of which aren’t set up.
You combine that with the fact that most customers are on our Team plan, which does not allow cross-project search, and you can easily understand why the user experience is bad. Data is not a substitute for critical thinking.
The test said it was successful, but the outcomes, which is what you base future decisions on, showed otherwise. The lesson here is not that A/B testing was at fault, but that running experiments is never a substitute for a vetted hypothesis.
A/B tests are useful to the extent that human behavior is aligned with a definition of correctness.
The last issue I’ll touch on goes back to the purpose of A/B testing, and our misuse of it at Sentry. There are two common places I’ve seen us go wrong with this at Sentry. The first I mentioned several times above, where we’re commonly lacking enough data points to reach an accurate conclusion (very often true in Enterprise software, and will be true in most testing methodologies for us).
The second is far more nuanced, but is key to why this is valuable in performance marketing and less so in engineering. A/B testing is testing *human behavior*, it is not testing correctness, yet we’ve tried to lean on this concept historically to validate if some code we’ve written is *more correct*.
…
“No A/B testing is certainly better than bad A/B testing”
Our objective at Sentry isn’t, and hasn’t ever been to be great at running this kind of experimentation, which may explain why many of these attempts have been more failure-prone. That is why we are choosing to continue to prioritize taste at Sentry.
That taste is curated through hiring, and comes from the team’s domain expertise, their diversity of background, and their learnings both building and using our product. It comes from talking with customers on Twitter, from engaging them in support tickets and on Github, it comes from direct transparent conversation.
This is the kind of data we use to inform our decisions here at Sentry. Sometimes those decisions, those subjective-but-informed decisions, will lead to a failure. Those failures help us make better decisions in the future.
It means we are optimizing for iteration speed — another one of our core values. That willingness to iterate, to fail and move forward quickly, that is what drives great outcomes, and that is the culture we want at Sentry.
I’d like to leave you with one lesson I’ve kept with me over the years. The strength of making a decision is making it. You can always make a new one later. Choose the obvious path forward, and if you don’t see one, find someone who does.
#2: Designer >> Founder/CEO/Salesperson 🤝
(From: Plain’s Simon Rohrbach) (Source: Twitter)
For most of my life, I've been a designer. Over my time building Plain, I've shifted most of my attention to sales. At this point, I probably spend 70% of my day with customers in one form or another.
Every founder I speak to finds this transition one of the hardest to manage. Here are a few things I've learned along the way.
1. First, a hard truth: You have to be the one selling.
You probably turn your nose up at ‘sales tactics’ and don’t want to become ‘that’ person. But simply put, your company depends on revenue and unless your co-founder is an Account Exec, the person best placed to go and get revenue is you.
…
2. You can’t decide how you sell. Your customers decide how they buy.
We started Plain with the assumption that we’d mostly sell our product bottom-up, rather than top-down. We wanted to build a product that was bought because people were excited about it, not one that had to be sold.
But here’s the catch: the end state for almost every B2B SaaS business is enterprise. It doesn’t mean you don’t have self serve (you should!), but winning enterprise deals is typically the only way to create a financially sustainable business.
…
3. Not all problems are sold equally.
Some products solve existing and known problems, but do so in novel, better or more specific ways that capitalise on market shifts. For example, we’re capitalising on a shift in how and where B2B support is done.
Other products solve previously unsolved problems. People sometimes describe these as ‘blue ocean’ markets. Which one you go after will have a dramatic effect on how you sell, and how easy/hard it is.
…
4. Sales is like user research
…
So think of your sales calls like research calls. The goal of a first call with a customer is to understand their pain, why they want to solve it, and whether this is going to be an opportunity worth pursuing.
Note that none of that involves you showing your product. Instead, it’s about understanding the problem.
For your first sales calls, try to spend the first 20mins just understanding the problem. Only demo to prove something specific. All the information you gain now will save you many weeks and months of roadmap.
5. A great sales process is about solving problems
A lot of people have the misconception that sales is doing demos and asking ‘So, will you buy?’ at the end. That’s totally not the case.
As a designer or engineer, your job is to listen to people and identify what problems they have.
Sales is no different.
To get a customer on board, you have to listen to them, identify what problems they have and solve those. The difference here is that you’re not just listening for product problems (those matter too, but in a different way), but process and positioning problems.
…
6. If you understand the problem well, ‘selling’ becomes easy.
The scenario to avoid is signing a customer without clearly understanding the problem you’re solving for them, and why that problem matters so much. If you don’t understand it, you won’t be clear about it with them, they won’t know what to expect from your product, and everyone loses.
…
7. You’re going to learn a new job – don’t do it alone.
When I first started out as a designer, I was naive, made stupid mistakes, and didn’t know what I didn’t know. (Sometimes I wonder if anything has actually changed since then). Over the years, I learned, was taught and mentored, and became a better designer.
But that took years, and a lot of help from others around me.
Your company doesn’t have years, but more likely around 18 months.
Just like you wouldn’t expect your first hire to go from junior to CRO in the space of 18 months, you won’t be able to make that shift either.
But you can go from not-so-good salesperson to ok-ish sales person in that time with a bit of help: Get yourself a sales coach, and do it now.
…
8. Having a product background is a superpower – use it.
One of the great advantages is being a designer – or product person of any discipline, really – is that you have all the tools needed to go from a customer call one minute, to iterating on your landing page copy and product the next.
Your interactions with customers give you better context than most on the problem your product needs to solve, and that puts you in a great position to iterate your product super quickly with a high degree of accuracy.
Connecting these two things is where the magic happens.
8. Sales techniques are there for a reason. Show up with humility and curiosity.
I encounter many product founders who turn their nose up at what they think are typical sales tactics.
For example, I used to *hate* the idea of outbounding people. I hated receiving outbounds myself. If my product solved an important problem, why do I have do spam people’s inboxes about it? They’ll hear about us from all the word of mouth!
Yes, they might. Eventually. You don’t control when your buyer at one of your must-win companies asks their friends about what customer support tool they use, or types "customer support tool" into Google. You do however control when they get an email from you.
These tactics are typically (not always, but typically) there because they work, and people smarter and more experienced than you figured them out and improved them over a long period of time.
…
But suffice to say, I’m having a great time doing sales. I used to hate it, and now I love it. There’s nothing quite like getting someone excited about the product you’ve worked so hard to build. It’s the most rewarding feeling in the world.
Related Relay reading:
Crossbeam’s co-founder, Bob Moore, on how founder-led sales is an instructive validation tool (“I think the most important way to enter a new market is founder-led sales. It isn’t a volume play, but it’s the ultimate ‘do things that don’t scale’ technique.”)
Airplane’s co-founder, Ravi Parikh, on arriving at the fundamental logic of early-stage selling as a technical founder (“It sounds very basic and very stupid. But you know, as first-time founders who are just engineers by background, our instinct was to engineer every system. And as basic and simple as it sounds, just the idea that a human being should be proactive in sort of getting in touch with people was sort of, in and of itself, a big revelation.”)
#3: Avoiding enterprise “funny money” 🔮
(From: Writer’s May Habib) (Source: In Depth)
It [enterprise] definitely is not for everybody.
You’re onsite. You’re doing workshops. You’re doing office hours. But it is so impactful. There are a lot of our enterprise customers who tell us that our account teams (the solution architects, customer engineers, account managers) know their organisations better than they do. That’s really cool.
…
Succeed in enterprise, first, and the deals will come, second.
The deals around POCs, innovation, and sort of, I call it funny money. You don’t really want that stuff. That is a distraction. It’s easy to cut. Your team would spend a lot of time…And I think that’s why a lot of founders ignore the enterprise. Or sequence their way through it years later.
[Knowing when you’re selling into ‘funny money’]
The person has an innovation title and the money is innovation money. Versus the CIO is funding it or literally the head of North America has it as their ops budget…
In generative AI, you’re in a constant state of reselling because it’s such a noisy market. Your buyer, your champion, there are so many shiny things coming their way. So many people promising to help them sell more, save more, etc.
I think in the enterprise the price is certainly bigger ACV. If you do it right, you’re so sticky, you’re going to help everybody ignore all that shiny stuff. I think it’s just so much easier in mid-market or SMB from a functionality perspective to just switch to something else.
Especially in generative AI where it can be confusing for a while. It’s going to be hard to scale a business if your buyers are constantly shifting focus instead of paying attention.
There are a lot of reasons to go after it. But I do think focusing on the actual urgency and pain and business need and not the ‘generative AI, let’s try stuff’ fund. It’s kind of like classic SaaS.
Don’t let the sexiness of the space distract from the fact that sooner or later folks will be asking to see ROI and actually for a lot of enterprise deals that happens really quickly. Meaning there’s a lot of work about work.
If you’re productizing that, whether it’s processes and approach or actually in the product (we do both). Your teams can spend a lot of cycles on that. And not actually deliver the value and the product and the rollout etc.
After you have proven the business value, that’s actually when the deal starts. There’s got to be a really compelling thing that has been proven already for you to get serious money out of the real budget. Serious money out of the real people.
It’s as simple as that.
Because we’re not Microsoft. We’re not Google. They could be fired for choosing Writer. And it is amazing to be able to see the trust with that team and community that develops around us building reference customers and just reference peers around that new champion/new buyer….
We get our communities together a lot at every level. The AI program directors, the managers, the practitioners, the executives…This is a really cool industry in that, it’s not like execs are like, ‘hey, person, three levels below me, go try out this thing.’ They’re hands-on.
We’ve had C-level people build apps in Writer. It’s so cool. …We’re doing another customer conference…the two people who’re giving the app studio workshop are customers are champions of customers. It’s not our product team.
…if you’re a founder in AI, the enterprise is a great place to be because there are so many people that are excited and the impact can be so large. But if they are sticking their neck out for you. You’ve got to be there for them. And that I think is the biggest thing.
Related Relay reading:
Nira’s co-founder, Hiten Shah, on how market timing heavily dictates enterprise selling (“Having sold Nira in the current market, I’ve learned that enterprise sales have much to do with market timing than most people appreciate.”)
🤝 Founder social:
Thanks for reading! 🌻
Team Relay (Chargebee for Startups)