SendGrid is a cloud email provider that handles sending for any email use case, from password resets to shipping confirmations to monthly newsletters. SendGrid offers sending via our API or SMTP service (for developers) and via our online email marketing tool to build and send campaigns (for marketers). Those Uber receipts you get during a Friday night out on the town? Uber can't just send those from their Gmail account—they were sent through SendGrid. And that email from Spotify with discounted Beyoncé tickets for being such an avid and loyal listener? Yep, that's SendGrid, too.
One of the most challenging and interesting projects I have worked on during my time at SendGrid thus far has been the pricing page on our marketing website. Our pricing model is complex—there are three paid plans with differing feature sets, and within each of these paid plans are additional volume buckets, and each plan + volume bucket combo comes with different costs for overages. On top of that, if you'd like to use our email marketing tool, it's an additional $10 per month for every 10,000 contacts you need to store in our system. We also offer a free trial, which mirrors our cheapest plan for the first 30 days, and then drops to a reduced sending volume after that. Oh, and we have high volume plans for our enterprise-level customers.
Are you confused, yet? Our potential customers sure were, and confused customers don't convert as well as customers who feel assured and confident. So naturally, our first order of business for this project was to figure out how to make this page more clear, and also reveal some more transparency into the nuances of our pricing model:
The pricing page worked okay for our developer persona who only needed to consider desired featured set and email send volume. For our marketer persona, however, this pricing page was pretty painful. The disconnect between the section of the page that outlined Marketing Campaigns features and pricing was visually separated from the rest of the plans above, so it was not clear that a volume-based plan was required to sign up for Marketing Campaigns. It was also very difficult for someone interesting in Marketing Campaigns to estimate their monthly cost, because there were so many inputs to consider. Additionally, we needed a layout that would scale as we added additional volume buckets within each plan.
To alleviate some of these pains, we tested a couple of different pricing pages and gauged the impact. The first iteration of this test did not include any qualitative usability testing—we jumped straight into A/B testing. These are the variations we experimented with:
We ran the A/B test on these pages for 2-3 weeks to see if we could see some statistically significant changes. While we did see an uptick in paid plan signups with both designs (36% for the version with the sliders and 7% for the version with the volume toggles), we did not hit significance with any of our goals. As such, we decided to take these learnings and return to the drawing board a couple of months later. In retrospect, I believe we should have let the A/B test run just a bit longer to see if we could hit significance with more data points in the mix.
However, since we didn't hit significance with our changes in the first iteration, we decided to really give the page an overhaul. This resulted in many a whiteboarding session, and it also meant taking the designs through qualitative testing before running any A/B tests. After a couple of rounds of testing, the designs transformed quite a bit—I don't happen to have all of the iterations any longer, but one of the more interesting options that didn't make it past qual testing included completely obscuring the volume buckets by asking for sending volume and contact storage first.
This was a clever concept that I believe we will come back to as we further improve our pricing page, but it caused some confused in testing when users wanted to understand how their monthly estimate broke down. When we revealed the estimate break down, we exposed the volume buckets, and this caused some hiccups that were tricky to work around.
Here is the pricing page that we finally settled on and moved into the A/B testing phase:
This two-step, progressive approach does not entirely obscure the volume buckets within each plan but instead helps the user choose the most economical plan based on their sending volume. We also made a multitude of other changes and enhancements, including revealing the prorated cost based on the date of signup, free plan callouts higher up on the page, and Marketing Campaigns pricing that was easier to understand.
With this version of the page, we saw a statistically significant 19% increase in free plan signups and a statistically significant 15% increase in overall plan signups. We did not, however, hit significance with paid plan signups—in fact, we saw a slight downtick in paid plan signups but this did not hit significance after running the test for nearly 6 weeks. Because we saw an uptick in free signups and a slight downtick in paid, our data analyst followed the cohort of users who were exposed to the new version of the pricing page for a couple of months after our A/B test and found that we saw an increase in revenue from the test cohort. As such, we decided to launch the new pricing page! This was a big win, as our pricing page hadn't changed in a couple of years prior to this launch.
The new pricing page has been live for about a year now, and we are ramping up to take on another overhaul with the learnings that we've gathered over the past 12 months. While the new page did increase overall signups, through usability testing we have found that hiding the plan matrix when a user firsts lands on the page is a pain point, especially for our developer persona who is more price-sensitive and really wants to understand the plan breakdown. We'd like to ease this tension in the next iteration of the page. We've also found that our developer persona, who is not using Marketing Campaigns, is sometimes confused by the contacts slider. Because our developer persona and marketing persona have such different needs when coming to this page, we will also be testing a model that splits out pricing for API/SMTP and pricing for Marketing Campaigns to help give each persona what they are looking for.