This post is a transcript from Karen White’s keynote presentation at
The Austin Homegrown API Meetup in May 2019.
So we’re going to be talking about redesigning a developer portal that your developers will actually want to use. I’m Karen White. I am a developer advocate at a company called Big Commerce; we are headquartered here in Austin.
I work with our developer ecosystem, representing their interests to our product and our documentation teams. And today I’m going to be talking to you about a project that we’ve been working on really over the last year, which was taking our developer portal from something that used to be pretty uninspired to something that we’re actually now very proud of and our developers love to use.
This project really kind of evolved in parallel with how we as a company think about developer experience. So to understand kind of why we undertook this project to begin with, you really have to understand a little bit about us as a company, right? We are a SaaS eCommerce platform, so think websites with the shopping cart. And when you think of SaaS, you might think of this.
The black box, right? So unlike an on premise solution where you know you have full access to everything, our core application, all of our services are running in the cloud. So if you’re a developer who’s used to kind of answering their own questions and solving problems by going straight to the source code, that’s not really gonna be an option in this case. So that can feel limiting to some developers.
But what we’re moving toward is a way to leverage all of the things that actually are pretty awesome about SaaS. So things like not having to worry about updates and not having to manage your own infrastructure, but also let some light into that black box.
And the way that we’re doing that is by really prioritizing API coverage and our strategy. So we’re opening APIs for all of the core components of our product. And what that means for our product team is, when they’re pushing out a new feature, part of that definition of done is also including an API that goes along with that feature. And then what we’re moving toward increasingly is more of an API-first approach. So we’ll build that public API first, make it available to the ecosystem, and then build our own core feature on top of that same API.
“We’re opening APIs for all of the core components of our product.”
And that brings us to kind of where we are today, which is this idea that we want to be the world’s most open SaaS company, but what does that mean really like to be open when you’re a SaaS company? So part of it is that API-first mentality, or at least that not complete until you have an API mentality. But it also means that we want you as a developer to be able to de-couple parts of the platform from the hole.
So, if you just want to use the catalog or if you just want to use the cart and checkout, that is something that you should be able to do. Or if you want to build a completely headless solution, use Big Commerce as your backend. Do use something else as your presentation layer. That’s what we’re moving toward as well.
That kind of flexibility is where we want to go as a company. But developers are clearly very central to that success. If you don’t empower your developers to do those things, then you’re not gonna be able to be successful there. So we have to give developers a place where they can find resources, get information, and do all of the things that they need to do to build those cool solutions on top of our APIs.
And this used to be that place. So, before we really came to this inflection point around the idea of an open SaaS API, first we were really more focused on getting developers to build apps and themes for our marketplace. So we had these two calls to action here at the top, see what kinds of apps that you can build or see what kinds of themes that you can build. And when you clicked on those links, they would actually take you to the app or the theme marketplace. So if I’m a developer and I’m on this homepage, I’d probably want to do one of three things, right? I want to quickly understand what this API is, how it works, and what I can do with it. I want to know the first steps that I need to take to start building an app, or I want to get started building a custom theme.
What I probably don’t want to do is go take a look at the app marketplace or the theme marketplace as though I were a non-technical user. So we definitely had some work to do with considering things like user journeys and how developers were actually going to use this space. But I think we were also kind of implicitly communicating some not so great things. Right?
We’re sort of saying by taking you over to the app marketplaces at the marketplace, we’re saying that we want you to build something that kinda starts benefiting us right away, but we kind of care less about actually helping you build that thing. And kind of beyond that, it felt a little tired, it felt like an afterthought and it felt like we weren’t really considering developers to be first-class users of our platform.
And, this is our portal after the redesign. Comparatively it feels fresh, it feels fun. But we’ve really considered here what developers are actually trying to accomplish. So we’ve pulled out links to all of the relevant resources, all of the main use cases and made them more discoverable kind of under the hood. We’ve re-platformed our API documentation—a shout out to Stoplight—and we’ve also consolidated everything onto one domain. So, it actually was not before.
We’ve reworked the menu structure and rethought this information architecture. There was kind of an element of house cleaning to this whole project, but it really was an opportunity to rework things from the ground up in a way that feels a lot more intentional and actually meets our users needs.
“It really was an opportunity to re-work things from the ground up in a way that feels a lot more intentional and actually meets our users needs.”
So when you begin this kind of renovation project, it can feel overwhelming, right? There are so many things that you want to do that we still want to do, honestly, that it’s really difficult to know where to begin. So you have to ask yourself, how does your developer portal actually fit with your company’s overall strategy? Something like documentation is going to be important. Clearly if you don’t have documentation, no one can figure out how to use your product. And that’s definitely not a good thing, but your developer portal is a lot more than just a place to stick your documentation.
There are all kinds of other reasons that developers are coming to this space. They want to connect with other members of the community, they want to manage their business relationship with you. Or, they might even be coming just to form a first impression about what developer experience is like on your platform.
We’re still kind of talking about the planning stages here, but it’s already time to talk about metrics, right? Because that’s going to influence what you measure later on down the line so that you can tell if your tactics are actually effective or not.
This framework might look familiar to some of you. This was popularized by Phil Leg getter in the DevRel world, and it’s become a really popular way to describe the funnel and all of these supporting activities that make up the developer user journey.
So… we’ve got awareness, acquisition, activation, retention, revenue, referral, and product.
Awareness is making sure that developers actually know that your product exists. Acquisition is going to look a little bit different depending on your product model, but it might be something like an account sign up. A activation is getting developers to actually take that first step and do the things. So, something like making the first API call.
Retention is when developers are actually using your product in production and maintaining that integration.
Revenue kind of speaks for itself.
Referral is when developers are actually advocating on your behalf and referring other developers to use your product.
And then, product is where things kind of come full circle and developers, either through their feedback or directly, are contributing back to the product itself.
And really, there are parts of your developer portal that can tie in to every single stage of that user. But, you really have to focus in on just a few of those stages so that you can focus your energy on where your company’s priorities are. Revenue is probably going to be a priority for your company. It is for most people and it might even be for your developer relations program, but that really should be more of an implicit goal rather than an explicit one.
Your portal is probably not going to be a revenue generating machine. And similarly, a product contribution is another really good goal to have. But depending on where your program is in its maturity—if you’re just getting started—that might not necessarily be the stage that you want to focus on right now. So, there really are always going to be more things that you want to do than you’re able to do at the time. You really have to pick a path and narrow your focus. But either way, you want to be thinking about what those goals are down the line so that you can think about how you’re going to measure that kind of success. So if you’re thinking about awareness and acquisition, for example, you’re going to be thinking about things like account signups, first API calls, right? So you have to build a way into your portal that you’re actually going to measure those things.
“You really have to pick a path and narrow your focus.”
One of the areas that you know where you should be focusing on is by talking to other internal stakeholders within your company. There are going to be teams outside of DevRel or documentation that have an interest in your developer portal—people like partner account managers. So these are the folks that are managing that business relationship with your community.
In our case: partners, our web development agencies, app developers that have a business relationship with us and then the product team. So they’re definitely gonna have an interest in making sure that these APIs that they’ve worked so hard to build are actually being documented in a way that encourages their use. But I think that we forget sometimes that the product team is also really hungry for feedback. So, they might be looking to your developer portal to source beta participants or just get general product feedback.
And then your leadership team. So, depending on your business model—and how developer experience is baked into the company culture—your developer portal may or may not be on their radar, but you do need to be able to kind of draw a straight line between the activities that you’re working on and then those high level goals that are coming down from the leadership team.
So at this point, we’ve talked a lot about what your company wants out of the developer portal, but what about the developers themselves?
In the beginning we did a lot of user feedback interviews with members of our developer community. Some of the things that we heard were already kind of on our radar. Things like a need for a client library in a language, more code samples. But we got a few pieces of feedback that really helped to see the portal through the user’s eyes. So, for example, one of the pieces of feedback that we got was to put all of our code samples on a dark background because that is consistent with the ideas that a lot of developers use, which sounds like such a simple thing, right? I mean it was definitely simple for us to put in place, but it has actually a pretty big impact on user experience. So before we got too much farther with this project, we wanted to understand that developer community in a deeper way. So we spent a few months doing research and putting together developer personas.
It was really important to me that these personas be rooted in data, right? So when we started, we did this research in-house actually. I might be kind of biased here, but I definitely think that this is something that you can do within your own team without necessarily hiring a third-party to do a study and still get a result that you can really base decisions around.
So we started by conducting a survey of the widest range of developers building on our product that we could. We distributed this to all of our registered partners, linked it from the documentation, sent it out on social, and then all in all we really got back a pretty decent response rate to base the personas off of.
And then our next step was to crunch the numbers. So I’m not a data analyst, but I definitely believe that you can still filter your data, use pivot tables and still pull out some insights that are really valuable. So what you’re looking for here are answers that are predicting other answers. So if a developer tells us that they’re self-taught, are they more likely to work in a particular programming language, or does their job title correlate with what a developer finds most frustrating day to day?
You want to start to form groupings that are based on a shared pain points, goals and frustrations. And that’s going to be more important for something like this than demographics, so things like age, location, or gender.
What we really want are personas that capture the priorities and the motivations of these developers. You also want to add some biographic detail and to give these personas some depth and make them feel real. But I also think that you should be a little careful with that. If you start getting too creative, this can really start to feel like a creative writing exercise and it’s going to undermine the way that people look at this as a valuable tool.
“What we really want are personas that capture the priorities and the motivations of these developers.”
So then once you get to this stage, you want to actually validate your assumption so far by talking to the developers that you represent. So we called in a smaller subsection of survey respondents for a second round of phone interviews, and those were more conversational, open-ended. We really wanted to learn who our developers were, how they work, what motivates them.
Now let’s shift gears a little bit and talk about kind of all of the things that make up your developer portal.
Developers who come to this space are trying to do one of four things probably, right? They’re looking for documentation, they’re looking for support, they want to connect with other members of the community, or they’re looking to do some kind of administrative task or manage their business relationship. Part of our journey and part of that process has been consolidating all those functions into a single portal.
We’re not all of the way yet there yet, but that’s definitely the goal. And that portal should cater to users who are at different stages of that developer journey. So a developer who’s just activated…so to speak… and created an account, they’re going to need access to documentation. They’re going to need to access to community.
Hopefully they don’t need support just yet at this point. But that should be something that’s accessible to them when they do. And then, if you support both technical and nontechnical users on your platform, which is the case for us, your access and authentication model needs a way to differentiate between those different user types.
“If you support both technical and nontechnical users on your platform, which is the case for us, your access and authentication model needs a way to differentiate between those different user types.”
And then last, if you have developers who have entered into some kind of business relationship with you—for example, by becoming a partner—they’re going to need a way to manage commissions, manage their app listings. Can I do all of those administrative things?
That is a lot of functionality that’s wrapped up into one single space. So what we think about is, how do we help developers navigate that intuitively? What we want to do, we want to do more with permissions, right? So there are different roles within an organization that are all going to need access to this developer portal. And we need to differentiate between a technical or developer role and somebody who’s in a business role because they’re doing different things.
And then how do you provide paths, right? Like how do you show developers intuitively what’s relevant to them? Beginners are going to need exploration materials. So things that let them try out the API quickly understand in a hands on way what this API is and what it does. And then experienced users are going to need references for quick lookups of information.
So building a developer portal like this is a journey, right? And we’re not 100% of the way where we want to be right now, but we think about it in terms of a maturity model where at level one you have this experience that is fragmented, it’s incomplete, and then at level four you’re providing a much more unified experience with this pathing and personalization.
So at level one, your portal might not even be adequate, honestly, for a developer to successfully complete an API call. It’s really this closed system. There’s no way to login. You might just have an email for support. And then at level two, we introduced some level of self-service capabilities. So a developer can generate a test or sandbox environment, and they can access documentation without any assistance from you.
And then at level three, we’re including really a lot of the hallmarks of a successful developer portal. We’ve got documentation in multiple programming languages, we’ve got code samples, and we’ve got an active community around the product.
At level four you have all of that, plus you introduce personalization, right? So when you login, we can identify you and we can show you the resources that are going to be relevant to you.
This is about where we are today. Over the next few months we’ll be kicking off phase two of our developer portal redesign and we hope to check all those boxes for level four as well.
So how do you gain a sense of where your program lies on that maturity model? So one way is by creating a blended scorecard that assesses how you’re doing across different areas. So we do this by scoring ourselves across a matrix of four categories.
So we’ve got openness and accessibility, environment and docs, API quality, ease-of-use, and then community and support. So this does two things, kind of, it gives you an idea of where you stand kind of overall and then how you’re doing on an individual category level basis. It really seems like metrics are kind of a hot topic in developer advocacy lately, but when it comes to scoring your developer portal, things can get really complicated. You have so much functionality that’s wrapped up in this space. You’ve got documentation, community, and all of those things actually have their own KPIs. So there’s not necessarily going to be one, you know magic number that tells you the health of your developer portal.
But what you really want to do is understand what your goals are in order to understand what user behaviors support those goals. So if you’re measuring page views, what goal does that support? Right? Or if your goal is activation, what are the user behaviors that would tell you that developers are actually making their first API call.
“Understand what your goals are in order to understand what user behaviors support those goals.”
And then even more importantly, which can you actually measure? So it’s also important to call out that in terms of measuring change, you need to establish a baseline. If you’re starting from a place where you have no data or bad data, it’s really not worth using that as a point of comparison. So you need to build up a certain amount of reliable data before you can use that as a starting point.
And changing behavior takes time. ‘Cause that’s really what we’re talking about here. So don’t be discouraged if you start putting these things into place and you don’t see any immediate change in your numbers. None of those benchmarks in the maturity model are going to be immediate gratification.
Honestly, in our experience, the only time that we’ve seen a really drastic jump in numbers was when we introduced single sign on for our community. So not too surprisingly, if you make it a lot easier to access a space, you’re going to see more engagement there. But that’s really been an outlier, right? Change is usually going to be a much more long-term process and you also are going to need the support from other internal stakeholders in your company to move that needle.
So, if your partner managers aren’t referring developers to the resources that you create, or if you don’t have product and leadership behind you, it’s going to be really difficult to actually see real change.
So redesigning a developer portal is a really big investment, but it’s also the biggest interface that you have with your developer community. So you want to make sure that you align your goals with the company’s strategy across all of the internal stakeholders who have an interest in developer activities. And you also want to make sure that you know your users because that is going to determine what you build. The more that you know about what your users want to accomplish and how they work, the better experience that you’re going to be able to provide for them.
“Redesigning a developer portal is a really big investment, but it’s also the biggest interface that you have with your developer community.”
And then finally, benchmark your progress against a maturity model that makes sense for your product and your community. It’s really tempting to just do what Twilio did or what Nexmo did. And there’s definitely something that we can learn from companies that are doing this really well, but there’s also not a one size fits all solution, so it’s not going to be an overnight process. But when you commit to a path, you’ll begin to create a developer portal that your company will be proud of and your developers will also love to use.
View the video and slides from Karen’s presentation
Want to learn more about APIs?
Join us for our next meetup!
About Author
Leave a Reply