This post is a transcript from Adam’s keynote presentation at
The Austin Homegrown API Meetup in September 2019.
Hey everybody. I’m Adam. I have been coming to the Austin API meetup on-and-off for five or six years and it was actually a little over four years ago that I was sitting, not over there because it wasn’t here, it was in a different location. It was much smaller and we had no nice food or drinks, but we still had good content.
I was sitting there and I was watching Jason Harmon talks, whoever was supposed to be presenting and our normal host was out sick. Jason was there and he did an impromptu presentation about how he was using Swagger at PayPal and I thought, “Oh, this is really interesting.” We looked at Swagger and my team said we couldn’t use it.
So, after watching him use Swagger for an hour, I thought, “I need to go back to my team and I need to say, ‘Hey, you know what? We can use Swagger.'” So, I go back and they’re like, “No, we looked at it before, we can’t use it.” I was like, “Look at it again, please, look at it again.” And sure enough it was like, “Oh look, Swagger 2 is out and we were looking at 1.2 before, so we can use it, Adam,” and I thought, “This is fantastic.” So, I got super psyched about it and I said,
“Let’s describe our whole API using it and then let’s fire up the Swagger UI and have our documentation automatically render from this API definition.”
So, we do that. It took us two weeks to describe our whole API.
It was hundreds of operations and we didn’t know what we were doing as well. So, we finally do that. We fire up Swagger UI and I say, “Huh, we must’ve done something wrong because this is not how it normally looks.”
Normally you can see what the request parameters are and for when it was showing our requests, it was just showing that it takes a body. I’m like, “Yeah, the request does take a body, but what is in that body?” And so, this led me to go down the path of eventually developing Redoc, which is a popular open source, open API documentation software. We built it for Rebilly and we open sourced it. We didn’t really promote it, we didn’t do much marketing, but it became very popular because a lot of people had that same exact problem that we had sure enough and today it’s used at approximately 80% of the Global 500, and ShipEngine too.
“This led me to go down the path of eventually developing Redoc, which is a popular open source, open API documentation software.”
So, I thought Global 501. 81% and the thing that I found interesting when I started seeing who’s using this is a lot of these companies, I didn’t even know that they have technology. It’s like you make furniture or speakers or TVs. Well, they all actually are using APIs, which is really fantastic. So, I think this is something that maybe wasn’t the case six years ago when I was sitting here in the meetup, but now everyone’s using APIs and a lot of people are familiar with open API, which was formerly known as Swagger.
So, when I said Swagger from earlier, that’s what I was talking about and naturally sitting in the audience has led me full circle to where I am today, having spun off from Rebilly, a new company, Redocly, where we make a commercial version of our API documentation software. This talk is about docs.
“When do you write the docs for an API?“
So, just want to do a poll—show of hands here— no shame in this. Who never writes the docs? So, you’ve got some APIs where you never write any docs for them? Does anybody use them or are you the only user?
Yeah. Okay. What about last? Who writes the docs last?
This is what we heard about earlier, this one case of something getting out there. Okay. So, let’s deploy it. Let’s get some users using it somehow and then let’s get the docs going. Who does it in parallel? So, we think you know what? We’re going to develop and write the docs at the same time. So, okay, we’ve got some people somewhere. Who writes the docs first? Nobody, right?
So, since I’ve been in this arena for a little while and I get to talk to a lot of interesting characters, I find it’s pretty interesting, but maybe it’s about 5% of the people who write the docs first and I’m not talking about the published novel, I’m talking about the rough draft, right? The rough draft version. So, usually when… And I have the privilege of working with about 50 different developers and many times there’s that developer who says, “When do you write the doc? Never.”
So, I have this thing, the smell test to see is this actually viable? So, this is my smell test to see when should you write the docs? Never. It might be it. It might be possible. So, here we go. Now I know a toilet is not an API, but it is an interface. So, look mom, no docs, right? We all can use it here, I think.
So, is it intuitive? Is it discoverable? And is it ubiquitous? Maybe not the talk you thought you were going to get tonight, but I’m going with it anyway and it may seem this way on the surface, but you know what? Totally, it’s not really ubiquitous; it depends on where you live in the world.
So, if you’re living where toilets are relatively new or you’re installing a toilet where you may have visitors from people who live somewhere where toilets are relatively new, then you might need some docs even though they’re pretty small and basic, it can be helpful. So, when it comes to my API, look, mom, no docs, she’s not happy. APIs are not intuitive. I know you use it the term that your endpoint names are intuitive, but they’re still not really that intuitive. Nobody would guess them. I mean, maybe they can guess a few of them.
“APIs are not intuitive.”
So in that case, they’re not necessarily discoverable either and they’re definitely not ubiquitous. Everybody’s API is different. So, using the process of elimination, which I use in my life, you’ve been voted off the island, you got to leave. Never. So, let’s take a look at last. Should we write the docs after we’ve finished developing the API? At least hopefully then we only need to write them once.
All right, look mom, it’s okay. We’ll write them last. Oh, okay and this is a really beautiful picture. So, I was thinking this looks really good. Let’s put all the developers in a nice barn. They’ll do their work and then after that, they’ll take a nap and they’ll place these API docs at the top of this thing. What is this thing? It’s a silo and out comes this beautiful documentation. Well, I have some problems with this approach. It actually creates some internal silos. It leads to increased rework, which you alluded to in that one case and could lead to decreased usage.
“Most people are writing APIs because they want someone to use them and it will create business value.”
All of this, writing the docs last, can lead to increased team tension and it’s not fun to work in a tense environment. So, using the process of elimination, I’ve crossed that one off the list.
Next: Parallel. All right. Docs parallel. Here I’m trying to draw a parallel between something, a wire frame, and the rough draft of your docs. Would you wire frame before parallel or after you build a user interface? In parallel of course, right? No. Yes. Before. So, we wire frame first. The wire frame is the equivalent of the very rough draft of the docs.
So, we don’t want to do this in parallel actually. Maybe we want to… If we want to make it more parallel, comparison. Maybe we want to refine the wire frames and go from low fidelity to high fidelity in parallel, but we want to start with something low fidelity first. So, now that we’ve picked our winner, what is the goal of this? Why write the docs first?
We want to have and encourage better teamwork and this is not just among developers, but among all departments. You can involve even marketing in writing the docs. Less rework, right? We want to prevent rework. It’s faster to rewrite a paragraph than it is to rewrite a module. Increase usage. We want people to use our APIs. I mean, nobody wants to build something that gets thrown away consistently.
Sometimes it’s okay, but if we get to throw away your own stuff, it feels good, but when someone else is throwing away your stuff, it doesn’t. We want to reduce support in questions. This is a common objective and this approach allows us to do this because we’re approaching it with more empathy for the user, who in this case is the developer who’s consuming our API and we want to reduce the time to usage. So, we don’t want someone to have to spend six months figuring out how to use our API. They probably won’t have the persistence to do it anyway. So, we want a quick start.
So, how? So, I say, “Yeah, I’m going to do this first, but I still have no docs. What am I going to do? How do I get started?” So, the thing that I like to start with is the foundational principles or my philosophy first is take an approach of docs like code. Has anybody heard of that phrase? Yeah? All right. So, it seems like it’s about 50/50 split. So, it is basically about treating your documentation just like you would treat your code, checking it in or using version control or source control. In our case, we like to use Git and GitHub.
SSOT. I saw this acronym for the first time just a couple of weeks ago and I was like, “What is this? Why? Why is this acronym here?” It took me awhile to figure out. It means: Single source of truth.
I say that phrase all the time and someone had abbreviated it. I was like, “Ah, okay. That’s much easier for me to type out.” SSOT. SSOT. Okay, so single source of truth. I like the DRY principle, which is don’t repeat yourself, but as it applies to documentation is if you’re in APIs, if you’re going to define an API, you write your API description or definition, you don’t want to then write contextual documentation that repeats parts of that definition because you have the likelihood that in the future those will diverge and you’ll have documentation that is no longer consistent with your API definition.
“So, to keep those together, stick with a single source of truth. If you write an API definition, use it to power your contextual documentation and focus on the business problem.”
So, make sure your API is solving something.
So, what is a docs skeleton? So, I started out with a skeleton. This is how I get started faster. I’m going to show you something in a couple minutes that I put together in about two hours and a lot of the reason I’m able to do that, not that it’s exceptionally fast is because I’m not starting from scratch, I’m starting from a skeleton. A skeleton, my skeleton is a self contained portal that enables an internal team to collaborate using the familiar docs like code. We get instant feedback on API definition mistakes.
My workflow is author, validate, commit and push. We use the GitHub flow, which is branch, commit, create a PR, peer review, merge. We can run the docs in our local environment.
I’m going to give you a walk through of the workflow. So, earlier I was thinking about what company should I start next? And I thought there’s this problem with anti money laundering compliance. You need to know your customer and it’s a really tedious and painful to do that. A lot of times you’re sending documents back and forth and the first time you send the document might not be appropriate. You might have missed something and it can take days or a week to get the documents and to know your customer. There’s got to be some tools that automate that and I found some.
There’s a couple that are really expensive and there’s a couple that just have little pieces of a solution. I thought, “You know what? This would be a great example for an API, because I found a problem that was bothering me and I want to solve it and I want to do it by creating an API.” So, I start by spinning up my portal, which is this Redocly software that we’ve created and that the portal is generating and let’s say this is my local host. So, I’ve got this running.
I’ve started by just stating the problem. What’s the problem here? What do we need? We need to save money, increase user conversions because users are dropping off. If you are in a regulated industry where your customers need to provide some sort of documentation to prove who they are to do business with you, they’re dropping off right now. So, we need to reduce that friction, increase user conversions, protect your data privacy. All of the tools I saw out there are cloud based tools.
“What companies want to send their data across the cloud to a third party to do this type of service? All of the products out there are doing that right now. Save time and costs. It’s really costly.”
People are manually reviewing these things and meet compliance mandates. That’s the main thing, right? Stay out of prison. So at least for me, that’s what I’m trying to do. So, I know we all have different things, but that’s mine. So, zoom in a little bit. So, what I did was I borrowed a couple of people. Yeah? Anti money laundering. Yeah. KYC is know your customer. Yeah. AML is one of the main laws, one of the main reasons you need to do that in certain industries like banking, but it’s not just banking.
They’ll need to know their customers. They need to verify, they need to do age verification or they should. So, started off by looking at the pain and just using some Markdown, which is comfortable for us. I’ve formulated this page and another page and I formulated the left nav and I borrowed because there are my customer avatar. This is Lou, he’s the money laundering regulatory officer at a company and he’s very process oriented and rule bound. He’s not easygoing and he’s not getting along very well with sales and marketing because they feel like he’s slowing them down. Yeah, he’s eating into their commissions.
So, I started thinking about the use cases. So, this was coming in from marketing, looking at the market size. Apparently this is a $750 million a year industry and growing and then digging into some of the use cases, specifically gather and analyze documents and in this case, what I did was we’ve integrated into the portal a mermaid JS.
I don’t know if anybody’s familiar with Mermaid JS, but it allows you to write in Markdown a code snippet where in Markdown you usually use the little three hash marks and then you can put the language that you want to type in like JS, right? And then you get code highlighting for JavaScript. If you put three hash marks and you write Mermaid. In our case, because we’ve integrated it, now I can write a specific syntax for Mermaid and in this case, I made a sequence diagram. I don’t know if it’s any good, but this is my vision of the sequence for your app, how your customer would be integrated between your app, your customer and re-comply now. This product.
So, then I also had the luxury of being able to bug my product designer and said, “Hey, I need a couple of wire frames.” It’s a customer journey where someone’s going to request their ID, right? And they’re either going to, if they’re on a mobile phone, they could take a picture or upload an image and they then they’ll be able to verify, they’ll see a preview of it, then they’ll verify it. Then if it doesn’t, if they upload something bad, right? We analyze it. Then we give them some pointers about how to make it better, like if it was cut off or blurry or something like that. Then when they do it again and they do it right, we give them this thank you and they continue on back to where they were in their journey and then this product would also need to have some sort of ongoing alerts.
That’s another use case is, “Oh, this person’s still my customer and I need to make sure they’re not in the sanctions list,” right? This person hasn’t bombed someone, bombed one of our allies. So, we need to make sure they’re not in the sanctions list.
Well, we have to do that by law or else we might go to prison and we need verification that we’ve done that so we can now monitor just by subscribing to some webhook, say, “Hey, here’s my customer list. I want to subscribe to that,” and since Lou is very rigid in his ways, he’ll still be able to do the manual reviews if he wants to, but he’ll have a nicer work bench for doing that. In here we pull in the corresponding APIs, use open API to describe some of the APIs that I thought we would need like creating a customer and I thought our response back might have some read only parameters like if their KYC document status, if they’re in the sanctions list or in the PEP list. PEP, another acronym is Politically Exposed Person. So, some industries you also have to report if you’re doing business with the politician or relative of a politician.
It’s just part of doing business and then I thought for the document analysis, how would this work? Well, we need to be able to create a file, right? So, someone needs to be able to create a file. Maybe they’ll be doing it from in-line as I had shown in that app, but maybe from a URL.
So, you use this feature, one of, if you’re not familiar with and in am open API schema, you can use certain relationships from adjacent schema, like one of, any of, all of. So, one of allows us to have two possible request bodies for this API because the file might come from a URL or you might just send up the base 64 encoded raw data and then we need to create the document itself. So, we need that. We need that file. The customer and the type of document that we’re uploading. Is this a proof of identity or is it a proof of address or what? I didn’t think this through fully, but it gives us a much better idea before we start actually building this, is this even what we want to build? Is this the right thing? And is it from a user’s perspective? Would they even be able to use this thing?
I don’t know. I mean, I think it needs to be refined for sure, it’s incomplete, but because I leveraged the skeleton, the portal and I have decent knowledge of open API. So, I don’t struggle with API definitions, I was able to put this together in about two hours. So, with the designer. I have zero design skills, but it is just the most common reason I get as to why we don’t write the docs first is we don’t have time to do that. We don’t have the time to do that and it seems like that’s valid because depending on where you’re at in your journey, when I was back here, I’m like, “I want do that, but I don’t know where to start,” is where I was at back here. I wanted to do that, but I didn’t know where to start and at the same time it involves leaving your normal comfort zone because you have to go and get the technical documentation writer and someone from marketing.
We can just get [a marketing person] and maybe a designer and you need to pull them all together and you’re doing something new that you haven’t done before, which is create a product before you really know what you’re creating. And so, I think the number one reason why people don’t do this is because of that discomfort or unfamiliar feeling. Once you’ve done it three times, you’ll be much better at it. No promise on the first two. The other thing that I’d really recommend is looking back at this slide, look at this wire frame with Comic Sans. It’s a disgrace to designers, right? It’s a disgrace to designers. It looks great to me though, but the ad designer, I could give a designer a heart attack by showing them this and telling them, “Seriously, this is what I want to build,” and I could probably do the same thing with the documentarian, right?
Because we want to start out as it’s a really rough draft. It’s a wire frame. We have to understand that this is going to be very far from perfect.
“We need the documentarian to be okay with putting out below-publishable quality, knowing that this is going to be used internally at first and that as the project evolves, they’ll need to continually improve and revise and make it polished and perfect.”
Or nothing’s perfect, but as close to perfect as possible. So, that’s another challenge that you might have to overcome as you pursue this route or this journey.
Any thoughts or questions?
View the video and slides from Adam’s presentation
Want to learn more about APIs?
Join us for our next meetup!
About Author
Leave a Reply