These are the news items I've curated in my monitoring of the API space that have some relevance to the API definition conversation and I wanted to include in my research. I'm using all of these links to better understand how the space is defining not just their APIs, but their schema, and other moving parts of their API operations.10 Jul 2018
I’m doing a lot of thinking regarding how JSON PATCH can be applied because of my work with Streamdata.io. When you proxy an existing JSON API with Streamdata.io, after the initial response, every update sent over the wire is articulated as a JSON PATCH update, showing only what has changed. It is an efficient, and useful way to show what has changed with any JSON API response, while being very efficient about what you transmit with each API response, reducing polling, and taking advantage of HTTP caching.
As I’m writing an OpenAPI diff solution, helping understand the differences between OpenAPI definitions I’m importing, and allowing me to understand what has changed over time, I can’t help but think that JSON PATCH would be a great way to articulate change of the surface area of an API over time–that is, if everyone loyally used OpenAPI as their API contract. Providing an OpenAPI diff using JSON PATCH would be a great way to articulate an API road map, and tooling could be developed around it to help API providers publish their road map to their portal, and push out communications with API consumers. Helping everyone understand exactly what is changing in way that could be integrated into existing services, tooling, and systems–making change management a more real time, “pipelinable” (making this word up) affair.
I feel like this could help API providers better understand and articulate what might be breaking changes. There could be tooling and services that help quantify the scope of changes during the road map planning process, and teams could submit OpenAPI definitions before they ever get to work writing code, helping them better see how changes to the API contract will impact the road map. Then the same tooling and services could be used to articulate the road map to consumers, as the road map becomes approved, developed, and ultimately rolled out. With each OpenAPI JSON PATCH moving from road map to change log, keeping all stakeholders up to speed on what is happening across all API resources they depend on–documenting everything along the way.
I am going to think more about this as I evolve my open API lifecycle. How I can iterate a version of my OpenAPI definitions, evaluate the difference, and articulate each update using JSON PATCH. Since more of my API lifecycle is machine readable, I’m guessing I’m going to be able to use this approach beyond just the surface area of my API. I’m going to be able to use it to articulate the changes in my API pricing and plans, as well as licensing, terms of service, and other evolving elements of my operations. It is a concept that will take some serious simmering on the back burners of my platform, but a concept I haven’t been able to shake. So I might as well craft some stories about the approach, and see what I can move forward as I continue to define, design, and iterate on the APIs that drive my platform and API research forward.
API versioning is almost always one of the top attended discussions at conferences I help organize, and one of the first questions I get in the QA sessions at workshops I conduct. People want to understand the “right way” to version, when it my experience there is rarely ever a “right way” to version your APIs. There are commonly held practices regarding sensible ways to version your APIs, as well as dominant patterns for how you version APIs, but there isn’t any 100% solid answer to the question, despite what many folks might say.
In my experience, the most commonly held approach to properly versioning your APIs (if you are going to), is to put the major and minor version in your header and / or combine it with content-type negotiation via your header. However, even with this knowledge being widely held, the most dominant pattern for versioning your APIs is sticking it in the URL of your API. I know many API providers who put the version in the header, despite many on their team fully being aware that it is something that should be put in the header. So, why is this? Why do people still do it the “wrong way”, even though then know how to do it the “right way”?
I feel like this phenomenon reflects the wider API space, and how upside down many API belief systems are. People put the version in the URL because it is easier for them, and it is easier for their developers to understand. While headers are a native aspect of developing using the web, they are still very foreign and unknown to most developers. While this shows the lack of web literacy that is rampant amongst developers, it also demonstrates why simple web APIs have dominated the landscape–they are easy for a wide segment of developers to understand. An aspect of why this whole API thing has worked that many technologists overlook, and take for granted as they try to push the next trend or solution on the sector.
While conducting workshops, I always teach the more sensible patterns around versioning, but I can’t always sell them as the “right way”. Because I don’t see a “right way”. I see people trying to get a job done, and reach a wide audience. I see people trying to keep things simple for their developers, and taking the path of least resistance. I see a whole lot of web literacy education that needs to occur across the tech sectors and in school. I just don’t see any perfect answer to the API versioning debate. I see a whole lot of interesting and useful patterns, and I see people doing the best they can with what they have. Reflecting why APIs work so well, because they are scrappy, often times simple, and allow people to get business done on the web using low cost, easy to understand approaches to making resources available.
I’m spending time learning from the API design guides of other leading API providers, absorbing their healthy practices, and assimilating them into my own consulting and storytelling. One API design guide I am learning a lot from is out of the Adidas Group, which contains a wealth of wisdom regarding not just the design of your API, but also the deployment and evolution of the API resources we are publishing.
One particularly interesting piece of advice I found within Adidas API design guidance were their rules for extending an API, which I think is pretty healthy advice for an API developer to think about.
Any modification to an existing API MUST avoid breaking changes and MUST maintain backward compatibility.
In particular, any change to an API MUST follow the following Rules for Extending:
- You MUST NOT take anything away (related: Minimal Surface Principle , Robustness Principle)
- You MUST NOT change processing rules
- You MUST NOT make optional things required
- Anything you add MUST be optional (related Robustness Principle)
NOTE: These rules cover also renaming and change to identifiers (URIs). Names and identifiers should be stable over the time including their semantics.
First of all, I think many API developers aren’t even thinking about what constitutes a breaking change most of the time. So having any guidance that makes them pause and think about this topic is a good thing. Second, I think we should be sharing more stories about when things break, helping folks think more about these elements–the problem is that many folks are embarrassed they introduced a breaking changes, and would rather not talk about it, let alone make it publicly known.
I am working my way through the API Stylebook, learning from all the API design guides it has aggregated. There is a wealth of knowledge in there to learn from, and contains topics that make for great stories here on the blog. I wish more API providers would actively publish their API governance strategy, so that we can keep aggregating, and learning from each other. Making the wider API space more consistent, and hopefully more reliable along the way.
I fell down the rabbit hole of the latest Facebook version release, trying to understand the deprecation of their User Insights API. The story of the deprecation of the API isn’t told accurately as part of the the regular release process, so I found myself thinking more deeply about how we tell stories (or don’t) around each step forward of our APIs. I have dedicated areas of my API research for the road map, issues, and change log for API operations, because their presence tell a lot about the character of an API, and their usage I feel paints and accurate painting of each moment in time for an API.
Facebook has a dedicated change log for their API platform, as well as an active status and issues pages, but they do not share much about what their road map looks like. They provide a handful of elements with each releases change log:
- New Features — New products or services, including new nodes, edges, and fields.
- Changes — Changes to existing products or services (not including Deprecations).
- Deprecations — Existing products or services that are being removed.
- 90-Day Breaking Changes — Changes and deprecations that will take effect 90 days after the version release date.
The presence, or lack of presence, of a road map, change log, status and issue pages for an API paints a particular picture of a platform in my mind. Also, the stories they tell, or do not tell with each release paint an evolving picture of where a platform is headed, and whether or not we want to participating in the journey. Facebook does better than most platforms I track on when it comes to storytelling, by also releasing a blog post telling the story of each release, providing separate posts for the Graph API, as well as the Marketing API. It is too bad that they omitted the deprecation of the Audience Insight API, which occurred at the time of this story.
While I consider the presence of building blocks like a change log, road map, issues and status page a positive sign for platforms. It still always requires reading between the lines, and staying in tune with each release to really get a feel for how well a platform puts these building blocks to work for the platform. Regardless, I think these building blocks do adequately paint a picture of the current state of a platform, it just usually happens to be the picture that platform wants you to see, not necessary the picture the platform consumers would like to see.
APIs are not forever, and eventually will go away. The trick with API deprecation is to communicate clearly, and regularly with API consumers, making sure they are prepared for the future. I’ve been tracking on the healthy, and not so healthy practices when it comes to API deprecation for some time now, but felt like Google had some more examples I wanted to add to our toolbox. Their approach to setting expectations around API deprecation is worthy of emulating, and making common practice across industries.
The Google Adwords API team is changing their release schedule, which in turns impacts the number of APIs they’ll support, and how quickly they will be deprecating their APIs. They will be releasing new versions of the API three times a year, in February, June and September. They will also be only supporting two releases concurrently at all times, and three releases for a brief period of four weeks, pushing the pace of API deprecation alongside each release. I think that Google’s approach provides a nice blueprint that other API provides might consider adopting.
Adopting an API release and sunset schedule helps communicate the changes on the horizon, but it also provides a regular rhythm that API consumers can learn to depend on. You just know that there will be three releases a year, and you have a quantified amount of time to invest in evolving integration before any API is deprecated. It’s not just the communication around the roadmap, it is about establishing the schedule, and establishing an API release and sunset cadence that API consumers can be in sync with. Something that can go a lot further than just publishing a road map, and tweeting things out.
I’ll add this example to my API deprecation research. Unfortunately the topic is one that is widely communicated around in the API space, but Google has long a strong player when it comes to finding healthy API deprecation examples to follow. I’m hoping to get to the point soon where I can publish a simple guide to API deprecation. Something API providers can follow when they are defining and deploying their APIs, and establish a regular API release and deprecation approach that API developers can depend on. It can be easy to get excited about launching a new API, and forget all about it’s release and deprecation cycles, so a little guidance goes a long way to helping API providers think about the bigger picture.
We all (well most of us) strive to deliver as stable of an API presence as we possibly can. It is something that is easier said than done. It is something that takes caring, as well as the right resources, experience, team, management, and budget to do APIs just right. It is something the API idols our there make look easy, when they really have invested a lot of time and energy into developing a agile, yet scalable approach to ensuring APIs stay up and running. Something that you might able to achieve with a single API, but can easily be lost between each API version, as we steer the ship forward.
I spend a lot of time at the developer portals of these leading API providers looking for interesting insight into how they are operating, and I though Stripe’s vision around versioning their API is worth highlighting. Specifically their quote about treating your API like they are real life physical infrastructure.
“Like a connected power grid or water supply, after hooking it up, an API should run without interruption for as long as possible.Our mission at Stripe is to provide the economic infrastructure for the internet. Just like a power company shouldn’t change its voltage every two years, we believe that our users should be able to trust that a web API will be as stable as possible.”
This is possible. This is how I view Amazon S3, and Pinboard. These are two APIs I depend on to make my business work. Storage and bookmarking are two essential resources in my world, and both these APIs have consistently delivered stable API infrastructure, that I know I can depend on. I think it is also interesting to note that one is a tech giant, while the other is a viable small business (not startup). Demonstrating for me that there isn’t a single path to being a reliable, stable, API provider, despite what some folks might believe.
I am spending a lot of time lately thinking of API infrastructure along the lines of our energy grid, and transit system. The analogies are not perfect, but I do feel like as time moves on, some of our API infrastructure will continue to become commoditized, deemed essential, and something we depend on just as much as power, gas, and other utilities. These are the APIs that will be sticking around, the ones that can prove their usefulness, and deliver reliable integrations that do not change with each funding season, or technology trend. I’m looking forward to getting beyond the wild west days of APIs, and moving into the stage where APIs are treated like they are infrastructure, not just some toy, or latest fad, and we can truly depend on them and build our businesses around.
If you think there is a link I should have listed here feel free to tweet it at me, or submit as a Github issue. Even though I do this full time, I'm still a one person show, and I miss quite a bit, and depend on my network to help me know what is going on.