CTOs, developers: how to assess quality of an external API?

Reading time ~4 minutes

Nowadays finding an external API in order to improve your service is getting easier and easier. More and more companies offer APIs. Problem is many developers/CTOs start the API integration right away while it should be the very last step! Before that you need to figure out whether the quality of this API matches some minimum requirements. Let me tell you how I do it. I hope it will help other CTOs and developers.

Quality of data

A lot of APIs expose data in order for you to enrich your system (this is not always the case of course, Stripe is not an enrichment API for example). This is essential that you check the quality of those data. It will take you a long time and I know you do not like testing! Neither do I but you cannot avoid building a serious test scenario here. If you realize data quality was not good enough only 2 weeks after finishing your API integration, trust me you’ll regret it…

Documentation

I recently fell upon an API which exposed great data (much better than his competitors in my opinion), but its documentation was… awful! Actually it almost did not exist. In addition to that it did not always respect the basic REST standards. How can you possibly integrate an external API if error codes are not properly documented ? Well the only solution is for you to test again and again in order to understand how things work behind the hoods. Reverse-engineering might be fun but it takes a lot of time. Remember you have no Github repo to explore here since source code is not available… Bad documentation is a lot of time lost for the devs and certainly bad surprises in the mid term.

Libraries

Can you consume the API with special libraries in your favorite language ? As a Python and Go developer I’m always glad to see APIs offering a Python lib (I know I can forget about Go for the moment). It can save you quite a lot of time, but first make sure the lib is mature enough and covers all the API features (not always the case).

Reputation of the vendor

Reputation can help you find out whether you’ll have bad surprises with your API in the future. By bad surprises I mean service interruption, features regression, or even end of the service… You can partly tackle that by asking yourself the following questions:

  • is this API popular on the internet (in general if you find little information, run away)? Are there a lot of articles/tutorials talking about it? Are those articles positive?
  • are some popular companies using it?
  • if the company developed libs, are they popular on Github? Are the issues on Github solved regularly?
  • were there recent updates of the API or was the last update released a long time ago?

Technical support

Make sure someone answers you quickly by email when you have an issue and the answer is relevant. If you’re based in Europe and the API is run by an American company, check whether time difference is not too much of a problem.

Respect standards

In my humble opinion, you choose only RESTful APIs today. If the API you’re in love with do not respect the REST standard, be suspicious. But keep in mind that it’s not perfectly clear what the REST standard is about, and each API implements its own rules (HTTP codes, POST requests encoding, …). Still, have a close look at the docs, and check that you do not see something original. Originality will slow you down…

Price

Of course price is very important. But be careful, API prices are not always easy to understand. Are you going to be charged per month for an unlimited amount of requests ? Charged per request ? If so are you going to be charged twice for 2 identical requests (in case of an enrichment API) or will the second request be free ? Are you going to be charged for a request returning no result (HTTP 404) ? Make sure you understand all the implications of pricing.

Quality of Service (QoS)

QoS is highly important. Basically you want the API to go fast and have as little downtime as possible. Unfortunately this is not an easy to test point. Indeed QoS may vary a lot over time, and many APIs offer 2 levels of QoS depending on whether you’re using the free version of the API or you paid for it… Sometimes you can also choose between different subscriptions with different levels of response time.

Parallel queries support

Depending on how you’re planning to integrate your API, you might want to speed things up by making multiple parallel queries to the API instead of using it sequentially. Personally I’m using Golang for that most of the time. If so be careful: many vendors do not support parallel queries, and when they do they always set up a limit. In that case make sure to ask them what this limit is (not always told in the docs) and adapt your script based on this.

This post will be a good memo for me. I hope for you too!

Existe aussi en français | También existe en Español

API Rate Limiting With Traefik, Docker, Go, and Caching

Limiting API usage based on advanced rate limiting rule is not so easy. In order to achieve this behind the NLP Cloud API, we're using a combination of Docker, Traefik (as a reverse proxy) and local caching within a Go script. When done correctly, you can considerably improve the performance of your rate limiting and properly throttle API requests without sacrificing speed of the requests. Continue reading