Where Do EU Citizens Emigrate Outside the EU? The Top 25 Destinations

Infographic: the 25 top emigration destinations outside the EU for people born in the EU in 2024, with their main EU countries of birth and the share aged 65 or over

When EU citizens emigrate, a little over half of them settle in another EU country, which free movement makes easy. Yet millions do leave the EU altogether, and I wanted to know where they go. The answer is less obvious than it seems: the top 25 emigration destinations outside the EU include Canada and Australia, as you would expect, but also Türkiye, Serbia, Brazil and Russia, and the people behind each figure turn out to be very different.

What the numbers measure

Every figure in the main ranking is the total number of EU-born emigrants living in a given country in mid-2024, that is, people who were born in one of the 27 EU countries and now reside in that destination. It is not a yearly flow: it adds up everyone who emigrated there, whether they moved last year or fifty years ago. That distinction matters, because it explains why some destinations rank high for reasons that have more to do with history than with relocation today.

The UK and the US lead by far

The United Kingdom is the first emigration destination outside the EU, with 3.41 million EU-born residents, and the United States comes second with 2.89 million. Together they account for almost half of the 13 million EU-born emigrants counted across the top 25. Switzerland (1.44 million) and Canada (1.12 million) are the only other countries above one million, followed by Türkiye (956,000) and Australia (835,000).

Beyond the top six, the figures drop quickly. Norway hosts 381,000 EU-born emigrants, Israel 340,000 and Serbia 288,000, and the last places on the list, Japan, Uruguay and Andorra, each host a little over 40,000.

Who goes where

The country of birth changes completely from one destination to the next. Poles and Romanians make up almost half of the EU-born emigrants in the UK (26% and 21%), which likely reflects the large movement that followed their countries’ accession to the EU. Germans are the largest group in the United States (39%) and in Switzerland (26%), while Italians remain the first group in Canada (16%) and Australia (18%).

Shared languages and old historical ties likely play a role too. Portuguese-born emigrants make up 55% of the EU-born population in Brazil, Spanish-born emigrants make up 59% in Uruguay and 42% in Argentina, and people born in France account for 80% in Morocco and 99% in New Caledonia, which is a French territory outside the EU. Thailand, for its part, splits almost evenly between Germans and French.

Retirees in some places, younger arrivals in others

Age is where the contrast is sharpest. In Canada and Australia, 39% of the European-born residents are aged 65 or over, and the share reaches 37% in Israel, which likely reflects waves of emigration that took place decades ago. In the United Kingdom, by contrast, only 10% are 65 or over, and the share falls to 8% in Japan, 7% in Norway and 4% in Iceland, likely because most people arrived more recently, many of them to work.

These age shares have an important limit, which I explain in the methodology below: they cover OECD countries only, they date from 2015/16, and they count everyone born in Europe rather than only in the EU.

When history explains the ranking

A few entries reflect history more than recent emigration, and I chose to keep them in the ranking while explaining them rather than dropping them:

  • Serbia: 77% of its EU-born residents were born in Croatia before Yugoslavia broke up, which likely explains most of this figure.
  • Türkiye: the EU-born population mixes people born in Bulgaria (36%), likely including many ethnic Turks who left in past decades, and people born in Germany (38%), likely including many children of Turkish migrants who returned home.
  • Russia and Ukraine: most of their EU-born residents were born in Germany or the Baltic states (Lithuania and Latvia make up 73% in Ukraine), which likely reflects moves within the Soviet bloc.
  • Israel: the largest groups were born in Romania (34%) and Poland (18%), and they likely include many older Jewish immigrants.
  • Latin America: the Spanish, Portuguese and Italian communities in Brazil, Venezuela, Argentina, Mexico, Chile and Uruguay likely reflect the great waves of 20th-century emigration.

How I built it

  • Number of EU-born emigrants: UN DESA international migrant estimates, 2024 edition, table by destination and origin, mid-2024. I added up the 27 EU countries of origin for each destination. Most countries count people by country of birth, but Thailand, Morocco, Japan and Andorra count them by citizenship.
  • Main countries of birth: the same UN DESA table, showing the two EU countries that make up the largest shares of each destination’s EU-born population.
  • Share aged 65 or over: the OECD Database on Immigrants in OECD Countries (DIOC) 2015/16, based on national censuses and registers. Its public files only break people down by region of birth, so I used everyone born in Europe as an approximation of the EU-born population.
  • Scope: I kept every country outside the EU27, including the EFTA countries Switzerland, Norway and Iceland. I left out Réunion, Guadeloupe and Martinique, because these French overseas departments are legally part of the EU.

Limits

The ranking counts all EU-born emigrants, whatever their age, so it includes retirees, children and people who emigrated decades ago. The age data is about ten years older than the main figures, covers only 13 of the 25 destinations, and includes people born in non-EU European countries such as the UK, Russia or Ukraine, so it should be read as an indication rather than an exact profile. I also looked for comparable data on self-employment and household type by EU country of birth, but I did not find any source that covers these destinations consistently, so I left them out rather than guess.

The full per-country data, with the sources and definitions for each destination, is available on request: email explore@juliensalinas.com.

If you spot an error, or know a better source for one of these countries, tell me and I will update the chart.

Where Should Tech Workers Relocate? 64 Hubs, Salary vs. Cost of Living

Infographic: monthly disposable income of a mid-level software engineer in 64 tech hubs, after income tax, social contributions and one-bedroom city-centre rent

A software engineer on a $200,000 salary in Silicon Valley keeps less each month than one on $160,000 in Austin. Once you subtract income tax and rent, $7,912 a month in the Valley versus $8,128 in Austin. That is why I stopped comparing salaries and started comparing what is left at the end of the month, for 64 tech hubs.

Why salary alone is misleading

Two things sit between a gross salary and your bank account: what the state takes (income tax and social contributions) and what your landlord takes. Both vary enormously from one hub to the next. In the data below, the share of gross pay going to tax and social contributions ranges from 0% (Dubai, Riyadh) to 41.5% (Bucharest). The rent for a one-bedroom apartment in the city centre ranges from $230 a month in Hyderabad to $4,411 in New York.

This is the second half of a pair. In my previous post I asked “can you actually get a work visa there?”. This one asks “is it worth going?”.

How I measured it

I used one persona everywhere: a mid-level software engineer, single, renting a one-bedroom apartment near the city centre.

  • Gross salary: the median base salary for Software Engineers on Levels.fyi, accessed on 16 October 2026.
  • Net salary: gross salary minus income tax and employee social contributions for a single person, computed with each country’s official 2026 rules.
  • Rent: the average one-bedroom city-centre rent on Numbeo.
  • Currency: everything converted to USD at the 16 October 2026 spot rate.
  • Result: monthly disposable income = net salary / 12 minus rent.

I left out groceries, transport and everything else. Rent is the one cost I could compare consistently across all 64 hubs. This is not a full cost-of-living index.

What I found

Seattle, Austin, Zurich and Silicon Valley lead.

  • Seattle: $8,529 a month
  • Austin: $8,128
  • Zurich: $8,107
  • Silicon Valley: $7,912

Twelve hubs leave more than $5,000 a month. Ten are in the US, plus Zurich and Tel Aviv ($5,641). All 11 US hubs sit in the top 13.

Silicon Valley has the highest salary, but is only fourth. The median base pay is $200,000, the highest of the 64 hubs. But 34% of it goes to tax and social contributions, against 26.8% in Seattle and 24.6% in Austin. Rent in San Jose is $3,086, against $2,454 in Seattle and $1,924 in Austin. Silicon Valley’s net monthly salary ($10,998) is almost the same as Seattle’s ($10,983). The rent difference is what pushes it down.

New York is a rent story. A $170,000 salary leaves $9,312 a month after tax, but rent ($4,411) is the highest on the chart. You keep $4,902, 13th.

Dubai vs. London. Dubai charges no income tax, so an $81,699 salary is all net. After rent you keep $4,401 a month. London pays $121,799 gross, but you keep $4,202. Close enough that I would call it a tie, and Dubai’s figure is graded “partial” (see limitations).

Paris vs. London. Paris leaves $2,336 a month, London $4,202, about 1.8 times more. The difference comes from pay, not rent: the London median is $121,799 gross against $68,256 in Paris, while London rent is $2,851 against $1,556.

China vs. Berlin. Beijing ($3,792), Hangzhou ($3,663) and Shanghai ($3,587) all beat Berlin ($3,544). Shenzhen ($3,494) is just below. Berlin pays more gross ($101,005 against $80,787 in Beijing), but 40.3% goes to tax and social contributions and rent is higher ($1,481 against $937 in Beijing).

Singapore, with a note. Singapore’s gross salary is $78,747, but you keep only $2,109 a month, roughly level with Madrid ($2,087). Rent is $2,924, and I deducted the 20% employee CPF contribution, which applies to citizens and permanent residents. An expat on an Employment Pass pays no CPF, so for them the figure would be higher.

The earlier “sweet spot” hubs. Amsterdam ($3,175), Seoul ($3,162), Paris ($2,336) and Tokyo ($1,909) offered the best jobs-versus-visa balance in my previous post. Here they land between $1.9k and $3.2k.

The bottom of the list. Ho Chi Minh City leaves $353 a month, Jakarta $571 and Mumbai $736. Seattle leaves about 24 times more than Ho Chi Minh City, and about 2.8 times the median hub (about $3.1k). One caution: this is in dollars, so it shows how far a salary goes if you spend in dollars, not how well someone lives locally.

Jobs vs. leftover money. Only six hubs combine a tech jobs score of 7+ with more than $5,000 a month: Seattle, Silicon Valley, Austin, Los Angeles, Boston and Tel Aviv. Apart from Zurich (visa difficulty 6) and Tel Aviv (4), every hub above $5,000 is a US hub with a visa difficulty score of 8. The money is where the visa is hardest.

Chart: the 10 best and 10 worst-placed tech hubs by monthly disposable income, excluding rough estimates
Scatter plot: tech jobs score vs. monthly disposable income for 64 tech hubs

Limitations

  • Levels.fyi medians cover all levels. They are self-reported and lean toward big tech and senior roles. I could not isolate “mid-level”. In emerging markets they probably overstate what a typical local engineer earns.
  • Health premiums and occupational pensions are excluded everywhere. I included mandatory government social contributions and statutory provident funds, but not private health insurance (mandatory in the Netherlands and Switzerland, employer-provided in the US) or employer and occupational pensions. This probably flatters Zurich and the US most.
  • Data quality varies. Of the 64 hubs, 25 are solid, 29 partial and 10 rough. “Solid” means a large Levels.fyi sample, enough Numbeo contributors and a complete tax model. “Partial” means one of those is weaker, such as a country-level salary used for a city or a simplified tax model. The 10 rough estimates are Miami, Copenhagen, Riyadh, Vilnius, Santiago, Bogotá, Cape Town, Bangkok, Nairobi and Lagos. Treat their rank as indicative only. Lagos comes out negative (-$439 a month) on a very small salary sample.
  • Miami uses a different source. Levels.fyi has no Miami page, so I used the US Bureau of Labor Statistics median for software developers in the Miami metro area ($132,650). It covers all employers and is not directly comparable.
  • Spot rates. I converted at one day’s exchange rate, so a currency move of a few percent can reorder close hubs. Hangzhou and Shenzhen use the China-wide Levels.fyi median.
  • Rent is crowd-sourced. Numbeo’s number of contributors differs by city.

The ranking is therefore reliable for large gaps (Seattle vs. Paris, or London vs. Mumbai) and not for small ones (Dubai vs. London, Berlin vs. Shenzhen).

Sources

  • Salaries: Levels.fyi, Software Engineer base salary, median, accessed 16 October 2026. Miami: US Bureau of Labor Statistics, Occupational Employment and Wage Statistics, May 2025, software developers, Miami metro area.
  • Rent: Numbeo, one-bedroom apartment in city centre, data updated mid-October 2026.
  • Exchange rates: ExchangeRate-API (open.er-api.com), 16 October 2026.
  • Taxes and social contributions (2026 rules): official authorities, for example the IRS and SSA (US), CRA (Canada), GOV.UK (UK), §32a EStG (Germany), ESTV and the canton of Zurich (Switzerland), Belastingdienst (Netherlands), IRAS and the CPF Board (Singapore), and the national authority of each other country.

The full per-country data, with the exact sources, assumptions and tax parameters for each hub, is available on request: email explore@juliensalinas.com.

If you spot an error, or a rule that changed, tell me and I will update the chart.


Julien Salinas · @juliensalinas · explore@juliensalinas.com · linkedin.com/in/julien-salinas-673b3811

Where Should Tech Workers Relocate? 64 Hubs, Jobs vs. Visa Difficulty

Infographic: 64 tech hubs plotted by tech job opportunity vs. visa difficulty for skilled tech workers

More tech workers are open to moving abroad for work than a few years ago. But “where is the job market good?” and “where can I actually get a visa?” are two different questions. I put both on one chart for 64 tech hubs.

Why this matters

A few data points I checked:

  • In the 2025 CERC/Ipsos Global Mobility Survey of 10,574 workers across 19 countries, “almost half (49%) of employees say they would consider relocating internationally for work; a notable increase compared to 2017, when 46% would consider the same” (Ipsos). The sponsor report frames this as relocation “for up to two years” and lists information technology (58%) as the second most open sector, behind utilities (59%) (CERC/WFPMA sponsor report).
  • The most famous tech visa is also a lottery. For the US H-1B cap in fiscal year 2026, USCIS counted about 339,000 eligible unique beneficiaries and selected 118,660 of them in the initial selection. (USCIS)
  • Rules keep moving. In 2026 alone, Germany raised its EU Blue Card salary thresholds (Make it in Germany), Sweden moved to a 90%-of-median salary requirement (Migrationsverket), Malaysia roughly doubled its Employment Pass salary minimums (ESD Malaysia), and Singapore announced higher Employment Pass salaries from 2027 (MOM). In the US, the $100,000 H-1B payment introduced in 2025 was blocked by the courts in 2026 while the government appeals (USCIS).

So I wanted a single view: where are the jobs, and where is the door actually open?

The findings

The sweet spot (many jobs, easier visa): Amsterdam, Paris, Tokyo, Seoul. All four combine strong tech ecosystems with a clear, employer-sponsored route.

  • Amsterdam: the Netherlands’ highly skilled migrant permit, through an IND-recognised employer.
  • Paris: France’s “Talent” residence card.
  • Tokyo: Japan’s engineer visa and its points-based highly skilled professional track.
  • Seoul: Korea’s E-7 professional visa.

None of them uses a lottery.

Close behind: Singapore, Tel Aviv, Dublin, Hong Kong.

  • Singapore and Tel Aviv score higher on jobs than any city in the top group, but their visas are a bit harder. Singapore has a salary bar plus a points test. Israel has a high minimum salary for foreign experts.
  • Dublin and Hong Kong are among the easiest visas on the chart for a solid job market. Ireland has its Critical Skills permit. Hong Kong’s Top Talent Pass doesn’t even require a job offer.
  • Berlin, Toronto-Waterloo and Bengaluru sit just behind.

The US: the most jobs, the hardest visa. Silicon Valley, New York, Boston and Seattle top the jobs axis. They also sit at the bottom of the visa axis. The H-1B is capped and allocated by lottery, now weighted by wage level. On top of that, policy has been unstable: a $100,000 fee was announced, then blocked by courts, and a separate fee has been proposed.

Other patterns:

  • London has a deep job market. But the UK Skilled Worker route needs a licensed sponsor and a £41,700 minimum salary, which puts it just outside the sweet spot.
  • Zurich and China’s hubs (Beijing, Shanghai, Shenzhen, Hangzhou) are hard. Switzerland has a fixed annual quota for non-EU workers. China uses a points-based permit system with rising salary thresholds.
  • Many smaller hubs are easy to enter but have fewer jobs. Examples are Tallinn, Vilnius, Helsinki, Dubai, Kuala Lumpur and Auckland. They’re worth a look if you already have an offer or work remotely.

Methodology

Tech jobs score (1-10). This is based mainly on the Startup Genome Global Startup Ecosystem Report 2025 ranking. I adjusted it with my judgment of each hub’s size and pay level. Hubs outside that ranking were scored on general market size only.

Visa difficulty score (1-10). Each hub starts at 1 and adds points:

  • +3 for a capped lottery (US H-1B)
  • +2 for a hard national quota, or +1 for a soft or very large quota or an overall cap on temporary workers
  • +1 if employer sponsorship is required
  • +1 for a labour-market test or local-hire/understudy requirement
  • +1 for a salary or points threshold that is high relative to local tech pay
  • +1 if typical processing takes over 3 months (+2 if over 6 months or tied to an annual cycle)
  • +1 each for: a language barrier in most tech jobs, recent tightening, policy instability or high fees or security concerns, heavy bureaucracy

Scores max out at 10. In practice they range from 2 to 8. The chart’s quadrant line sits at 4.5 on visa difficulty and 5.5 on jobs.

Verification. On 13 October 2026, I checked the main skilled-worker route in each of the 43 countries behind the 64 hubs against official immigration pages.

  • 37 countries: I opened and read official pages, fully for 28 and partially for 9.
  • Vietnam: checked against the full text of the relevant decree on a legal database.
  • 5 countries: the official sites could not be fetched (blocked, bot checks or dead links). These are India, Israel, Australia, Finland and Colombia, and their scores rely on secondary sources.

A few inputs are also flagged as secondary-only. Examples include Korea’s E-7 salary floor, Japan’s processing times and Poland’s Blue Card salary. The full per-country data (exact URLs read and inputs that remain unverified) is available on request: DM me or email explore@juliensalinas.com.

Assumptions:

  • The person is a skilled tech worker from outside the EU, with a job offer.
  • Nationality matters a lot. EU citizens move freely within the EU, and bilateral deals exist, so your personal result may differ.
  • Points were nudged slightly apart on the chart when scores tie.

Caveats

  • These are editorial scores, not measurements. The rubric makes my judgment explicit and consistent, but it is still judgment.
  • Rules change fast. Several changed during 2026, and the US fee situation could flip again after appeal. Check the official page before you act.
  • A visa route existing doesn’t mean you’ll get it. Approval depends on your profile, your employer and the specific role.
  • The jobs score leans on startup ecosystems. It under-weights places with large corporate or outsourcing tech employment.
  • Cost of living, salaries after tax, language and quality of life aren’t on this chart. They matter as much as the two axes.

If you spot an error or a rule that changed, tell me and I’ll update the chart.


Julien Salinas · @juliensalinas · explore@juliensalinas.com

API Rate Limiting With Traefik, Docker, Go, and Caching

Limiting API usage based on advanced rate limiting rule is not so easy. In order to achieve this behind the NLP Cloud API, we’re using a combination of Traefik (as a reverse proxy) and local caching within a Go script. When done correctly, you can considerably improve the performance of your rate limiting and properly throttle API requests without sacrificing speed of the requests.

In this example we’re showing how to delegate the rate limiting of every API request to a dedicated microservice thanks to Traefik and Docker. Then in this dedicated microservice, we will count the number of requests recently made in order to authorize or not the new request.

Traefik As A Reverse Proxy

In order to set up an API gateway, Traefik and Docker are a very good combination.

Traefik

The idea is that all your API requests should be first routed to a Docker container containing a Traefik instance. This Traefik instance acts as a reverse proxy so it will do things like authentication, filtering, retrying, … and eventually routing the user request to the right container.

For example, if you are making a text summarization request on NLP Cloud, you will first go through the API gateway that will take care of authenticating your request and, if successfully authenticated, your request will be routed to a text summarization machine learning model contained in a dedicated Docker container hosted on a specific server.

Both Traefik and Docker are easy to use, and they make your program quite easy to maintain.

Why Use Go?

A rate limiting script will necessarily have to handle a huge volume of concurrent requests.

Go is a good candidate for this type of application as it processes your requests very quickly, and without consuming too much CPU and RAM.

Traefik and Docker were both written in Go, which must not be a coincidence…

A naive implementation would be to use the database to store API usage, count past user requests, and rate limit requests based on that. It will quickly raise performance issues as making a DB request every single time you want to check a request will overwhelm the DB and create tons of unnecessary network accesses. The best solution is to manage that locally in memory. The flip side, of course, it that in-memory counters are not persistent: if you restart your rate limiting application, you will lose all your ongoing counters. It should not be a big deal in theory for a rate limiting application.

Delegating API Rate Limiting To a Dedicated Microservice Thanks To Traefik And Docker

Traefik has many interesting features. One of them is the ability to forward authentication to a dedicated service.

Traefik Auth Forwarding

Basically, each incoming API request will first be forwarded to a dedicated service. If this service returns a code 2XX code, then the request is routed to the proper service, otherwise it is rejected.

In the following example, we will use a Docker Compose file for a Docker Swarm cluster. If you’re using another container orchestrator like Kubernetes, Traefik will work very well too.

First, create a Docker Compose file for your API endpoint and enable Traefik:

version: "3.8"

services:
  traefik:
    image: "traefik"
    command:
      - --providers.docker.swarmmode
  api_endpoint:
    image: path_to_api_endpoint_image
    deploy:
      labels:
        - traefik.http.routers.api_endpoint.entrypoints=http
        - traefik.http.services.api_endpoint.loadbalancer.server.port=80
        - traefik.http.routers.api_endpoint.rule=Host(`example.com`) && PathPrefix(`/api-endpoint`)

Then add a new service dedicated to rate limiting and ask Traefik to forward all requests to it (we will code this Go rate limiting service a bit later):

version: "3.8"

services:
  traefik:
    image: traefik
    command:
      - --providers.docker.swarmmode
  api_endpoint:
    image: path_to_your_api_endpoint_image
    deploy:
      labels:
        - traefik.http.routers.api_endpoint.entrypoints=http
        - traefik.http.services.api_endpoint.loadbalancer.server.port=80
        - traefik.http.routers.api_endpoint.rule=Host(`example.com`) && PathPrefix(`/api-endpoint`)
        - traefik.http.middlewares.forward_auth_api_endpoint.forwardauth.address=http://rate_limiting:8080
        - traefik.http.routers.api_endpoint.middlewares=forward_auth_api_endpoint
  rate_limiting:
    image: path_to_your_rate_limiting_image
    deploy:
      labels:
        - traefik.http.routers.rate_limiting.entrypoints=http
        - traefik.http.services.rate_limiting.loadbalancer.server.port=8080

We now have a full Docker Swarm + Traefik configuration that first forwards requests to a rate limiting service before eventually routing the request to the final API endpoint. You can put the above in a production.yml file and start the application with the following command:

docker stack deploy --with-registry-auth -c production.yml application_name

Note that only the headers of the requests are forwarded, not the content of the requests. This is for performance reasons. So if you want to authenticate a request based on the body of this request, you will need to come up with another strategy.

Handling Rate Limiting With Go And Caching

The Traefik and Docker configurations are ready. We now need to code the Go microservice that will take care of rate limiting the requests: users only have the right to 10 requests per minute. Above 10 requests per minute, every request will be rejected with a 429 HTTP code.

package main

import (
  "fmt"
  "time"
  "log"
  "net/http"

  "github.com/gorilla/mux"
  "github.com/patrickmn/go-cache"
)

var c *cache.Cache

// updateUsage increments the API calls in local cache.
func updateUsage(token) {
  // We first try to increment the counter for this user.
  // If there is no existing counter, an error is returned, and in that
  // case we create a new counter with a 3 minute expiry (we don't want
  // old counters to stay in memory forever).
  _, err := c.IncrementInt(fmt.Sprintf("%v/%v", token, time.Now().Minute()), 1)
  if err != nil {
  c.Set(fmt.Sprintf("%v/%v", token, time.Now().Minute()), 1, 3*time.Minute)
  }
}

func RateLimitingHandler(w http.ResponseWriter, r *http.Request) {
  // Retrieve user API token from request headers.
  // Not implemented here for the sake of simplicity.
  apiToken := retrieveAPIToken(r)
  
  var count int

  if x, found := c.Get(fmt.Sprintf("%v/%v", apiToken, time.Now().Minute())); found {
    count = x.(int)
  }

  if count >= 10 {
    w.WriteHeader(http.StatusTooManyRequests)
    return
  }

  updateUsage(apiToken)

  w.WriteHeader(http.StatusOK)
}

func main() {
 r := mux.NewRouter()
 r.HandleFunc("/", RateLimitingHandler)

 log.Println("API is ready and listening on 8080.")

 log.Fatal(http.ListenAndServe(":8080", r))
}

As you can see, we’re using the Gorilla toolkit in order to create a small API, listening on port 8080, that will receive the request forwarded by Traefik.

Once the request is received, we extract the user API token from the request (not implemented here for the sake of simplicity), and check the number of requests made by the user associated with this API token during the last minute.

The request counter is stored in memory thanks to the go-cache library. Go-cache is a minimalist caching library for Go that is very similar to Redis. It automatically handles important things like cache expiry. Storing the API counters in memory is crucial as it is the fastest solution, and we want this code to be as fast as possible in order not to slow down API requests too much.

If the user has made more than 10 requests during the current minute, the request is rejected with a 429 HTTP error code. Traefik will see that this 429 error is not a 2XX code, so it won’t allow the user request to reach the API endpoint, and it will propagate the 429 error to the user.

If the request is not rate limited, we automatically increment the counter for this user.

I recommend that you deploy this Go application within a simple “scratch” container (FROM scratch): it is the lightest way to deploy Go binaries with Docker.

Conclusion

As you can see, implementing a rate limiting gateway for your API is not that hard, thanks to Traefik, Docker and Go.

Of course, rate limiting based on a number of requests per minute is only a first step. You might want to do more advanced things like:

  • Rate limiting per minute, per hour, per day, and per month
  • Rate limiting per API endpoint
  • Have a variable rate limit per user depending on the plan he subscribed to
  • Check concurrency

So many interesting things we can’t mention in this article!

If you have questions please don’t hesitate to reach out to me.

Existe aussi en français | También existe en Español
API Analytics With Time-Series Thanks to TimescaleDB

Tracking API usage can be quite a technical challenge due to the high speed and volume of requests. Yet, having accurate analytics for your API is crucial, especially if you rely on that to invoice your customers. It is possible to be both fast and accurate with a time-series database called TimescaleDB. This is actually the solution we implemented behind NLP Cloud.

What Is API Analytics And Why Is It Hard?

API analytics is about retrieving various metrics related to the usage of your API.

For example, behind the NLP Cloud API we want to know the following things:

  • How many requests were made during the last minute, the last hour, the last day, the last month, and the last year
  • How many requests were made per API endpoint, and per user
  • How many words were generated by our text generation NLP models (like GPT-J)
  • How many characters were used by our multilingual NLP add-on

All these metrics are important in order to better understand how our API is used by our customers. Without such data, we’re unable to know which NLP models are the most used, who are our more important customers, etc.

But even more importantly, some of these metrics are used for invoicing! For example, customers who subscribed to a “pay-as-you-go” plan, are charged based on the number of words they generated with our API.

A very high volume of data is going through our API gateway, which is a challenge in terms of performance. It is very easy to either slow down the API, or lose some data.

So this is crucial that such an API analytics system is both fast and reliable.

TimescaleDB To The Rescue

TimescaleDB is a PostgreSQL database that has been optimized for time-series.

TimescaleDB

Basically, Timescale is optimized for a high volume of atomic writes. It is perfect for a use case where you write tons of data on a very regular basis, but almost never alter this data, and only read the data occasionally.

Timescale comes with interesting tools that make time-series easier. For example, they have so-called “continuous aggregates”. Such aggregates are a way to automatically “down-sample” your data on a regular basis. Down-sampling means that you remove old data after some time, and only keep some aggregates of this data (based on sums, counts, averages, etc.). It it crucial for 2 reasons:

  • Time-series can grow very quickly, so it is a very good way to save some disk space
  • Reading from a table stuffed with data can be painfully slow. It is much easier to read the data from an aggregated table that contains less data.

As opposed to other solutions like InfluxDB, TimescaleDB is a pure SQL solution, so the learning curve is quite low, and it will make the integration much easier. For example at NLP Cloud we’re interfacing with TimescaleDB in both Python and Go applications and we’re able to use our usual PostgreSQL libraries.

Installation

You can install TimescaleDB as a system package, but it’s simpler to install it as a Docker container.

First pull the Docker image:

docker pull timescale/timescaledb:latest-pg14

Then start your container and pass a password for your DB:

docker run -d --name timescaledb -p 5432:5432 -e POSTGRES_PASSWORD=password timescale/timescaledb:latest-pg14

Data Structure In TimescaleDB

In this example, we want to store API requests. We want each request to contain the following:

  • The time of the request
  • The id of the user who made the request
  • The API endpoint used during the request

The first time your launch TimescaleDB, you will need to create several things.

First init the TimescaleDB extension.

CREATE EXTENSION IF NOT EXISTS timescaledb;

Create the table that will store API requests, like we would do in any PostgreSQL DB:

CREATE TABLE IF NOT EXISTS api_calls (
  time TIMESTAMPTZ NOT NULL,
  user_id TEXT NOT NULL,
  endpoint TEXT NOT NULL
);

Then we create a so-called “hypertable” out of it:

SELECT create_hypertable('api_calls', 'time', if_not_exists => TRUE);

Hypertables are the heart of TimescaleDB. They automatically add many smart things in order to manage your data efficiently.

We now create a specific view out of your api_calls table called api_calls_per_hour. It is a view that will store aggregated data coming from api_calls. Every hour, the number of API requests in api_calls will be counted and put in api_calls_per_hour. The view will be much faster to query since it contains much less data than the initial api_calls table.

CREATE MATERIALIZED VIEW IF NOT EXISTS api_calls_per_hour
WITH (timescaledb.continuous) AS
SELECT time_bucket('1 hour', time) as bucket, user_id, endpoint,
COUNT(time)
FROM api_calls
GROUP BY bucket, user_id, endpoint;

Last of all, we create a continuous aggregate policy and a retention policy. Both will be managed by background workers. Most of the time everything works fine, but it you start having a lot of policies you might run out of background workers and you will see some error messages in your logs. In that case, the trick is to increase your number of background workers in /var/lib/postgresql/data/postgresql.conf.

The continuous aggregate policy will take care of regularly down-sampling the data from api_calls and putting it in api_calls_per_hour. The retention policy will take care of deleting old data from api_calls so you will never run out of disk space:

SELECT add_continuous_aggregate_policy('api_calls_per_hour',
  start_offset => INTERVAL '1 day',
  end_offset => INTERVAL '1 hour',
  schedule_interval => INTERVAL '1 minute',
  if_not_exists => TRUE);

SELECT add_retention_policy('api_calls', INTERVAL '90 days', if_not_exists => TRUE);

As you can see it was not too complex.

Inserting Data

In your application, you can now connect to your Timescale DB and insert requests.

For example, here is how you would do it in Python:

import psycopg2

conn = psycopg2.connect(
  "host=timescaledb dbname={} user={} password={}".format("name", "user", "password"))
cur = conn.cursor()
cur.execute("INSERT INTO api_calls (time, user_id, endpoint) VALUES (%s, %s, %s)",
  (datetime.now(), "1584586", "/v1/gpu/bart-large-cnn/summarization"))
conn.commit()
cur.close()
conn.close()

And now in Go:

import (
  "github.com/jackc/pgx/v4"
  "github.com/jackc/pgx/v4/pgxpool"
)

func main(){
timescaledbURL := fmt.Sprintf("postgres://%s:%s@timescaledb:5432/%s", "user", "password", "name")
timescaledbDatabase, err := pgxpool.Connect(context.Background(), timescaledbURL)
if err != nil {
  log.Fatalf("Cannot connect to TimescaleDB database: %v. Stopping here.", err)
}

query := `INSERT into api_calls (time, user_id, endpoint) VALUES ($1, $2, $3)`
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
  defer cancel()

  _, err := timescaledbDatabase.Exec(ctx, query, time.Now(), "1584586", "/v1/gpu/bart-large-cnn/summarization")
  if err != nil {
    log.Printf("Cannot insert metric in TimescaleDB: %v", err)
  }
}

Important point: you most likely don’t want to slow down user API requests because of potentially slow processing on the TimescaleDB side. The solution is to insert your data asynchronously, so the user API response returns even if the data is not inserted in your DB yet. But this is beyond the scope of this article.

In order to improve the throughput, you can also insert several API requests all at once. The idea is that you would first need to cache some requests in memory, and then save many of them in DB at once after some time.

Data Visualization

Many data visualization tools exist. I like Grafana because it is easy to plug it into TimescaleDB, and the chart capabilities are countless.

Here is a nice tutorial about how to set up TimescaleDB with Grafana: see it here.

Grafana

Conclusion

TimescaleDB is a powerful tool for time-series, and this is a great solution if you want to properly analyze your API usage.

As you can see, setting up and using TimescaleDB is quite easy. Careful though: TimescaleDB can quickly use a lot of RAM, so make sure to keep that in mind before provisioning your server instance.

If you have questions please don’t hesitate to ask!

Existe aussi en français | También existe en Español