Developing and deploying a whole website in Go (Golang)

Reading time ~9 minutes

In my opinion Go (Golang) is a great choice for web development:

  • it makes non-blocking requests and concurrency easy
  • it makes code testing and deployment easy as it does not require any special runtime environment or dependencies (making containerization pretty useless here)
  • it does not require any additional frontend HTTP server like Apache or Nginx as it already ships with a very good one in its standard library
  • it does not force you to use a web framework as everything needed for web development is ready to use in the std lib

A couple of years back the lack of libraries and tutorials around Go could have been a problem, but today it is not anymore. Let me show you the steps to build a website in Go and deploy it to your Linux server from A to Z.

The Basics

Let’s say you are developing a basic HTML page called love-mountains. As you might already know, the rendering of love-mountains is done in a function, and you should launch a web server with a route pointing to that function. This is good practice to use HTML templates in web development so let’s render the page through a basic template here. This is also good practice to load parameters like the path to templates directory from environment variables for better flexibility.

Here is your Go code:

package main

import (
    "html/template"
    "net/http"
)

// Get path to template directory from env variable
var templatesDir = os.Getenv("TEMPLATES_DIR")

// loveMountains renders the love-mountains page after passing some data to the HTML template
func loveMountains(w http.ResponseWriter, r *http.Request) {
    // Build path to template
    tmplPath := filepath.Join(templatesDir, "love-mountains.html")
    // Load template from disk
    tmpl := template.Must(template.ParseFiles(tmplPath))
    // Inject data into template
    data := "La Chartreuse"
    tmpl.Execute(w, data)
}

func main() {
    // Create route to love-mountains web page
    http.HandleFunc("/love-mountains", loveMountains)
    // Launch web server on port 80
    http.ListenAndServe(":80", nil)
}

Retrieving dynamic data in a template is easily achieved with {{.}} here. Here is your love-mountains.html template:

<h1>I Love Mountains<h1>
<p>The mountain I prefer is {{.}}</p>

HTTPS

Nowadays, implementing HTTPS on your website has become almost compulsory. How can you switch your Go website to HTTPS?

Linking TLS Certificates

Firstly, issue your certificate and private key in .pem format. You can issue them by yourself with openssl (but you will end up with a self-signed certificate that triggers a warning in the browser) or you can order your cert from a trusted third-party like Let’s Encrypt. Personally, I am using Let’s Encrypt and Certbot to issue certificates and renew them automatically on my servers. More info about how to use Certbot here.

Then you should tell Go where your cert and private keys are located. I am loading the paths to these files from environment variables.

We are now using the ListenAndServeTLS function instead of the mere ListenAndServe:

[...]

// Load TLS cert info from env variables
var tlsCertPath = os.Getenv("TLS_CERT_PATH")
var tlsKeyPath = os.Getenv("TLS_KEY_PATH")

[...]

func main() {
    [...]
    // Serve HTTPS on port 443
    http.ListenAndServeTLS(":443", tlsCertPath, tlsKeyPath, nil)
}

Forcing HTTPS Redirection

For the moment we have a website listening on both ports 443 and 80. It would be nice to automatically redirect users from port 80 to 443 with a 301 redirection. We need to spawn a new goroutine dedicated to redirecting from http:// to https:// (principle is similar as what you would do in a frontend server like Nginx). Here is how to do it:

[...]

// httpsRedirect redirects HTTP requests to HTTPS
func httpsRedirect(w http.ResponseWriter, r *http.Request) {
    http.Redirect(
        w, r,
        "https://"+r.Host+r.URL.String(),
        http.StatusMovedPermanently,
    )
}

func main() {
    [...]
    // Catch potential HTTP requests and redirect them to HTTPS
    go http.ListenAndServe(":80", http.HandlerFunc(httpsRedirect))
    // Serve HTTPS on port 443
    http.ListenAndServeTLS(":443", tlsCertPath, tlsKeyPath, nil)
}

Static Assets

Serving static assets (like images, videos, Javascript files, CSS files, …) stored on disk is fairly easy but disabling directory listing is a bit hacky.

Serving Files from Disk

In Go, the most secured way to serve files from disk is to use http.FileServer. For example, let’s say we are storing static files in a static folder on disk, and we want to serve them at https://my-website/static, here is how to do it:

[...]
http.Handle("/", http.FileServer(http.Dir("static")))
[...]

Preventing Directory Listing

By default, http.FileServer performs a full directory listing, meaning that https://my-website/static will display all your static assets. We don’t want that for security and intellectual property reasons.

Disabling directory listing requires the creation of a custom FileSystem. Let’s create a struct that implements the http.FileSystem interface. This struct should have an Open() method in order to satisfy the interface. This Open() method first checks if the path to the file or directory exists, and if so checks if it is a file or a directory. If the path is a directory then let’s return a file does not exist error which will be converted to a 404 HTTP error for the user in the end. This way the user cannot know if he reached an existing directory or not.

Once again, let’s retrieve the path to static assets directory from an environment variable.

[...]

// Get path to static assets directory from env variable
var staticAssetsDir = os.Getenv("STATIC_ASSETS_DIR")

// neuteredFileSystem is used to prevent directory listing of static assets
type neuteredFileSystem struct {
    fs http.FileSystem
}

func (nfs neuteredFileSystem) Open(path string) (http.File, error) {
    // Check if path exists
    f, err := nfs.fs.Open(path)
    if err != nil {
        return nil, err
    }

    // If path exists, check if is a file or a directory.
    // If is a directory, stop here with an error saying that file
    // does not exist. So user will get a 404 error code for a file or directory
    // that does not exist, and for directories that exist.
    s, err := f.Stat()
    if err != nil {
        return nil, err
    }
    if s.IsDir() {
        return nil, os.ErrNotExist
    }

    // If file exists and the path is not a directory, let's return the file
    return f, nil
}

func main() {
    [...]
    // Serve static files while preventing directory listing
    mux := http.NewServeMux()
    fs := http.FileServer(neuteredFileSystem{http.Dir(staticAssetsDir)})
    mux.Handle("/", fs)
    [...]
}

Full Example

Eventually, your whole website would look like the following:

package main

import (
    "html/template"
    "net/http"
    "os"
    "path/filepath"
)

var staticAssetsDir = os.Getenv("STATIC_ASSETS_DIR")
var templatesDir = os.Getenv("TEMPLATES_DIR")
var tlsCertPath = os.Getenv("TLS_CERT_PATH")
var tlsKeyPath = os.Getenv("TLS_KEY_PATH")

// neuteredFileSystem is used to prevent directory listing of static assets
type neuteredFileSystem struct {
    fs http.FileSystem
}

func (nfs neuteredFileSystem) Open(path string) (http.File, error) {
    // Check if path exists
    f, err := nfs.fs.Open(path)
    if err != nil {
        return nil, err
    }

    // If path exists, check if is a file or a directory.
    // If is a directory, stop here with an error saying that file
    // does not exist. So user will get a 404 error code for a file/directory
    // that does not exist, and for directories that exist.
    s, err := f.Stat()
    if err != nil {
        return nil, err
    }
    if s.IsDir() {
        return nil, os.ErrNotExist
    }

    // If file exists and the path is not a directory, let's return the file
    return f, nil
}

// loveMountains renders love-mountains page after passing some data to the HTML template
func loveMountains(w http.ResponseWriter, r *http.Request) {
    // Load template from disk
    tmpl := template.Must(template.ParseFiles("love-mountains.html"))
    // Inject data into template
    data := "Any dynamic data"
    tmpl.Execute(w, data)
}

// httpsRedirect redirects http requests to https
func httpsRedirect(w http.ResponseWriter, r *http.Request) {
    http.Redirect(
        w, r,
        "https://"+r.Host+r.URL.String(),
        http.StatusMovedPermanently,
    )
}

func main() {
    // http to https redirection
    go http.ListenAndServe(":80", http.HandlerFunc(httpsRedirect))

    // Serve static files while preventing directory listing
    mux := http.NewServeMux()
    fs := http.FileServer(neuteredFileSystem{http.Dir(staticAssetsDir)})
    mux.Handle("/", fs)

    // Serve one page site dynamic pages
    mux.HandleFunc("/love-mountains", loveMountains)

    // Launch TLS server
    log.Fatal(http.ListenAndServeTLS(":443", tlsCertPath, tlsKeyPath, mux))
}

Plus your love-mountains.html template:

<h1>I Love Mountains<h1>
<p>The mountain I prefer is {{.}}</p>

Testing, Deploying and Daemonizing with Systemd

Having a solid and easy test/deploy process is very important from an efficiency standpoint and Go really helps in this regard. Go is compiling everything within a single executable, including all dependencies (except templates but the latter are not real dependencies and this is better to keep them apart for flexibility reasons anyway). Go also ships with its own frontend HTTP server, so no need to install Nginx or Apache. Thus this is fairly easy to test your application locally and make sure it is equivalent to your production website on the server (not talking about data persistence here of course…). No need to add a container system like Docker to your build/deploy workflow then!

Testing

To test your application locally, compile your Go binary and launch it with the proper environment variables like this:

TEMPLATES_DIR=/local/path/to/templates/dir \
STATIC_ASSETS_DIR=/local/path/to/static/dir \
TLS_CERT_PATH=/local/path/to/cert.pem \
TLS_KEY_PATH=/local/path/to/privkey.pem \
./my_go_website

That’s it! Your website is now running in your browser at https://127.0.0.1.

Deploying

Deployment is just about copying your Go binary to the server (plus your templates, static assets, and certs, if needed). A simple tool like scp is perfect for that. You could also use rsync for more advanced needs.

Daemonizing your App with Systemd

You could launch your website on the server by just issuing the above command, but it is much better to launch your website as a service (daemon) so your Linux system automatically launches it on startup (in case of a server restart) and also tries to restart it in case your app is crashing. On modern Linux distribs, the best way to do so is by using systemd, which is the default tool dedicated to management of system services. Nothing to install then!

Let’s assume you put your Go binary in /var/www on your server. Create a new file describing your service in the systemd directory: /etc/systemd/system/my_go_website.service. Now put the following content inside:

[Unit]
Description=my_go_website
After=network.target auditd.service

[Service]
EnvironmentFile=/var/www/env
ExecStart=/var/www/my_go_website
ExecReload=/var/www/my_go_website
KillMode=process
Restart=always
RestartPreventExitStatus=255
Type=simple

[Install]
WantedBy=multi-user.target

The EnvironmentFile directive points to an env file where you can put all your environment variables. systemd takes care of loading it and passing env vars to your program. I put it in /var/www but feel free to put it somewhere else. Here is what your env file would look like:

TEMPLATES_DIR=/remote/path/to/templates/dir
STATIC_ASSETS_DIR=/remote/path/to/static/dir
TLS_CERT_PATH=/remote/path/to/cert.pem
TLS_KEY_PATH=/remote/path/to/privkey.pem

Feel free to read more about systemd for more details about the config above.

Now:

  • launch the following to link your app to systemd: systemctl enable my_go_website
  • launch the following to start your website right now: systemctl start my_go_website
  • restart with: systemctl restart my_go_website
  • stop with: systemctl stop my_go_website

Replacing Javascript with WebAssembly (Wasm)

Here is a bonus section in case you are feeling adventurous!

As of Go version 1.11, you can now compile Go to Web Assembly (Wasm). More details here. This is very cool as Wasm can work as a substitute for Javascript. In other words you can theoretically replace Javascript with Go through Wasm.

Wasm is supported in modern browsers but this is still pretty experimental. Personally I would only do this as a proof of concept for the moment, but in the mid term it might become a great way to develop your whole stack in Go. Let’s wait and see!

Conclusion

Now you know how to develop a whole website in Go and deploy it on a Linux server. No frontend server to install, no dependency hell, and great performances… Pretty straightforward isn’t it?

If you want to learn how to build a Single Page App (SPA) with Go and Vue.js, have a look at my other post here.

Existe aussi en français

API Rate Limiting With Traefik, Docker, Go, and Caching

Limiting API usage based on advanced rate limiting rule is not so easy. In order to achieve this behind the NLP Cloud API, we're using a combination of Docker, Traefik (as a reverse proxy) and local caching within a Go script. When done correctly, you can considerably improve the performance of your rate limiting and properly throttle API requests without sacrificing speed of the requests. Continue reading