A Step-by-Step Guide to Go Internationalization (i18n) & Localization (l10n)

Go is a statically compiled language that gained a lot of popularity lately due to the fact that is simple, performant and fits really well with developing cloud applications. It has a strong, yet poorly documented sub-package level base library that deals with a lot of aspects related to internationalization (i18n) and localization (l10n), such as character encodings, text transformations, and locale-specific text handling. Let's see what we can do to master this library and make our Go applications locale aware.

The package we are referring to is thegolang.org/x/text and if utilized correctly you can be pretty much cover a lot of parts when it comes to globalizing your apps. It comes with a set of abstractions to make easier to work with translatable messages, formatting, plural rules, Unicode and much more.

This article is going to consist of 2 parts. The first part is an overview of the golang.org/x/text package and the utilities it provides in terms of formatting and localization. Go excels at building microservice based architectures so in the second part, in order not to break this tradition, we are going to make a localization Server microservice that will help us understand the big picture of i18n and l10n support in Go.

For the purposes of this tutorial, I will be using the latest Go v1.10 and the code for this tutorial is hosted on Github.

Let’s get going.

Overview of  the package

Most messages in Go programs pass through either thefmt  or one of the template packages.

The golang.org/x/text consists of multiple levels of sub-packages that offer lots of utilities and functions to format localized strings using a fmt style API. Let’s see how we can use it in practice.

Messages and Catalogs

A message is some form of content to be conveyed to the user. Each message is identified by a key, which can have many forms. You can create a message printer like that:

You need to supply a Language Tag when you call the NewPrinter function. Language tags are used whenever you want to specify a language. There are many ways you can create a tag such as:

  • Using predefined tags. For example:

    The whole list of predefined tags is listed here.
  • From a string value. For example:
  • By composing parts of type Tag, Base, Script, Region, Variant, []Variant, Extension, []Extension or error. For example:

    If you specify an invalid language tag you will get an instance of the Und Tag which denotes an Undefined Tag.

If you want to learn more about the language API see this doc here.

Coming back to our messages we can assign a new printer using a different language and print the formatted strings. The library will take care any localized formatting variants for you:

If you run this program you will get:

Now in order to print translated messages, we need to add them to the message catalog so that the Printer can find them for the right language tag.

A Catalog defines collections of translated format strings. Think of it as a set of per-language dictionaries with translations for a set of keys. In order to use catalogs, we need to populate them with translations.

In practice, translations will be automatically injected from a translator-supplied data source. Let’s see how we can do it manually:

If you run this program you will get the following output:

Caution: The keys you specify when you use theSetString  method are case and line sensitive, which means that if you try to usePrintLn  or add an end of line char\n then it won’t work:

Typically you don’t create catalogs but let the library handle them for you. You can also have the option to build ones programmatically using the catalog.Builder function.

Handling Plurals

For cases when you need to add multiple string translations depending on plural values, you need to add special calls to configure that in your translation catalogs. The sub-package golang.org/x/text/feature/plural  exposes a function called SelectF that is used to define multiple linguistic plurals in a text.

I give below some typical usages of this function:

If you run this program you will get the following output:

The cases as provided in this function can support several variations such aszero , one , two , few ,many  and it can also match comparisons such as >x  or<x .

String interpolation in Messages

In some other cases where you want to handle further possible variants of a message, you can assign placeholder variables that can handle some specific cases of linguistic features. For instance, in the previous example where we used the plural can be written as:

The catalog.Var assigns a special tag to the first string parameter minutes so it can be substituted with a more relevant translation based on the value of the%d parameter.

Formatting Currency

Package  golang.org/x/text/currency deals with currency formatting rules.

For currency, there are some useful functions to print locale-specific strings regarding amounts. For example here are some ways you can format them:

And the result will be:

Loading messages

When you work with translations typically you will need to load the translations before so that the application can use them. You can think of those files as static resources. You have a few options on how you deploy those files with the application:

Manually setting the translation strings

The simplest way to organize the translations is to have them assigned into the application binary. You will have to manually create an array of entries that will be used on init to load the messages into the default catalog. Then on your application, you only have to switch locale using theNewPrinter function.

Bellow is an example application by loading translations on init:

If you run this program then it will print:

In practice, while this way is simple to implement, it’s not scalable enough. It works only for small applications with few translations. You will have to manually set the translation strings and it’s tricky to automate. For all other reasons, it’s recommended to automatically load messages where I explain in detail how to do it next.

Automatic loading of messages

Traditionally, most localization frameworks have grouped data in per-language dynamically-loaded files. You can distribute those files to translators and have them merged into your app when they are ready.

To assist in this process the authors have included a helper CLI tool calledgotext that is used for managing text in Go source code.

Let’s start by making sure that you have the latest version:

Running this tool will only show the options available and thehelp  switch will not show any other info:

For the purposes of this tutorial let’s use the update flag which performs a multi-step process of extracting the translation keys to a file and updating the code for loading them into catalogs for ease of use.

Create a file main.go and add a fewPrintF  calls and make sure you include the comment for thego:generate  command

File: main.go

Run the following commands:

Then fix the import to include the catalog.go file:

File: main.go

Now if you see the project structure there are some files created:

The locales folder contain the translation messages in the format that the library supports. Typically you want to provide translations for this. Create a new file namedmessages.gotext.json and provide translations for the Greek language.

File: locales/el/messages.gotext.json

Now run thego generate  command and the program next and see that the translations are happening:

In case you are interested, the rewrite flag searches for references tofmt in the source code and replaces them with thep.Print  functions. For example, let’s say we have the following program:

File: main.go

If you run the following command:

Then the main.go will turn into:

Example Microservice

This is the second part of the article where we can utilize in practice what we learned about the golang/x/text package. We are going to build a simple HTTP server that will serve an endpoint that will accept a user language parameter. Then It will try to match this parameter with the list of supported languages and then serve a translated response based on the most suitable locale.

First, make sure you have all dependencies installed:

Start by creating an application skeleton:

File: main.go

This example HTTP server does not handle translations yet. We can do that by replacing the call tofmt.FprintF  with the call top.FprintF

Add the following line to your source code and run the go generate command:

Provide translations for the missing entries:

File: locales/el/messages.gotext.json

Run the command go generate again and add a reference to the catalog package inmain.go :

File: main.go

Now in order to determine which language we need to switch when the user requests a resource from the API we need to add a Matcher object that will be used to determine the best match out of our supported locales when provided with a list of language tags.

Create a new Matcher by providing the list of supported locales from themessage.DefaultCatalog  that is populated from thegotext  tool:

File: main.go

Add your function to match the correct language based on the request parameters:

File: main.go

I only supplied a parameter parsed from the query string. You can mix and match also additional tags parsed from a cookie or an Accept-Language header.

Now you only need to wrap your handler functionPrintMessage  with thewithMessagePrinter and grab the printer from the context:

File: main.go

Start the server and issue some requests to see the translations happening:

The world is your oyster from now on…

Use PhraseApp

PhraseApp supports many different languages and frameworks, including Go. It allows to easily import and export translations data and search for any missing translations, which is really convenient. On top of that, you can collaborate with translators as it is much better to have professionally done localization for your website. If you’d like to learn more about PhraseApp, refer to the Getting Started guide. You can also get a 14-days trial. So what are you waiting for?

Conclusion

In this article, we explored how Go manages localization using thegolang/x/text  package and we implemented an example web server that serves translations and explaining how all pieces fit together. As the official documentation lacks in terms of practical examples, I hope that this article explained out the principles of adding i18n, in a simple manner, to your Go applications and could be used as a good future reference.

This is by no means an exhaustive guide as every application has different needs and scope requirements. Please stay put for more detailed articles regarding this subject.

 

A Step-by-Step Guide to Go Internationalization (i18n) & Localization (l10n)
5 (100%) 11 votes
Related Posts
Comments