Make PO files ❤️ Localization in ASP.NET Core 1.0

Mar 14, 2016     Viewed 4654 times    1 Comments
Posted in #Localization  #Pluralization  #PO 

gettext is an internationalization and localization (i18n) system commonly used for writing multilingual programs on Unix-like computer operating systems. The most commonly used implementation of gettext is GNU gettext released by the GNU project in 1995. [wikipedia]

The GNU gettext mainly using PO (Portable Object) files which are the files that contains the actual translations.

PO files is famous and widely used in many open source projects such as WordPress Orchard CMS and much more. In this article we will see how to support the PO files in ASP.NET Core 1.0 using Localization APIs & Roslyn.

Let us have a look to the following file:

msgid "
msgstr "
"Project-Id-Version: Localization Sample\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2016-03-14 15:27+0100\n"
"PO-Revision-Date: 2016-03-14 02:44+0100\n"
"Last-Translator: Hisham Bin Ateya <hishamco_2007@yahoo.com>\n"
"Language-Team: ES <es@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"

#: Startup.cs:76
msgid "Hello"
msgstr "Hola"

#: Startup.cs:75
msgid "Request Localization Sample"
msgstr "Solicitud de localizaci&oacute;n de la muestra"

#: Startup.cs:77,78
msgid "1 apple"
msgid_plural "{0} apples"
msgstr[0] "1 manzana"
msgstr[1] "{0} manzanas"

The above file is es-ES.po which is target the Spanish Language, the files start with few lines (headers) which is nothing but metadata about the file. If we notice there are some interested lines such as "Plural-Forms: nplurals=2; plural=(n != 1);\n" which contains the number of the plural forms in Spanish as well as the plural form expression.

The rest of the file contains a pair of msgid & msgstr which they represent the key or the word or the phrase to be translated and the value that contains the translation itself. Perhaps there is a msgid_plural will show up in the files, and this represent the plural key for a certain word or phrase, followed by an array of msgstr that contain all the plural forms, the key part here is the translation are determined by the above plural form in our case plural=(n != 1), so in case n is equal to 1 the msgstr[0] will show up otherwise msgstr[1].

Now let us dig into the source code.

The Entry class represents a key-value pair for both singular and plural forms.

public class Entry
{
public string SingularId { get; set; }
public string PluralId { get; set; }
public List<string> Values { get; set; }
}
The Resource class contains the metadata of the culture info and the translation them self.
public class Resource
{
public string Language { get; set; }
public int PluralsNo { get; set; }
public string PluralForm { get; set; }
public List<Entry> Entries { get; } = new List<Entry>();
}
The POParser class is the core component that parse the entire PO file and produce a meaningful Resource object.
public class POParser
{
private string _path;
private readonly Resource _resource = new Resource();

public Resource Resource
{
get { return _resource; }
}

public POParser(string path)
{
_path = path;
}

public void Parse()
{
if (!File.Exists(_path))
{
return;
}

using (var reader = new StreamReader(_path))
{
string line, key, pluralKey, value;
while ((line = reader.ReadLine()) != null)
{
if (string.IsNullOrEmpty(line))
{
continue;
}

if (line[0] == '"')
{
if (line.StartsWith("\"Plural-Forms"))
{
_resource.PluralsNo = Convert.ToInt32(line.Substring(line.IndexOf("nplurals=") + 9, 1));
_resource.PluralForm = line.Substring(line.IndexOf("plural=") + 7, line.LastIndexOf(";") - (line.IndexOf("plural=") + 7));
}
continue;
}

if (line[0] == '#')
{
continue;
}

if (line.StartsWith("msgid"))
{
key = line.Substring(7).Trim('"');

if (string.IsNullOrEmpty(key))
{
continue;
}

line = reader.ReadLine();

if (line.StartsWith("msgstr"))
{
value = line.Substring(8).Trim('"');
_resource.Entries.Add(new Entry() { SingularId = key, Values = new List<string>() { value } });

}
else if (line.StartsWith("msgid_plural"))
{
pluralKey = line.Substring(14).Trim('"');
_resource.Entries.Add(new Entry() { SingularId = key, PluralId = pluralKey, Values = new List<string>() });
while ((line = reader.ReadLine()) != null)
{
if (line.StartsWith("msgstr"))
{
value = line.Substring(line.IndexOf(" ") + 1).Trim('"');
_resource.Entries.Last().Values.Add(value);
}
}
}
}
}
}
}
}
The POStringLocalizerFactory is responsible for create the POStringLocalizer object.
public class POStringLocalizerFactory : IStringLocalizerFactory
{
private static string _resourcesRelativePath;

public POStringLocalizerFactory(IOptions<LocalizationOptions> localizationOptions)
{
if (localizationOptions == null)
{
throw new ArgumentNullException(nameof(localizationOptions));
}

_resourcesRelativePath = localizationOptions.Value.ResourcesPath ?? string.Empty;
}

public IStringLocalizer Create(Type resourceSource)
{
var cachedReourse = Cache.GetOrAdd();
return new POStringLocalizer(cachedReourse);
}

public IStringLocalizer Create(string baseName, string location)
{
return Create(null);
}

private static class Cache
{
private static readonly Dictionary<string, Resource> _lookup = new Dictionary<string, Resource>();

public static Resource GetOrAdd()
{
var culture = CultureInfo.CurrentCulture.ToString();

if(!_lookup.ContainsKey(culture))
{
var path = Path.Combine(_resourcesRelativePath, culture + ".po");
var parser = new POParser(path);

parser.Parse();
_lookup.Add(culture, parser.Resource);
}
return _lookup[culture];
}
}
}
The POStringLocalizer class is responsible to fetch the translation from the underlying source.
public class POStringLocalizer : IStringLocalizer
{
private readonly Resource _resource;

public POStringLocalizer(Resource resource)
{
_resource = resource;
}

public LocalizedString this[string name]
{
get
{
var value = GetString(name);
return new LocalizedString(name, value ?? name, resourceNotFound: value == null);
}
}

public LocalizedString this[string name, params object[] arguments]
{
get
{
var format = GetString(name);
var value = string.Format(format ?? name, arguments);
return new LocalizedString(name, value, resourceNotFound: format == null);
}
}

public IStringLocalizer WithCulture(CultureInfo culture)
{
CultureInfo.DefaultThreadCurrentCulture = culture;
return new POStringLocalizer(_resource);
}

public IEnumerable<LocalizedString> GetAllStrings(bool includeAncestorCultures)
{
return null;
}

private string GetString(string name)
{
if (_resource == null)
return name;
var resource = _resource.Entries.SingleOrDefault(e => e.SingularId == name);
return resource == null ? name : resource.Values[0];
}

public string Plural(string name, int count)
{
if (!_resource.Entries.Any(e => e.PluralId == name))
{
return name;
}

int index = GetPluralRule(count);
return string.Format(_resource.Entries.Single(e => e.PluralId == name).Values[index], count);
}

private int GetPluralRule(int n)
{
var culure = CultureInfo.Culture.ToString();
if(culture == "fr")
{
return Convert.ToInt32(n > 1);
}
else
{
return (n != 1 ? 1 : 0);
}
}
}

The code of GetPluralRule is very stupid :) I made it for the sake of the demo, but you can use the one that I mentioned in my previous article Pluralization in ASP.NET Core 1.0.

Also you can use the Roslyn APIs to evaluate the plural form expression that available in the PO file on fly. The code may change to something like this:

private async int GetPluralRule(int n)
{
var expression = _resource.PluralForm.Replace("n", n.ToString());
object result = await CSharpScript.EvaluateAsync(expression);
return Convert.ToInt32(result);
}

In the above function I used the Scripting APIs which is a part of the Roslyn source code, which provide a rich APIs to deal with and execute a CSharp scripts at the run-time, for more information about Roslyn Scripting APIs check this link, also you can check the entire source code which is available on github.

At the end of this article I hope we make PO ❤️ Localization in ASP.NET Core 1.0.

Twitter Facebook Google + LinkedIn


1 Comment

Tristan (12/20/2016 5:46:58 AM)

Hey, if you need a tool to manage localization projects more easily, which is also integrated with GitHub, try this collaborative online tool https://poeditor.com/
It fully supports gettext .po files and has all sorts of features that make it possible to automate the workflow.


Leave a Comment