Difference between revisions of "Translations / i18n / localizations for programs"

From Free Pascal wiki
(BiDiMode)
(looking for language files)
(4 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 
{{Translations_/_i18n_/_localizations_for_programs}}
 
{{Translations_/_i18n_/_localizations_for_programs}}
==Overview==
 
  
This is about how a program can use different strings for various languages like english, chinese, german, finnish, italian and any other language.
+
This is about how a program can use different strings for various languages like English, Chinese, German, Finnish, Italian and any other language.
 +
 
 +
* '''i18n''' is short for '''internationalization''' (18 being the number of characters between i and n of the word internationalization)
 +
* '''l10n''' is short for '''localization''' (10 being the number of characters between l and n of the word localization)
 +
 
  
 
==Quick i18n==
 
==Quick i18n==
Line 9: Line 12:
  
 
For your information the most used languages in the world ([http://www.redlinels.com/2014/01/10/most-widely-spoken-languages/ source]) are:
 
For your information the most used languages in the world ([http://www.redlinels.com/2014/01/10/most-widely-spoken-languages/ source]) are:
    * 1 - Chinese: With more than 1.2 billion native speakers in the world
+
 
    * 2 - Spanish: Spanish occupies the No. 2 spot and is spoken in approximately 30 countries.
+
* 1 - Chinese: With more than 1.2 billion native speakers in the world
    * 3 - English: 335 million worldwide—about 5% of the world’s population. (This does not takes into account second language spoken).
+
* 2 - Spanish: Spanish occupies the No. 2 spot and is spoken in approximately 30 countries.
    * 4 - Hindi: Spoken by over 260 million people.
+
* 3 - English: 335 million worldwide—about 5% of the world’s population. (This does not takes into account second language spoken).
    * 5 - Arabic: Spoken in almost 60 countries around the world.
+
* 4 - Hindi: Spoken by over 260 million people.
    * 6 - Portuguese: Population of Brazil is over 200 million. The population of Portugal is just over 10 million.
+
* 5 - Arabic: Spoken in almost 60 countries around the world.
    * 7 - Bengali: The main language of Bangladesh (population, 155 million) and one of India’s many official languages.
+
* 6 - Portuguese: Population of Brazil is over 200 million. The population of Portugal is just over 10 million.
    * 8 - Russian and Japanese.
+
* 7 - Bengali: The main language of Bangladesh (population, 155 million) and one of India’s many official languages.
 +
* 8 - Russian and Japanese.
 +
 
 
===poedit===
 
===poedit===
  
Line 23: Line 28:
 
===Translating Resourcestrings===
 
===Translating Resourcestrings===
  
This is the way to store a resourcestring in an unit:
+
This is the way to store a resourcestring in a unit:
<syntaxhighlight>resourcestring
+
 
 +
<syntaxhighlight lang="pascal">
 +
resourcestring
 
   Caption1 = 'Some text';
 
   Caption1 = 'Some text';
   HelloWorld1 = 'Hello World';</syntaxhighlight>
+
   HelloWorld1 = 'Hello World';
 +
</syntaxhighlight>
  
 
Resourcestrings are like normal string constants, that means you can assign them to any string:
 
Resourcestrings are like normal string constants, that means you can assign them to any string:
<syntaxhighlight>Label1.Caption := HelloWorld1;</syntaxhighlight>
+
 
 +
<syntaxhighlight lang="pascal">Label1.Caption := HelloWorld1;</syntaxhighlight>
  
 
===Compiling into .po files===
 
===Compiling into .po files===
Line 40: Line 49:
  
 
Your folder structure will look something like this:
 
Your folder structure will look something like this:
<syntaxhighlight>
+
 
 +
<syntaxhighlight lang="text">
 
project1\po_files\
 
project1\po_files\
 
project1\po_files\project1.po
 
project1\po_files\project1.po
Line 49: Line 59:
  
 
When you have the translation finished in the .po file, you want to compile it to .mo to load it faster since is a binary format. To convert to .mo you can use poedit and go to File > Compile as .mo.
 
When you have the translation finished in the .po file, you want to compile it to .mo to load it faster since is a binary format. To convert to .mo you can use poedit and go to File > Compile as .mo.
 +
 +
Note that [[Everything else about translations]] has a section about where a Lazarus app looks for its language files.
  
 
===Automatic translation===
 
===Automatic translation===
Line 54: Line 66:
 
When you have the .mo files ready, put them in the '''locale''' or '''languages''' folder right to your application executable, then include the unit '''DefaultTranslator''' and that's all. The translation will be done automatically.
 
When you have the .mo files ready, put them in the '''locale''' or '''languages''' folder right to your application executable, then include the unit '''DefaultTranslator''' and that's all. The translation will be done automatically.
  
<syntaxhighlight>uses
+
<syntaxhighlight lang="pascal">
DefaultTranslator;
+
uses
 +
  DefaultTranslator;
 
</syntaxhighlight>
 
</syntaxhighlight>
  
You want to distribute only the .mo files in the '''locale''' or '''languages''' directory, since .po files are usefull only for making the translation and compiling them into .mo.
+
You want to distribute only the .mo files in the '''locale''' or '''languages''' directory, since .po files are usefull only for making the translation and compiling them into a .mo file.
  
 
Your folder structure will look something like this:
 
Your folder structure will look something like this:
<syntaxhighlight>
+
 
 +
<syntaxhighlight lang="text">
 
project1\project1.exe
 
project1\project1.exe
 
project1\locale\
 
project1\locale\
Line 70: Line 84:
 
===Testing translations===
 
===Testing translations===
  
When you have everything ready you want to test if translations looks fine in your application for each language you have. Automatic translation has a feature that you can use in order to test each language quickly.
+
When you have everything ready. you want to test if the translations looks fine in your application for each language you have. Automatic translation has a feature that you can use in order to test each language quickly.
  
 
You must run your executable with the command line parameter '''--lang''' followed by the language code of your choice.
 
You must run your executable with the command line parameter '''--lang''' followed by the language code of your choice.
Line 76: Line 90:
 
You will run your executable like this in order to test Spanish translation:
 
You will run your executable like this in order to test Spanish translation:
  
<syntaxhighlight>
+
<syntaxhighlight lang="text">
 
project1.exe --lang es
 
project1.exe --lang es
 
</syntaxhighlight>
 
</syntaxhighlight>
Line 84: Line 98:
 
You can do this with the IDE. Go to '''Run > Run Parameters ...'''. In that window in the input '''Command line parameters (without application name)''' write this:
 
You can do this with the IDE. Go to '''Run > Run Parameters ...'''. In that window in the input '''Command line parameters (without application name)''' write this:
  
<syntaxhighlight>
+
<syntaxhighlight lang="text">
 
--lang it
 
--lang it
 
</syntaxhighlight>
 
</syntaxhighlight>
Line 92: Line 106:
 
===Final steps===
 
===Final steps===
  
These steps are to get a better result of your entire translation.
+
These steps are to get a better result for your entire translation.
  
 
====Translate LCL====
 
====Translate LCL====
Line 98: Line 112:
 
To get everything translated you must include the LCL translations into your application '''locale''' folder.
 
To get everything translated you must include the LCL translations into your application '''locale''' folder.
  
Copy everything inside the folder '''C:\lazarus\lcl\languages''' to yout '''locale''' folder. Then you will have the LCL translated for your application.
+
Copy everything inside the folder '''C:\lazarus\lcl\languages''' to your '''locale''' folder. Then you will have the LCL translated for your application.
  
 
====Format Settings====
 
====Format Settings====
  
For Windows you must read [[Step-by-step instructions for creating multi-language applications#Format settings]] to get format settings in right place.
+
For Windows you must read [[Step-by-step instructions for creating multi-language applications#Format settings]] to get format settings in the right place.
  
Under Linux, BSD and Mac OSX there are several locales defining things like time and date format or the thousand separator. In order to initialize the RTL you need to include the '''clocale''' unit in the uses section of your program (.lpr file).
+
Under Linux, BSD and macOS there are several locales defining things like time and date format or the thousands separator. In order to initialize the RTL you need to include the '''clocale''' unit in the uses section of your program (.lpr file).
  
 
====BiDiMode====
 
====BiDiMode====
Line 110: Line 124:
 
[[BidiMode]] is for languages like Arabic that read and write right to left. See also [http://www.w3.org/International/questions/qa-scripts.en Script direction and languages] for a list of LTR (Left To Right) and RTL (Right To Left) languages and most used languages in general.
 
[[BidiMode]] is for languages like Arabic that read and write right to left. See also [http://www.w3.org/International/questions/qa-scripts.en Script direction and languages] for a list of LTR (Left To Right) and RTL (Right To Left) languages and most used languages in general.
  
With this unit you can determine what BiDiMode is right for you.
+
With this unit you can determine what BiDiMode is right for you. This code is tested in Lazarus Trunk and may or not work in current release (see the changes on this page to get the old source).
  
 
First save this unit as ubidimodetools.pas:
 
First save this unit as ubidimodetools.pas:
  
<syntaxhighlight>unit ubidimodetools;
+
<syntaxhighlight lang="pascal">
 +
unit ubidimodetools;
  
 
{$mode objfpc}{$H+}
 
{$mode objfpc}{$H+}
Line 129: Line 144:
  
 
uses
 
uses
   FileUtil, LCLProc;
+
   FileUtil, LCLProc, LazUTF8;
  
 
function GetLang: string;
 
function GetLang: string;
Line 136: Line 151:
 
   i: integer;
 
   i: integer;
 
begin
 
begin
 +
  Result := '';
 
   { We use the same method that is used in LCLTranslator unit }
 
   { We use the same method that is used in LCLTranslator unit }
  
Line 148: Line 164:
  
 
   if Result = '' then
 
   if Result = '' then
     LCLGetLanguageIDs(Result, T);
+
     LazGetLanguageIDs(Result, {%H-}T);
 
end;
 
end;
  
// ToDO: Add a full list of languages specifing their BiDiMode in the function GetBiDiMode.
 
 
function GetBiDiMode: TBiDiMode;
 
function GetBiDiMode: TBiDiMode;
 
begin
 
begin
Line 162: Line 177:
 
end;
 
end;
  
end.
+
end.
 
</syntaxhighlight>
 
</syntaxhighlight>
  
 
When you create a form do this:
 
When you create a form do this:
  
<syntaxhighlight>
+
<syntaxhighlight lang="pascal">
 
BiDiMode := GetBiDiMode;
 
BiDiMode := GetBiDiMode;
 
</syntaxhighlight>
 
</syntaxhighlight>
  
 
'''Note:''' You must be sure that you have the proper translations in order to get it working.
 
'''Note:''' You must be sure that you have the proper translations in order to get it working.
 +
 +
'''Note 2:''' Maybe this is not neccesary at all since Windows can change the BiDiMode automatically.
  
 
===Technical Details===
 
===Technical Details===
 
  
 
====gettext====
 
====gettext====
Line 180: Line 196:
 
The main technology involved in the process of translations is [https://www.gnu.org/software/gettext/manual/gettext.html GNU gettext]. FPC comes with the gettext unit.
 
The main technology involved in the process of translations is [https://www.gnu.org/software/gettext/manual/gettext.html GNU gettext]. FPC comes with the gettext unit.
  
<syntaxhighlight>uses
+
<syntaxhighlight lang="pascal">
gettext;</syntaxhighlight>
+
uses
 +
  gettext;
 +
</syntaxhighlight>
  
 
====PO====
 
====PO====
  
PO – Portable Object. This is the file that you receive back from the translators. It’s a text file that includes the original texts and the translations.
+
PO – Portable Object. This is the file that you receive back from the translators. It’s a text file that includes the original text and the translations.
  
 
====MO====
 
====MO====
  
MO – Machine Object. The MO file includes the exact same contents as PO file. The two files differ in their format. While a PO file is a text file and is easy for humans to read, MO files are compiled and are easy for computers to read. The unit gettext implements TMOFile and has several procedures to do the translation from .mo files, if you want to use it.
+
MO – Machine Object. The MO file includes the exact same contents as the PO file. The two files differ in their format. While a PO file is a text file and is easy for humans to read, MO files are compiled and are easy for computers to read. The unit gettext implements TMOFile and has several procedures to do the translation from .mo files, if you want to use it.
  
<syntaxhighlight>
+
<syntaxhighlight lang="pascal">
 
unit gettext;
 
unit gettext;
  
Line 209: Line 227:
 
==Everything else about translations==
 
==Everything else about translations==
  
Here goes all the translations stuff that was here in the past. There are a lot of articles and stuff about translations that can cause trouble to novices. To keep it simple these other stuff is moved here: [[Everything else about translations]]. Please keep this article as clean as possible.
+
Here is all the translation material that was here in the past. There are a lot of articles about translations that can cause trouble to novices. To keep it simple this other material has been moved to: [[Everything else about translations]]. Please keep this article as clean as possible.
 +
 
 +
==See also==
  
===See also===
 
 
* [[IDE_Development#Translations.2C_i18n.2C_lrt_files.2C_po_files|IDE Development: Translations, i18n, lrt, po files]]
 
* [[IDE_Development#Translations.2C_i18n.2C_lrt_files.2C_po_files|IDE Development: Translations, i18n, lrt, po files]]
 
* [[Getting_translation_strings_right|Getting translation strings right]]
 
* [[Getting_translation_strings_right|Getting translation strings right]]

Revision as of 11:14, 15 November 2020

Deutsch (de) English (en) español (es) français (fr) 日本語 (ja) 한국어 (ko) português (pt) русский (ru) 中文(中国大陆)‎ (zh_CN)

This is about how a program can use different strings for various languages like English, Chinese, German, Finnish, Italian and any other language.

  • i18n is short for internationalization (18 being the number of characters between i and n of the word internationalization)
  • l10n is short for localization (10 being the number of characters between l and n of the word localization)


Quick i18n

This is intended as be a quick guide to introduce yourself in the world of translations and get things done quickly. Check also Language Codes and BidiMode.

For your information the most used languages in the world (source) are:

  • 1 - Chinese: With more than 1.2 billion native speakers in the world
  • 2 - Spanish: Spanish occupies the No. 2 spot and is spoken in approximately 30 countries.
  • 3 - English: 335 million worldwide—about 5% of the world’s population. (This does not takes into account second language spoken).
  • 4 - Hindi: Spoken by over 260 million people.
  • 5 - Arabic: Spoken in almost 60 countries around the world.
  • 6 - Portuguese: Population of Brazil is over 200 million. The population of Portugal is just over 10 million.
  • 7 - Bengali: The main language of Bangladesh (population, 155 million) and one of India’s many official languages.
  • 8 - Russian and Japanese.

poedit

The best known tool is a program called poedit. poedit is a tool for translators. It produces both PO and MO as output.

Translating Resourcestrings

This is the way to store a resourcestring in a unit:

resourcestring
  Caption1 = 'Some text';
  HelloWorld1 = 'Hello World';

Resourcestrings are like normal string constants, that means you can assign them to any string:

Label1.Caption := HelloWorld1;

Compiling into .po files

Resourcestrings are compiled into .po files if you enable i18n in the Lazarus IDE. Go to Project > Project Options > i18n > Enable i18n. When you recompile your application the .po files will be updated. You may also select a directory where .po files will be stored, recommended po_files.

The default translation is recommended to be in English or the default language of your application, since it will be loaded if no other translation file is found.

When you have your project1.po copy and paste it and rename it to project1.es.po or one of the Language Codes of your choice. Then you will have 2 languages: English and Spanish. Then you must send to the translator the files for translation.

Your folder structure will look something like this:

project1\po_files\
project1\po_files\project1.po
project1\po_files\project1.es.po

Converting .po files to .mo files

When you have the translation finished in the .po file, you want to compile it to .mo to load it faster since is a binary format. To convert to .mo you can use poedit and go to File > Compile as .mo.

Note that Everything else about translations has a section about where a Lazarus app looks for its language files.

Automatic translation

When you have the .mo files ready, put them in the locale or languages folder right to your application executable, then include the unit DefaultTranslator and that's all. The translation will be done automatically.

uses
  DefaultTranslator;

You want to distribute only the .mo files in the locale or languages directory, since .po files are usefull only for making the translation and compiling them into a .mo file.

Your folder structure will look something like this:

project1\project1.exe
project1\locale\
project1\locale\project1.mo
project1\locale\project1.es.mo

Testing translations

When you have everything ready. you want to test if the translations looks fine in your application for each language you have. Automatic translation has a feature that you can use in order to test each language quickly.

You must run your executable with the command line parameter --lang followed by the language code of your choice.

You will run your executable like this in order to test Spanish translation:

project1.exe --lang es

And you will see the translated application.

You can do this with the IDE. Go to Run > Run Parameters .... In that window in the input Command line parameters (without application name) write this:

--lang it

Then Run (F9) and you will see the translated application.

Final steps

These steps are to get a better result for your entire translation.

Translate LCL

To get everything translated you must include the LCL translations into your application locale folder.

Copy everything inside the folder C:\lazarus\lcl\languages to your locale folder. Then you will have the LCL translated for your application.

Format Settings

For Windows you must read Step-by-step instructions for creating multi-language applications#Format settings to get format settings in the right place.

Under Linux, BSD and macOS there are several locales defining things like time and date format or the thousands separator. In order to initialize the RTL you need to include the clocale unit in the uses section of your program (.lpr file).

BiDiMode

BidiMode is for languages like Arabic that read and write right to left. See also Script direction and languages for a list of LTR (Left To Right) and RTL (Right To Left) languages and most used languages in general.

With this unit you can determine what BiDiMode is right for you. This code is tested in Lazarus Trunk and may or not work in current release (see the changes on this page to get the old source).

First save this unit as ubidimodetools.pas:

unit ubidimodetools;

{$mode objfpc}{$H+}

interface

uses
  Classes, SysUtils;

function GetLang: string;
function GetBiDiMode: TBiDiMode;

implementation

uses
  FileUtil, LCLProc, LazUTF8;

function GetLang: string;
var
  T: string; // unused FallBackLang
  i: integer;
begin
  Result := '';
  { We use the same method that is used in LCLTranslator unit }

  for i := 1 to Paramcount - 1 do
    if (ParamStrUTF8(i) = '--LANG') or (ParamStrUTF8(i) = '-l') or
      (ParamStrUTF8(i) = '--lang') then
      Result := ParamStrUTF8(i + 1);

  //Win32 user may decide to override locale with LANG variable.
  if Result = '' then
    Result := GetEnvironmentVariableUTF8('LANG');

  if Result = '' then
    LazGetLanguageIDs(Result, {%H-}T);
end;

function GetBiDiMode: TBiDiMode;
begin
  case GetLang of
    // Arabic
    'ar': Result := bdRightToLeft;
    else
      Result := bdLeftToRight;
  end;
end;

end.

When you create a form do this:

BiDiMode := GetBiDiMode;

Note: You must be sure that you have the proper translations in order to get it working.

Note 2: Maybe this is not neccesary at all since Windows can change the BiDiMode automatically.

Technical Details

gettext

The main technology involved in the process of translations is GNU gettext. FPC comes with the gettext unit.

uses
  gettext;

PO

PO – Portable Object. This is the file that you receive back from the translators. It’s a text file that includes the original text and the translations.

MO

MO – Machine Object. The MO file includes the exact same contents as the PO file. The two files differ in their format. While a PO file is a text file and is easy for humans to read, MO files are compiled and are easy for computers to read. The unit gettext implements TMOFile and has several procedures to do the translation from .mo files, if you want to use it.

unit gettext;

...

TMOFile = class

...

  procedure GetLanguageIDs(var Lang, FallbackLang: string);
  procedure TranslateResourceStrings(AFile: TMOFile);
  procedure TranslateUnitResourceStrings(const AUnitName:string; AFile: TMOFile);
  procedure TranslateResourceStrings(const AFilename: String);
  procedure TranslateUnitResourceStrings(const AUnitName:string; const AFilename: String);

Everything else about translations

Here is all the translation material that was here in the past. There are a lot of articles about translations that can cause trouble to novices. To keep it simple this other material has been moved to: Everything else about translations. Please keep this article as clean as possible.

See also