Difference between revisions of "Using resourcestrings"

From Free Pascal wiki
Jump to navigationJump to search
(added language template)
(remove from Category: Pages with syntax highlighting errors [my bad], remove Category: Localization already taken care of via transcluded {{Using resourcestrings}})
 
(5 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{Using_resourcestrings}}
+
{{Using resourcestrings}}
  
The .rst file is created to provide a mechanism to localize your application.
+
Resource strings provide a mechanism to internationalize (and to some degree localize) your application.
Currently, only one localization mechanism is provided: gettext.
 
  
The steps are as follows:
+
== rationale ==
 +
Resource strings are constants with an extra level of indirection.
 +
The default value (as written in your [[Source code|source code]]) is stored in the executable program.
 +
That means it is always available, even if loading the translation fails.
  
# Compiler creates .rst file.
+
== common steps ==
# rstconv tool converts to .po (input for gettext) This file can be translated to many languages. All standard gettext tools can be used.
+
The first step is to ''use'' resource strings for every string that potentially needs translation.
# Gettext creates .mo files.
+
In this tutorial we want to internationalize our Hello World program:
# .mo files are read by gettext unit and all resourcestrings are translated.
+
<syntaxhighlight lang="delphi" highlight="2-3">
 +
program helloWorld(input, output, stdErr);
 +
resourceString
 +
hello = 'Hello world';
 +
begin
 +
writeLn(hello);
 +
end.
 +
</syntaxhighlight>
 +
The [[FPC]] creates a file ending in <syntaxhighlight lang="text" inline>.rsj</syntaxhighlight> (resource string [[JSON]]) for every compiled module containing a <syntaxhighlight lang="delphi" inline>resourceString</syntaxhighlight> section.
 +
For the above program it looks like this (pretty-format added):
 +
<syntaxhighlight lang="json">
 +
{
 +
  "version": 1,
 +
  "strings": [
 +
    {
 +
      "hash": 20229140,
 +
      "name": "helloworld.hello",
 +
      "sourcebytes": [72, 101, 108, 108, 111, 32, 119, 111, 114, 108, 100],
 +
      "value": "Hello world"
 +
    }
 +
  ]
 +
}
 +
</syntaxhighlight>
 +
This intermediate file format was created, so the compiler does not need any knowledge of the utilized translation tool.
 +
It just needs to create <syntaxhighlight lang="text" inline>.rsj</syntaxhighlight> files.
  
The calls needed to translate all resourcestrings are in the objpas unit.
+
The following steps differ on the used internationalization mechanism.
They are documented.
+
As of 2021 the FPC only supports [[GNU]] gettext entirely.
 +
The reason gettext was chosen is that it is more or less standard on Unix-like systems, like [[Linux]] or [[FreeBSD]].
  
Nothing stops people from creating a mechanism that does not depend on gettext.
+
== GNU get text ==
One could implement a mechanism to create resource DLL's (as delphi does) which
+
=== limitations ===
contain the translated texts. In fact, output to .rc files, i.e. source texts for
+
As mentioned, GNU gettext is widely available on Unix-like systems.
resource compiler, is already available in rstconv - however, portable functions
+
However, gettext is horribly inefficient.
for loading the texts from such DLLs are missing (see point 3 below). The same
+
Also, gettext is context insensitive, that means it operates on the string itself:
applies to the third output format supported by rstconv at the moment, IBM OS/2 MSG
+
Sometimes the same word/sentence must be translated differently according to the context, but this is not possible.
files.
 
  
The reason gettext was chosen is that it's more or less standard on Unix.
+
=== procedure ===
But Gettext is horribly inefficient, so if someone has a better idea, please do.
+
# Convert the <syntaxhighlight lang="text" inline>.rsj</syntaxhighlight> file into GNU gettext’s portable object format:<syntaxhighlight lang="bash">rstconv -f po -i helloWorld.rsj -o helloWorld.po</syntaxhighlight>
Plus, GetText is context insensitive (it operates on the string itself),
+
# Create a working copy of the generated file, e.&#8239;g.<syntaxhighlight lang="bash">cp helloWorld.po helloWorld-de.po</syntaxhighlight>and translate it with your favorite translation tool or simply with a text editor.<syntaxhighlight lang="gettext">#: helloworld:hello
which is a drawback: sometimes the same word/sentence must be translated
+
msgid "Hello world"
differently according to the context, and this is not possible.
+
msgstr "Hallo Welt"
  
 +
</syntaxhighlight>
 +
# Convert the PO file to an MO (machine object) file using gettext’s utility <syntaxhighlight lang="text" inline>msgfmt</syntaxhighlight>:<syntaxhighlight lang="bash">msgfmt -o helloWorld-de.mo helloWorld-de.po</syntaxhighlight>
 +
# Integrate gettext in your program:<syntaxhighlight lang="delphi" highlight="2-3,7">program helloWorld(input, output, stdErr);
 +
uses
 +
gettext;
 +
resourceString
 +
hello = 'Hello world';
 +
begin
 +
translateResourceStrings('helloWorld-%s.mo');
 +
writeLn(hello);
 +
end.</syntaxhighlight>The <syntaxhighlight lang="text" inline>%s</syntaxhighlight> is substituted with the locale’s name. Examining an <syntaxhighlight lang="text" inline>strace</syntaxhighlight> you will see, that for example first <syntaxhighlight lang="text" inline>helloWorld‑de.mo</syntaxhighlight> is attempted to open. If it does not exist, it tries <syntaxhighlight lang="text" inline>helloWorld‑de_DE.mo</syntaxhighlight>. If neither file exists, the resource string’s default values as specified in your source code are used.
 +
# Run the program and celebrate your achievement:<syntaxhighlight lang="bash">LANG='de_DE.UTF-8' ./helloWorld</syntaxhighlight>
 +
For more details see the {{Doc|package=FCL|unit=gettext|text=gettext unit documentation}}.
 +
 +
== resource DLLs ==
 +
The FPC’s standard RTL does not yet contain portable functions for loading texts from resource [[DLL]]s containing translations (as [[Delphi]] does), but the first step would be:
 +
# Convert the <syntaxhighlight lang="text" inline>.rsj</syntaxhighlight> file into a resource compiler script:<syntaxhighlight lang="bash">rstconv -f rc -i helloWorld.rsj -o helloWorld.rc</syntaxhighlight>
 +
 +
== message files ==
 +
The FPC currently does not support this, but the first step would be:
 +
# Convert the <syntaxhighlight lang="text" inline>.rsj</syntaxhighlight> file in an IBM OS/2 message file:<syntaxhighlight lang="bash">rstconv -f msg -c HWX -i helloWorld.rsj -o helloWorld.msg</syntaxhighlight>The component identifier indicated by <syntaxhighlight lang="text" inline>‑c</syntaxhighlight> may be any three upper-case [[ASCII]] letters, but it should be unique within your project.
 +
 +
== alternatives ==
 
To implement another mechanism, 3 things are needed:
 
To implement another mechanism, 3 things are needed:
 
# Update rstconv so it can output another format.
 
# Update rstconv so it can output another format.
# Tools to manipulate the other format.  
+
# Tools to manipulate the other format.
# Implement a unit that loads the other format at runtime.
+
# Implement a unit that loads the other format at [[runtime|run-time]].
 
+
[[Routine]]s to directly manipulate a program’s resource string tables, i.&#8239;e. the extra level of indirection for resource strings, were available in the objpas unit {{gitlab||fpc|commit/4494565a154b1c6b313c9ddc3a09f7102719c5b3|until revision 31687}}.
This is also the reason we create an intermediate file format: this was the compiler
 
needs no knowledge of the translation tool. It just needs to create the .rst file.
 
 
 
An alternate way of doing it would e.g. be create a ini file per language, with a
 
section for each unit used, and a key for each string.
 
  
english.ini:
+
== history ==
[sysutils]
+
Formerly, the FPC created text files ending in <syntaxhighlight lang="text" inline>.rst</syntaxhighlight>.
SErrInvalidDateTime="%S" is not a valid date/time indication.
+
While changing to the JSON format, the name of the [https://www.freepascal.org/docs-html/user/userse45.html <syntaxhighlight lang="text" inline>rstconv</syntaxhighlight> utility] was preserved.
dutch.ini:
+
<syntaxhighlight lang="text" inline>rstconv</syntaxhighlight> can still convert <syntaxhighlight lang="text" inline>.rst</syntaxhighlight> files.
[sysutils]
+
<syntaxhighlight lang="text">
SErrInvalidDateTime="%S" is geen geldige datum/tijd aanduiding.
+
# hash value = 20229140
 +
helloworld.hello='Hello world'
 +
</syntaxhighlight>
  
This would allow reuse of various files.
+
== see also ==
 +
* [[Step-by-step instructions for creating multi-language applications]]
  
[[Category:Localization]]
+
[[Category: Tutorials]]

Latest revision as of 18:10, 25 January 2022

English (en) español (es) Bahasa Indonesia (id) русский (ru)

Resource strings provide a mechanism to internationalize (and to some degree localize) your application.

rationale

Resource strings are constants with an extra level of indirection. The default value (as written in your source code) is stored in the executable program. That means it is always available, even if loading the translation fails.

common steps

The first step is to use resource strings for every string that potentially needs translation. In this tutorial we want to internationalize our Hello World program:

program helloWorld(input, output, stdErr);
resourceString
	hello = 'Hello world';
begin
	writeLn(hello);
end.

The FPC creates a file ending in .rsj (resource string JSON) for every compiled module containing a resourceString section. For the above program it looks like this (pretty-format added):

{
  "version": 1,
  "strings": [
    {
      "hash": 20229140,
      "name": "helloworld.hello",
      "sourcebytes": [72, 101, 108, 108, 111, 32, 119, 111, 114, 108, 100],
      "value": "Hello world"
    }
  ]
}

This intermediate file format was created, so the compiler does not need any knowledge of the utilized translation tool. It just needs to create .rsj files.

The following steps differ on the used internationalization mechanism. As of 2021 the FPC only supports GNU gettext entirely. The reason gettext was chosen is that it is more or less standard on Unix-like systems, like Linux or FreeBSD.

GNU get text

limitations

As mentioned, GNU gettext is widely available on Unix-like systems. However, gettext is horribly inefficient. Also, gettext is context insensitive, that means it operates on the string itself: Sometimes the same word/sentence must be translated differently according to the context, but this is not possible.

procedure

  1. Convert the .rsj file into GNU gettext’s portable object format:
    rstconv -f po -i helloWorld.rsj -o helloWorld.po
    
  2. Create a working copy of the generated file, e. g.
    cp helloWorld.po helloWorld-de.po
    
    and translate it with your favorite translation tool or simply with a text editor.
    #: helloworld:hello
    msgid "Hello world"
    msgstr "Hallo Welt"
    
  3. Convert the PO file to an MO (machine object) file using gettext’s utility msgfmt:
    msgfmt -o helloWorld-de.mo helloWorld-de.po
    
  4. Integrate gettext in your program:
    program helloWorld(input, output, stdErr);
    uses
    	gettext;
    resourceString
    	hello = 'Hello world';
    begin
    	translateResourceStrings('helloWorld-%s.mo');
    	writeLn(hello);
    end.
    
    The %s is substituted with the locale’s name. Examining an strace you will see, that for example first helloWorld‑de.mo is attempted to open. If it does not exist, it tries helloWorld‑de_DE.mo. If neither file exists, the resource string’s default values as specified in your source code are used.
  5. Run the program and celebrate your achievement:
    LANG='de_DE.UTF-8' ./helloWorld
    

For more details see the gettext unit documentation.

resource DLLs

The FPC’s standard RTL does not yet contain portable functions for loading texts from resource DLLs containing translations (as Delphi does), but the first step would be:

  1. Convert the .rsj file into a resource compiler script:
    rstconv -f rc -i helloWorld.rsj -o helloWorld.rc
    

message files

The FPC currently does not support this, but the first step would be:

  1. Convert the .rsj file in an IBM OS/2 message file:
    rstconv -f msg -c HWX -i helloWorld.rsj -o helloWorld.msg
    
    The component identifier indicated by ‑c may be any three upper-case ASCII letters, but it should be unique within your project.

alternatives

To implement another mechanism, 3 things are needed:

  1. Update rstconv so it can output another format.
  2. Tools to manipulate the other format.
  3. Implement a unit that loads the other format at run-time.

Routines to directly manipulate a program’s resource string tables, i. e. the extra level of indirection for resource strings, were available in the objpas unit until revision 31687.

history

Formerly, the FPC created text files ending in .rst. While changing to the JSON format, the name of the rstconv utility was preserved. rstconv can still convert .rst files.

# hash value = 20229140
helloworld.hello='Hello world'

see also