spelling

From Free Pascal wiki
Revision as of 10:37, 20 February 2018 by Dbannon (talk | contribs) (→‎Linux)
Jump to navigationJump to search

Using hunspell with Lazarus

This page is under development as of Feb, 2018

This page is about using the hunspell library with Lazarus. It outlines a model that does, sort of, work. You will almost certainly need to make some changes for your specific purposes but hopefully, this page will give you a good start.

Firstly, in the forum, there are several references to some code that will work with the hunspell library. The hunspell.pas unit is based heavily on these blocks of code. Most have no license information and an assumption is being made that they are "common knowledge" and thus free of any constraint. I have added a bit that manages the problem of finding the library and dictionary files. And established a reasonable interface.

Further, user rvk from the forum has build a windows 64 bit DLL as there seemed little alternative for Windows users.


About Hunspell

Hunspell is an active open source project, distributed under the Mozilla Public License. The hunspell library is used in products like Libra Office, Open Office and Firefox. Its can be made work on Windows, Linux and Mac (and probably heaps of other platforms). See the platform Specific pages below. Dictionaries for Hunspell are readily available and may already be installed on many machines. Even if you cannot access another application's library, you can use its dictionary.

Dictionaries for Hunspell come as a *.dic and *.aff file pair. For example, the Australian dictionary consists of en_AU.dic and en_AU.aff . The 'en' indicates its english and the 'AU' says its specifically for Australia. As an English speaker, I note that the en_US dictionaries always seem to be installed and I add the Australian ones. I don't know how widespread this pattern is in non English speaking systems.

Platform Specific

Linux

Many Linux distributions will have Hunspell installed by default along with the appropriate language dictionaries. If not, its probably just a case of using the distribution's package manager. If all else fails, grab the source from the hunspell github site and build it yourself. Linux users are like that.

To see if you do in fact have a hunspell library installed, try this command - ldconfig -p | grep hunspell. Similarly, you can probably find some dictionaries with ls -l /usr/share/hunspell. If that does not work, try find /usr -name *.aff, it will take a bit longer ..

Windows

Installing the hunspell library on windows is more of an issue. There is apparently no pre compiled 'kit' available and most Windows apps that use Hunspell appear to bind it staticly thus no hunspell.dll left lying around for you to use. But just to be sure, try searching for *hunspell*.dll. The Hunspell github site lists a recipe to build one but it involves installing MSYS2 and is quite involved. The resulting DLL also needs a couple of gcc DLLs along as well.

Fortunately user rvk on the Lazarus Forum build us a nice statically (ie stand alone) linked DLL using Microsoft Visual Studio Community 2015. As such, you can use and distribute this DLL with your programme subject to the Mozilla Public License.

Mac

The author's Mac appears to have had the Hunspell library installed when Sierra was installed. But maybe, just maybe, it came along with Firefox. I'd like some feed back ....

To see if you aleady have a hunspell library installed, try this command - find / 2>&1 | grep "\hunspell", it will run for some time, depending on how many files on your system. It will likely find several files including some in your XCode directory. However, end users will probably not have XCode installed. The one particularly interesting file to me was /usr/lib/libhunspell-1.2.dylib. Version 1.2 is a bit older than elsewhere but it worked fine.

If you don't find a usable library, I suggest you install one using brew, see link below.

Next issue issue is you will need some dictionaries. Similar command, find / 2>&1 | grep "\.aff", again, slow, it searches your whole disk. I found /Applications/Firefox.app/Contents/Resources/dictionaries/en-US.aff. And a quick 'ls' assured me that there was a matching 'en-US.dic' files so all good.

A more experienced Mac user might like to suggest better search strategies. Please !

The Hunspell Unit

Usage

For this demo, you'll need a form with two TMemos, Memo1 and MemoMsg. A button, ButtonSpell, a Tlistbox, Listbox1.

First you must create the hunspell object and see if it found a library its own way, here is an example in the FormCreate() method ....

uses hunspell;
var
    Form1: TForm1;
    Sp: THunspell;
    DictPath : AnsiString;              

procedure TForm1.FormCreate(Sender: TObject);
begin
    SetDefaultDicPath();
    Sp := THunspell.Create();
    if Sp.ErrorMessage = '' then begin
        MemoMsg.append('Library Loaded =' + Sp.LibraryFullName);
        ButtonSpell.enabled := CheckForDict();
    end else
        MemoMsg.append(SP.ErrorMessage);
end;

In this example, we are writing some status message to MemoMsg, its an easy way to see whats happening. The ButtonSpell is NOT enabled until the dictionaries have been set. Wait for it ....

We now need two methods, one reads a nominated directory looking for any likely dictionary files, the other manages the decisions. If we find just one dictionary set, use it, if we find none, complain. But if we find several, and thats most likely, we have to ask the user which dictionary (ie language) they wish to use.

function TForm1.FindDictionary(const Dict : TStrings; const DPath : AnsiString) : boolean;
var
    Info : TSearchRec;
begin
    Dict.Clear;
    if FindFirst(AppendPathDelim(DPath) + '*.dic', faAnyFile and faDirectory, Info)=0 then begin
        repeat
            Dict.Add(Info.Name);
        until FindNext(Info) <> 0;
    end;
    FindClose(Info);
    Result := Dict.Count >= 1;
end;

function TForm1.CheckForDict() : boolean;
begin
    Result := False;
    EditDictPath.Caption := DictPathAlt;
    if not FindDictionary(ListBox1.Items, DictPath) then
        MemoMsg.Append('ERROR - no dictionaries found in ' + DictPath);
    if ListBox1.Items.Count = 1 then begin                   // Exactly one returned.
        if not Sp.SetDictionary(AppendPathDelim(DictPath) + ListBox1.Items.Strings[0]) then
            MemoMsg.Append('ERROR ' + SP.ErrorMessage)
        else
            MemoMsg.Append('Dictionary set to ' + DictPath + ListBox1.Items.Strings[0]);
    end;
    Result := SP.GoodToGo;   // only true if count was exactly one or FindDict failed and nothing changed
end;

Ah, you say, but where are we looking for the dictionaries ? Sadly, I don't have a good solution for that. Here is where I found mine -

procedure TForm1.SetDefaultDicPath();
begin
    {$ifdef LINUX}
    DictPath := '/usr/share/hunspell/';
    {$ENDIF}
    {$ifdef WINDOWS}
    DictPath := ExtractFilePath(Application.ExeName);
    //DictPath := 'C:\Program Files\LibreOffice 5\share\extensions\dict-en\';
    {$ENDIF}
    {$ifdef DARWIN}
    DictPath := '/Applications/Firefox.app/Contents/Resources/dictionaries/';
    //DictPathAlt := ExtractFilePath(Application.ExeName);
    {$endif}
end;

Maybe, if other users contribute where they found usable hunspell dictionaries, we can build a list for each platform. Or just take the easy way out and tell user to get some dictionaries and put them into the application directory on Windows and Mac. Your thoughts very welcome....

So far, if there is exactly one dictionary set in the indicated directory, all good. But what if there are several ? Our list box has a list of them, if the user double clicks one, they trigger this method -

procedure TForm1.ListBox1DblClick(Sender: TObject);
begin
    if ListBox1.ItemIndex > -1 then
        ButtonSpell.enabled := Sp.SetDictionary( AppendPathDelim(DictPath) + ListBox1.Items.Strings[ListBox1.ItemIndex]);
    if SP.ErrorMessage = '' then begin
        MemoMsg.Append('Good To Go =' + booltostr(Sp.GoodToGo, True));
        MemoMsg.Append('Dictionary set to ' + AppendPathDelim(DictPath) + ListBox1.Items.Strings[ListBox1.ItemIndex]);
    end else
        MemoMsg.append('ERROR ' + SP.ErrorMessage);
end;

Assuming we now have everything "good to go" we can press the Spell button and trigger this -

procedure TForm1.ButtonSpellClick(Sender: TObject);
begin
    if not Sp.Spell(Edit1.text) then begin
        Memo1.Lines.BeginUpdate;
        Sp.Suggest('badspeller', Memo1.lines);
        Memo1.Lines.EndUpdate;
    end else
        Memo1.Lines.Clear;
end;

Memo1 now contains some suggestions for better ways to spell badspeller !

Important ! Don't forget to free up our hunspeller object, memory leaks are evil !

procedure TForm1.FormDestroy(Sender: TObject);
begin
    Sp.free;
    Sp := nil;
end;

This module will do considerably more but its presented here in its most stripped down form for readability.

A note to people new to Lazarus, methods with "(Sender: TObject)" shown above cannot just be pasted into your source, use the Form's Object Inspector to create the events first, then paste my sample code into the method.

Actual Code

(sorry, it really is too long to paste into and out of a wiki page but that seems my only option)

{$MODE objfpc}{$H+}
unit hunspell;

{   Hunspell interface.
    Based on code that seems to appear in lots of places in the Lazarus Forum
    and elsewhere.

    With additions and corrections by dbannon to make it a little easier to use.

    As such, its assumed to be free to use by anyone for any purpose.
}

{   A Unit to connect to the hunspell library and check some spelling.
    First, create the class, it will try and find a library to load.
    Check ErrorMessage.
    Then call SetDictionary(), with a full filename of the dictionary to use.
    If GoodToGo is true, you can call Spell() and Suggests()
    otherwise, look in ErrorString for what went wrong.

    Look in FindLibrary() for default locations of Library.
}


interface
uses Classes, dynlibs;

type
  THunspell_create = function(aff_file: PChar; dict_file: PChar): Pointer; cdecl;
  THunspell_destroy = procedure(spell: Pointer); cdecl;
  THunspell_spell = function(spell: Pointer; word: PChar): Boolean; cdecl;
  THunspell_suggest = function(spell: Pointer; out slst: PPChar; word: PChar): Integer; cdecl;
  THunspell_analyze = function(spell: Pointer; var slst: PPChar; word: PChar): Integer; cdecl;
  THunspell_stem = function(spell: Pointer; var slst: PPChar; word: PChar): Integer; cdecl;
  THunspell_free_list = procedure(spell: Pointer; var slst: PPChar; n: integer); cdecl;
  THunspell_get_dic_encoding = function(spell: Pointer): PChar; cdecl;
  THunspell_add = function(spell: Pointer; word: PChar): Integer; cdecl;
  THunspell_remove = function(spell: Pointer; word: PChar): Integer; cdecl;

   { THunspell }

  THunspell = class
  private
    Speller: Pointer;
        { Loads indicated library, returns False and sets ErrorMessage if something wrong }
    function LoadHunspellLibrary(LibraryName: AnsiString): Boolean;
  public
    	    { set to True if speller is ready to accept requests }
    GoodToGo : boolean;
    	    { empty if OK, contains an error message if something goes wrong }
    ErrorMessage : ANSIString;
            { Will have a full name to library if correctly loaded at create }
    LibraryFullName : string;
            { Will have a "first guess" as to where dictionaries are, poke another name in
            and call FindDictionary() if default did not work }
    constructor Create();
    destructor Destroy; override;
            { Returns True if word spelt correctly }
    function Spell(Word: string): boolean;
            { Returns with List full of suggestions how to spell Word }
    procedure Suggest(Word: string; List: TStrings);
            { untested }
    procedure Add(Word: string);
            { untested }
    procedure Remove(Word: string);
            { returns a full library name or '' if it cannot find anything suitable }
    function FindLibrary(out FullName : AnsiString) : boolean;
            { returns true if it successfully set the indicated dictionary }
    function SetDictionary(const FullDictName: string) : boolean;
    function SetNewLibrary(const LibName : string) : boolean;
  end;

var Hunspell_create: THunspell_create;
var Hunspell_destroy: THunspell_destroy;
var Hunspell_spell: Thunspell_spell;
var Hunspell_suggest: Thunspell_suggest;
var Hunspell_analyze: Thunspell_analyze;
var Hunspell_stem: Thunspell_stem;
var Hunspell_get_dic_encoding: Thunspell_get_dic_encoding;
var Hunspell_add: THunspell_add;
var Hunspell_free_list: THunspell_free_list;
var Hunspell_remove: THunspell_remove;

var HunLibLoaded: Boolean = False;
var HunLibHandle: THandle;

implementation

uses LazUTF8, SysUtils, {$ifdef linux}Process,{$endif} LazFileUtils, Forms;
// Forms needed so we can call Application.~

{ THunspell }

function THunspell.LoadHunspellLibrary(libraryName: Ansistring): Boolean;
begin
    Result := false;
    HunLibHandle := LoadLibrary(PAnsiChar(libraryName));
    if HunLibHandle = NilHandle then
        ErrorMessage := 'Failed to load library ' + libraryName
    else begin
        Result := True;
        Hunspell_create := THunspell_create(GetProcAddress(HunLibHandle, 'Hunspell_create'));
        if not Assigned(Hunspell_create) then Result := False;
        Hunspell_destroy := Thunspell_destroy(GetProcAddress(HunLibHandle, 'Hunspell_destroy'));
        if not Assigned(Hunspell_destroy) then Result := False;
        Hunspell_spell := THunspell_spell(GetProcAddress(HunLibHandle, 'Hunspell_spell'));
        if not Assigned(Hunspell_spell) then Result := False;
        Hunspell_suggest := THunspell_suggest(GetProcAddress(HunLibHandle, 'Hunspell_suggest'));
        if not Assigned(Hunspell_suggest) then Result := False;
        Hunspell_analyze := THunspell_analyze(GetProcAddress(HunLibHandle, 'Hunspell_analyze'));  // not used here
        if not Assigned(Hunspell_analyze) then Result := False;
        Hunspell_stem := THunspell_stem(GetProcAddress(HunLibHandle, 'Hunspell_stem'));           // not used here
        if not Assigned(Hunspell_stem) then Result := False;
        Hunspell_get_dic_encoding := THunspell_get_dic_encoding(GetProcAddress(HunLibHandle, 'Hunspell_get_dic_encoding'));   // not used here
        if not Assigned(Hunspell_get_dic_encoding) then Result := False;
        Hunspell_free_list := THunspell_free_list(GetProcAddress(HunLibHandle, 'Hunspell_free_list'));
        if not Assigned(Hunspell_free_list) then Result := False;
        Hunspell_add := THunspell_add(GetProcAddress(HunLibHandle, 'Hunspell_add'));
        if not Assigned(Hunspell_add) then Result := False;
        Hunspell_remove := THunspell_remove(GetProcAddress(HunLibHandle, 'Hunspell_remove'));
        if not Assigned(Hunspell_remove) then Result := False;
        HunLibLoaded := Result;
    end;
    if not Result then ErrorMessage := 'Failed to find functions in ' + LibraryName;
end;

constructor THunspell.Create();
begin
    ErrorMessage := '';
    if Not FindLibrary(LibraryFullName) then begin
        ErrorMessage := 'Cannot find Hunspell library';
        exit();
    end;
    LoadHunspellLibrary(LibraryFullName);    // will flag any errors it finds
    Speller := nil;           // we are not GoodToGo yet, need a dictionary ....
end;

destructor THunspell.Destroy;
begin
    if (HunLibHandle <> 0) and HunLibLoaded then begin
        if Speller<>nil then hunspell_destroy(Speller);
        Speller:=nil;
        if HunLibHandle <> 0 then FreeLibrary(HunLibHandle);
        HunLibLoaded := false;
    end;
    inherited Destroy;
end;

function THunspell.Spell(Word: string): boolean;
begin
    Result := hunspell_spell(Speller, PChar(Word))
end;

procedure THunspell.Suggest(Word: string; List: TStrings);
var i, len: Integer;
	SugList, Words: PPChar;
begin
    List.clear;
    try
        len := hunspell_suggest(Speller, SugList, PChar(Word));
        Words := SugList;
        for i := 1 to len do begin
            List.Add(Words^);
            Inc(PtrInt(Words), sizeOf(Pointer));
        end;
    finally
        Hunspell_free_list(Speller, SugList, len);
    end;
end;

procedure THunspell.Add(Word: string);
begin
    Hunspell_add(Speller, Pchar(Word));
end;

procedure THunspell.Remove(Word: string);
begin
    Hunspell_remove(Speller, Pchar(Word));
end;

function THunspell.FindLibrary(out FullName : ANSIString):boolean;
var
    {$ifdef LINUX} I : integer = 1; {$endif}
    Info : TSearchRec;
    Mask : ANSIString;
begin
    Result := False;
    {$IFDEF LINUX}
    // Assumes ldconfig always returns same format, better than searching several dirs
    if RunCommand('/bin/bash',['-c','ldconfig -p | grep hunspell'], FullName) then begin
        while UTF8Pos(' ', FullName, I) <> 0 do inc(I);
        if I=1 then exit();
        UTF8Delete(FullName, 1, I-1);
        UTF8Delete(FullName, UTF8Pos(#10, FullName, 1), 1);
        Result := True;
    end;
    exit();
    {$ENDIF}
    {$IFDEF WINDOWS}		// Look for a dll in application home dir.
    Mask := '*hunspell*.dll';
    FullName := ExtractFilePath(Application.ExeName);
    {$endif}
    {$ifdef DARWIN}
    Mask := 'libhunspell*';
    FullName := '/usr/lib/';
    {$endif}
    if FindFirst(FullName + Mask, faAnyFile and faDirectory, Info)=0 then begin
        FullName := FullName + Info.name;
        Result := True;
    end;
    FindClose(Info);
end;

function THunspell.SetDictionary(const FullDictName: string) : boolean;
var
    FullAff : string;
begin
    FullAff := FullDictName;
    UTF8Delete(FullAff, UTF8Length(FullAff) - 2, 3);
    FullAff := FullAff + 'aff';
    if Speller <> Nil then
        hunspell_destroy(Speller);
    Speller := hunspell_create(PChar(FullAff), PChar(FullDictName));
    GoodToGo := Speller <> Nil;
    if not GoodToGo then
        ErrorMessage := 'Failed to set Dictionary ' + FullDictName;
    Result := GoodToGo;
end;

function THunspell.SetNewLibrary(const LibName: string): boolean;
begin
    LibraryFullName := LibName;
    Result := LoadHunspellLibrary(LibraryFullName);
end;

end.

Further Reading and Links

https://github.com/hunspell/hunspell

https://github.com/Homebrew - probably sensible way to get hunspell on your mac if its not already there.

The 64bit Windows DLL will appear here on the next release of tomboy-ng (thats because putting it in the tomboy-ng repository is convenient).