1 How to use this dictionary framework
2 ====================================
4 Each dictionary has its own directory using the ISO639 language code. Eg
5 Afrikaans uses af. In each directory is an aspell, myspell and in some cases
6 an ispell directory - these contain the speller specific information.
27 | | |-- README_af_ZA.txt
30 | | |-- wordlist.nieuwoudt.in
34 aspell - needs word-list-compress which is contained in the aspell package for
35 your system. Other tools needed to package and build are located in the utils
38 myspell - the tools needed are in the utils directory
46 The framework can build ispell but it is disabled by default because we are not
47 completely sure how to build ispell dictionaries.
51 Copyright - details of the copyright holders etc. The actual copyright text
52 is added based on a line in the info.in file.
53 info.in - some basic definitions for the aspell package including copyright
54 holders, license, language name etc. Check the Afrikaans one for a good
55 understanding of its construction. If you need more details then look at the
56 instructions in the latest aspell dictionary build system. ?URL?
57 What can be problematic in this file is the "special" line:
58 special ' **- - -*- 4 -*- 6 -*-
59 It is simple to understand. The above says ' (apostrophe) is permissible at the
60 beginning of a word and the middle. - (dash) is allowed in the middle of a
65 README_lang_REGION.txt - a README containing Copyright info and installation
67 lang_REGION.aff - the affix file. Mostly these are based on ispell affix
68 compression. Follow the instructions ?here? for converting an ispell affix file
69 to myspell. An affix file allows you to compress a wordlist and expand it to
70 the exact same set of words. If you have not developed affix rules for the
71 language then the minimum you need is a SET and TRY line.
73 SET - the character set used by the language. Only ISO8859 character sets can
74 be used in MySpell. Although I think you can define your own internal mapping
75 if your language does not match an ISO charset. (Need to confirm this)
76 TRY - a list of letters in order of frequency. The python script
77 src/wordlist/letter-frequency.py allows you to create a frequency list.
80 MAP - map similar characters eg eêë
81 REP - create REPlacement maps that are useful for mapping common spelling
82 mistakes. eg REP ph f - as in phone.
87 FIXME add more details on creating these.
89 Setting up a new language
90 -------------------------
91 Apart from input files required by each of the different spellcheckers, as
92 listed above, the main requirements are a wordlist and some definitions in the
95 The wordlist is simply a text list of words one per line - currently we store these in UTF-8 to
96 ensure ease of use in the future. Lines that start with a # are treated as
97 comments and removed when the wordlist is processed. The wordlists can be
98 called anything although we name them wordlist.*.in. But as you list them in
99 the Makefile you can name them as you please. We have kept existing wordlists
100 in tact and used separate files for new additions. In English we have grouped
101 similar concepts together eg. bird names, city names, etc. Some languages
102 group words according to parts of speech which may aid later use with advances
103 in grammar checkers or in agglutinative languages that may have rules as to how
104 words may be joined together.
106 The Makefile calls the generic Makefile called utils/Makefile.language. The
107 language Makefile contains a number of definitions such as the name of the
108 language its character set, etc. If you need to understand some of the build
109 process steps then look at the generic Makefile.
111 Add a VERSION file. We default to using the date as spellcheckers are really
112 enhancements and refinements of wordlists, so a newer date should always
113 indicate a better spellchecker.
115 Also add you language to the Makefile in the dict/ directory. Both as a build
116 rule and as a TARGET.
122 make - generate all dictionaries for that language
123 make count - will return simple stats on your wordlist
124 make aspell - create the aspell dictionary (relatively quick)
125 make myspell - create a myspell dictionary (looooong)
126 make clean - cleans up all packaged files
130 All outputs are placed in the respective spellchecker directories.
132 aspell - creates a tarball that would be compiled and installed on the target platform
134 myspell - creates a few outputs.
135 lang_REGION.zip - the basic MySpell spellchecker usable in OpenOffice.org
136 pack-lang_REGION.xip - as above but installable by the offline installer
137 lang-REGION.xpi - the same as lang_REGION.zip but installable in Mozilla
139 Resources - South African
140 -------------------------
142 Common Names of fish - http://www.fishbase.org/search.cfm
144 Robert's Birds list - http://web.uct.ac.za/depts/fitzpatrick/docs/listintro.html
145 http://www.wildlifesafari.info/south_african_birds.htm
146 http://www.wildlifesafari.info/southern_africa_bird_checklist.htm
147 Trees - http://www.wildlifesafari.info/southern_africa_tree_list.html
149 http://www.unep-wcmc.org/index.html?http://sea.unep-wcmc.org/isdb/CITES/Taxonomy/?displaylanguage=eng~main
150 http://www.e-gnu.com/check_005.html
151 http://www.e-gnu.com/check_003.html
152 http://www.e-gnu.com/check_004.html
154 http://www.jse.co.za/listed/companies/la.html
156 http://africanhistory.about.com/cs/southafrica/a/sa_new_name.htm
157 www.sapo.co.za - get downloadable postal codes for names of towns and suburbs