This is a program of moprhological analysis (Russian, German, and English languages).
This program is distributed under the Library GNU Public Licence, which is in the file
COPYING.
This program was written by Andrey Putrin, Alexey Sokirko.
The project started in Moscow in Dialing
Company (Russian and English language). The German part was created
at Berlin-Brandenburg Academy of Sciences and Humanities in Berlin (the project DWDS).
The Russian lexicon is based upon Zaliznyak's Dictionary .
The German lexicon is based upon Morphy system (http://www-psycho.uni-paderborn.de/lezius/).
The English lexicon is based upon Wordnet.
The project uses a regular expression library "PCRE" (Perl Compatible Regular Expressions).
We test compilation only with version 6.4. Other versions were not tested.
One should download this version from the official site and install it
to the default place. If you do not want to install it or you do not have enough
rights to do it, then you should create two environment variables:
1. RML_PCRE_LIB, that points to PCRE library directory, where
libpcre.a and libpcrecpp.a should be located, for example:
export RML_PCRE_LIB=~/RML/contrib/pcre-6.4/.libs
2 RML_PCRE_INCLUDE, that points to PCRE include catalog,
where "pcrecpp.h" is located, for example
export RML_PCRE_INCLUDE=~/RML/contrib/pcre-6.4
The system has been developed under Windows 2000 (MS VS 6.0), but
has also been compiled and run under Linux(GCC). It should work with
minor changes on other systems.
Website of DDC: www.aot.ru, https://sf.net/projects/morph-lexicon/
I compiled all sources with gcc 3.2. Lower versions are not supported.
Contents of the this source archive
1. The main morphological library (Source/LemmatizerLib).
2. Library for grammatical codes (Source/AgrgamtabLib).
3. Test morphological program (Source/TestLem)..
4. Library for working with text version of the dictionaries (Source/MorphWizardLib).
5. Generator of morphological prediction base (Source/GenPredIdx).
6. Generator of binary format of the dictionaries (Source/MorphGen).
=================================================
====== Installation =====
=================================================
Unpacking
* Create a catalog and register a system variable RML, which points
to this catalog:
mkdir /home/sokirko/RML
export RML=/home/sokirko/RML
* Put "lemmatizer.tar.gz", "???-src-morph.tar.gz"
to this catalog, "???" can be "rus", "ger" or "eng"
according to what you have downloaded. Unpack it
tar xfz lemmatizer.tar.gz
tar xfz ???-src-morph.tar.gz
Compiling morphology
0. Do not forget to set RML_PCRE (see above)
1. cd $RML
2. ./compile_morph.sh
This step should create all libraries and a test program $RML\Bin\TestLem.
Building Morphological Dictionary
1. cd $RML
2. ./generate_morph_bin.sh <lang>
where <lang> can be Russian, German according to the dictionary
yo have downloaded.
The script should terminate with message "Everything is OK".
You can test the morphology
$RML\Bin\TestLem <lang>
If something goes wrong, write me to sokirko@yandex.ru.
======================================================
========== MRD-file ============
======================================================
This section describes the format of a mrd-file. Mrd-file is a text
file which contains one morphological dictionary for one natural language.
MRD is an abbreviation of "morphological dictionary".
The usual place for this file is
$RML/Dicts/SrcMorph/xxxSrc/morphs.mrd,
where xxx can be "Eng", "Rus" or "Ger" depending on the language.
The encoding of the file depends also upon the language:
* Russian - Windows 1251
* German - Windows 1252
* English - ASCII
Gramtab-files
A mrd-file refers to a gramtab-file, which is
language-dependent and which contains all possible full morphological
patterns for the words. One line in a gramtab-file looks like as follows:
<ancode> <unused_number> <part_of_speech> <grammems>
An ancode is an ID, which consists of two letters and which uniquely
identifies a morphological pattern. A morphological pattern consists of
<part_of_speech> and <grammems>. For example, here is a line from the English
gramtab:
te 1 VBE prsa,pl
Here "te" is an ancode, "VBE" is a part of speech, "prsa,pl" are grammems,
"1" is the obsolete unused number.
In mrd-files we use ancodes to refer to a morphological pattern.
Here is the list of all gramtab-files:
* Russian - $Rml/Dicts/Morph/rgramtab.tab
* German - $Rml/Dicts/Morph/ggramtab.tab
* English - $Rml/Dicts/Morph/egramtab.tab
Common information
All words in a mrd-file are written in uppercase.
One mrd-file consists of the following sections:
1. Section of flexion and prefix models;
2. Section of accentual models;
3. Section of user sessions;
4. Section of prefix sets;
5. Section of lemmas.
Each section is a set of records, one per line. The number of all records
of the section is written in the very beginning of the section at
a separate line. For example, here is a possible variant
of the section of user sessions:
1
alex;17:10, 13 October 2003;17:12, 13 October 2003
"1" means that this section contains only one record, which is written
on the next line, thus this section contains only two lines.
Section of possible flexion and prefix models
Each record of this section is a list of items. Each item
describes how one word form in a paradigm should be built. The whole list
describes the whole paradigm (a set of word forms with morphological patterns).
The format of one item is the following:
%<flexion>*<ancode>
or %<flexion>*<ancode>*<prefix>
where
<flexion> is a flexion (a string, which should be added to right of the word base)
<prefix> is a prefix (a string, which should be added to left of the word base)
<ancode> is an ancode.
Let us consider an example of an English flexion and prefix model:
%F*na%VES*nb
Here we have two items:
1. <flexion> = F; <ancode> = na
2. <flexion> = VES; <ancode> = nb
In order to decipher ancodes we should go the English gramtab-file.
There we can find the following lines:
na NOUN narr,sg
nb NOUN narr,pl
If base "lea" would be ascribed to this model, then its paradigm
would be the following:
leaf NOUN narr,sg
leaves NOUN narr,pl
It is important, that each word of a morphological dictionary
should contain a reference to a line in this section.
Section of possible accentual models
Each record of this section is a comma-delimited list of numbers, where
each number is an index of a stressed vowel of a word form(counting
from the end). The whole list contains a position for each word
form in the paradigm.
If an item of an accentual model of word is equal to 255, then it
is undefined, and it means that this word form is unstressed.
Each word in the dictionary should have a reference to
an accentual model, even though this model can consist only of empty items.
For one word, the number and the order of items in the accentual model
should be equal to the number and the order of items in the flexion and
prefix model. For example we can ascribe to word "leaf" with the paradigm
leaf NOUN narr,sg
leaves NOUN narr,pl
the following accentual model:
2,3
It produces the following accented paradigm:
le'af NOUN narr,sg
le'aves NOUN narr,pl
Section of user section
This is a system section, which contains information about user edit
sessions.
Section of prefix sets
Each record of this section is a comma-delimited list of strings, where
each string is a prefix, which can be prefixed to the whole word. If a prefix
set is ascribed to a word, it means, that the words with these prefixes
can also exist in the language. For example, if "leaf" has
the prefix set "anti,contra", it follows the existence of words "antileaf",
"contraleaf".
A flexion and prefix model can contain
also a reference to a prefix, but this prefix is for
one separate word form, while a prefix set is ascribed to the whole word
paradigm.
Section of lemmas
A record of this section is a space-separated tuple of the following format:
<base> <flex_model_no> <accent_model_no> <session_no> <type_ancode> <prefix_set_no>
where
<base> is a base (a constant part of a word in its paradigm)
<flex_model_no> is an index of a flexion and prefix model
<accent_model_no> is an index of an accentual model
<session_no> is an index of the session, by which the last user edited this word
<type_ancode> is ancode, which is ascribed to the whole word
(intended: the common part of grammems in the paradigm)
"-" if it is undefined
<prefix_set_no> is an index of a prefix set, or "-" if it is undefined