Code Monkey home page Code Monkey logo

lang-esu's Introduction

Finite State Morphological Analyzer for Central Alaskan Yup'ik

Maturity Lemma count GitHub issues License Doc Build Status CI/CD Build Status

Download nightly / CI/CD installation packages for testing (contains the core zhfst file(s)):

Windows MacOS Mobile

NB!! Note that the nightly / CI/CD installation packages are not tested for language quality, and might contain regressions and errors.

Central Alaskan Yup'ik (CAY) (ISO 639-3: esu) is a member of the Inuit-Yupik-Unangan (IYU) (also known as Eskimo-Aleut) language family. It is spoken in south-west Alaska among the Yukon-Kuskokwim Delta and Bristol Bay areas.

This CAY morphological analyzer/generator can analyze (break the word into its individual parts) and generate (build a word from its individual parts) Yup'ik words. The underlying form is composed of morphemes represented by the analysis of Jacobson's (1984/1995/2012) dictionary and grammar book. It is build using the open source HFST tools. This project was developed with hfst 3.15.2.

Download and test speller files

The speller files downloadable at the top of this page (the *.bhfst files) can be used with divvunspell, to test their performance. These files are the exact same ones as installed on users' computers and mobile phones. Desktop and mobile speller files differ from each other in the error model and should be tested separately — thus also two different downloads.

Documentation

Documentation can be found at:

FST Stack

  1. lexc - esu.lexc - lexicon and morphotactics
  2. xfst - esu.lexc.xfst - orthography → phonemes, morphologically conditioned allomorphy
  3. twolc - esu.twol - morphophonology, phonologically conditioned allomorphy
  4. twolc - esu.stress.twol - prosodic stress marking on vowels used for prosodic adjustments
  5. xfst - esu.twol.xfst - prosodic adjustments, phonemes → orthography

Installation

  1. Install dependency: HFST tools. Tested with hfst 3.15.2.
    Note: MacOS users may have an easier time installing HFST with this Homebrew tap.

  2. Run make to run the Makefile and compile the FST stack

Output Files

  • esu.ana.hfstol: Morphological analyser

    • Input: calinrituq
    • Output: cali-–nrite[V→V][V][Intr][Ind][S_3Sg]
  • esu.gen.hfstol: Morphological generator

    • Input: cali-–nrite[V→V][V][Intr][Ind][S_3Sg]
    • Output: calinrituq
  • esu.seg.hfstol: Morphological segmenter

    • Input: calinrituq
    • Output: cali>nrit>uq

Usage

In the unix terminal, run this command to use the HFST lookup program:

echo "wordToInput" | hfst-optimized-lookup hfstName

where wordToInput is the Yup'ik word to analyze/generate and the hfstName is either: esu.ana.hfstol | esu.gen.hfstol | esu.seg.hfstol.

License

This Central Alaskan Yup'ik FST morphological analyzer/generator and the associated source code is available under the GNU Affero General Public License (GNU AGPL v.3):

Copyright (C) 2020 Lonny Alaskuk Strunk

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/.

Citing

If you use language data from more than one GiellaLT language, consider citing our LREC 2022 article on whole infra:

Linda Wiechetek, Katri Hiovain-Asikainen, Inga Lill Sigga Mikkelsen, Sjur Moshagen, Flammie Pirinen, Trond Trosterud, and Børre Gaup. 2022. Unmasking the Myth of Effortless Big Data - Making an Open Source Multi-lingual Infrastructure and Building Language Resources from Scratch. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 1167–1177, Marseille, France. European Language Resources Association.

If you use bibtex, following is as it is on ACL anthology:

@inproceedings{wiechetek-etal-2022-unmasking,
    title = "Unmasking the Myth of Effortless Big Data - Making an Open Source
    Multi-lingual Infrastructure and Building Language Resources from Scratch",
    author = "Wiechetek, Linda  and
      Hiovain-Asikainen, Katri  and
      Mikkelsen, Inga Lill Sigga  and
      Moshagen, Sjur  and
      Pirinen, Flammie  and
      Trosterud, Trond  and
      Gaup, B{\o}rre",
    booktitle = "Proceedings of the Thirteenth Language Resources and Evaluation
    Conference",
    month = jun,
    year = "2022",
    address = "Marseille, France",
    publisher = "European Language Resources Association",
    url = "https://aclanthology.org/2022.lrec-1.125",
    pages = "1167--1177"
}

lang-esu's People

Contributors

albbas avatar bbqsrc avatar flammie avatar ftyers avatar lonnystrunk avatar rueter avatar snomos avatar trondtr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

ianturner3

lang-esu's Issues

Use `main` branch? Common dir layout?

We have switched branches/branch names from master to main. I have merged your commit 83a5a2f with main. It would be nice if you could continue your work in the main branch 🙂

To avoid heavy branch merges with high risk of conflicts in the future, I have locked (but not removed) the master branch. I hope this is ok.

Also, I notice you have a different source dir structure. To reap the most of the GiellaLT infrastructure it would be good to follow the general layout. For now I will move your srcV2 dir to within src/fst/, but with a symlink in the old location, so that your build will work as well.

Hm, come to think of it: your Makefile will block the creation of the GiellaLT Makefile, so the GiellaLT build will most likely fail. We need to figure out a way to solve that.

+a/+aar invalid tag

/usr/share/giella-core/scripts/taglist2make_CG_tags_regex.sh disamb-tags.txt > make-disamb-CG-tags.regex
ERROR:
The following tags do not follow the Giella tag conventions
(either +TAG, TAG+ or +TAG+) and can not be converted to the CG
format required for proper tokeniser functionality within the
Giella infrastructure:

+a/+aar
make[4]: *** [Makefile:865: make-disamb-CG-tags.regex] Error 1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.