Code Monkey home page Code Monkey logo

apertium-crh's Introduction

Apertium

Apertium is an open-source rule-based machine translation toolchain and ecosystem. It facilitates the creation of consistent and transparent machine translation systems by relying on deterministic linguistic rules rather than statistical or neural models. Apertium's tools are designed to be language-agnostic and platform-independent, making them suitable for a wide range of languages and applications.

Project Overview

Apertium's framework is based on finite-state transducers, which enable efficient and accurate processing of natural languages. The language data used by Apertium is stored in XML and other human-readable text formats, organized into modular single-language packages and translation pairs. This modularity allows for the reuse of language data across multiple translation systems.

Features

  • Rule-Based Translation: Consistent and understandable translations based on deterministic rules.
  • Finite-State Transducers: Efficient language processing using advanced computational models.
  • Language-Agnostic Tools: Broad applicability across multiple languages.
  • Modular Design: Reusable language packages simplify the development of new translation pairs.

Installation

Apertium provides binaries for several platforms, including Debian, Ubuntu, Fedora, CentOS, OpenSUSE, Windows, and macOS. Both nightly builds and official releases are available. If you are on a supported platform, it is recommended to use the pre-built binaries.

For more information, see the Apertium Installation Guide.

Building from Source

If you need to modify Apertium’s behavior or are on a platform that is not officially supported, follow these steps to build from source.

Requirements

Compiling

$ autoreconf -fvi
$ ./configure
$ make

Usage

Apertium can be used to translate text between supported languages. Assuming the relevant language data (here the Spanish-Catalan translator) has been installed, translation can be achieved with the following command:

$ apertium spa-cat input.txt output.txt

The apertium executable can also use piped streams:

$ echo "La casa es roja." | apertium spa-cat

Language data which has been compiled but not installed can be used with the -d flag:

$ echo "La casa es roja." | apertium -d ./apertium-spa-cat spa-cat

Formats other than plaintext can be specified with the -f flag:

$ apertium -f html spa-cat input.html output.html

Data packages may provide modes besides the main translation mode. Use the -l flag to list them.

$ apertium -l
$ apertium -l -d ./apertium-spa-cat

Additional Tools

This repository also provides the following executables:

Pipeline Modules

  • apertium-extract-caps, apertium-restore-caps: Handle capitalization
  • apertium-pretransfer: Split compound analyses into separate words for processing by apertium-transfer
  • apertium-posttransfer: Clean up repeated spaces
  • apertium-tagger: Perform statistical part-of-speech tagging
  • apertium-transfer, apertium-interchunk, apertium-postchunk: Structural transfer modules (documentation)
  • apertium-wblank-attach, apertium-wblank-detach, apertium-wblank-mode: Handle word-bound blanks

Build Tools

These programs are used in the process of compiling linguistic data packages.

  • apertium-compile-caps: Compile capitalization-handling rules for use by apertium-restore-caps (documentation)
  • apertium-gen-modes: Process the modes.xml file, which specifies what translation and analysis modes a data package provides
  • apertium-preprocess-transfer: Process structural transfer rule files for use by apertium-transfer
  • apertium-validate-acx, apertium-validate-crx, apertium-validate-dictionary, apertium-validate-interchunk, apertium-validate-modes, apertium-validate-postchunk, apertium-validate-tagger, apertium-validate-transfer: Validators for various XML rule formats

Format Handlers

For each supported file format, there is a deformatter named apertium-des[NAME] (e.g. apertium-deshtml) which reads formatted text from standard input and writes Apertium stream format to standard output. There is also a corresponding set of reformatters which do the reverse and are named apertium-re[NAME] (e.g. apertium-rehtml). These programs rarely need to be invoked directly, since they are handled by the apertium executable.

Most of the format handlers are currently deprecated in favor of Transfuse.

License

This project is licensed under the GNU General Public License v2.0. See the COPYING file for details.

For more information, visit Apertium or the Apertium Wiki.

apertium-crh's People

Contributors

ftyers avatar ilnarselimcan avatar jonorthwash avatar manitk avatar memduhg avatar mr-martian avatar sevilaybayatli avatar sushain97 avatar tinodidriksen avatar unhammer avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

manitk

apertium-crh's Issues

Installed modes are missing files

modes.xml includes some modes with install="yes", but the required
files aren't installed.

Some generic suggestions:

  • -lexc and -twol modes probably aren't useful to users

  • -spell modes should depend on --enable-ospell

  • .deps files are never installed, so any modes using them shouldn't be
    installed.

  • Messages for package app-dicts/apertium-crh-9999:

  • Failed to find '/usr/share/apertium/apertium-crh/.deps/crh.twol.hfst' in install image.

  • QA: missing files required for mode crh-twol.

  • Failed to find '/usr/share/apertium/apertium-crh/.deps/crh.LR.lexc.hfst' in install image.

  • QA: missing files required for mode crh-lexc.

  • Failed to find '/usr/share/apertium/apertium-crh/crh.zhfst' in install image.

  • QA: missing files required for mode crh-spell.

  • Failed to find '/usr/share/apertium/apertium-crh/.deps/acceptor.default.hfst' in install image.

  • QA: missing files required for mode crh-tokenise.

Create a seed locale in CLDR

This language is lacking an entry in the Common Locale Data Repository (CLDR). The core
data used for creating a seed locale should ideally be submitted to the CLDR trac
before the 1st April, 2018.

You can use the ticket about Tuvan as a template.

Note that if a two-letter code is available (e.g. av for Avar) then it should be used instead of the three-letter code.

The process is as follows:

  • Provide the characters in the '''Exemplar sets'''
  • Provide the singular and plural forms (for Turkic only two matter)
  • Check the territory information and copy the relevant lines
  • Provide a character--character transliteration scheme (it doesn't have to be perfect).

In addition you can provide a seed locale file based on the Chuvash example.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.