The current stages for training include:
untrained
training
trained

However, training can be broken down into the headers found in kaldi-helpers/.../run.sh

Build a bridge between the webserver and the kaldi layer

move relevant code from api/model.py into new api/lexicon.py. need to do anything in kaldi (wrappers) dir?

@bp.route("/list", methods=['GET', 'POST'])

Changing tier name to something and save setting should update the state. JSON object {tier: something} is being sent OK, check where it is failing in the server

After the last big refactor, the api has changed significantly. Only the basic commands for running a bare minimum pipeline exist

Errors that happen internally to elpis are currently displayed by the flask debugger. Instead, they should be caught and processed in the response JSON object

Option to export with

• Phone
• Word segmented
• Immediate-time adjacent words grouped into a single utterance
• segmentation by voice activity
• no segmentation (single annotation entire file)

When a current step fails, the menu highlighting is lost. Investigate ways to retain step highlighting that is independent of api status/return data.

@bp.route("/list", methods=['GET', 'POST'])
def list_existing():
kaldi: KaldiInterface = app.config['INTERFACE']
return jsonify({
"status": "ok",
"data": kaldi.list_models()
})

Should use POST where there is change to state, eg POST to /new. Use GET where there is no change to state, eg GET /list.

Check that results are written as Del, Ins, Sub (alphabetical order)

bring KH into the elpis repository,
rename docker image

m = kaldi.new_dataset(request.json['name'])

Include the date/name/success model table on the new trans page so users can identify which is the model they want to use (sometimes name isn't enough, good to see WER)

Send dropped files to the server and pair them together on the front end.

React routes specify the location of main pages on the front end, however, these routes don't exist on the server. For example, if you start at the index (/), then you can access the naming page (/naming), however, if you try and access /naming directly, then you'// get a 404.

The base class currently has a complete load method. The other classes that need their load methods reviewed/coded/re-coded are: KaldiInterface, Dataset, Model, Transcription

as in Kaldi-helpers, add organisation and author headers

some of the end points return files, but on error we still return a json string

currently, it is fixed to 'Phrase' in Dataset.init

have api objects save and load to/from a json file

eg the u̱t-Maʻin data has hyphen noun marker and glottal stop,
the human transcription is provided as da-u de̱'e̱
which the cleaning produces four tokens da u de̱ e̱

(The Gossip King.eaf,T=tx@IbrahimYohanna,B=20535,E=21807)

currently, if you share elpis local repo into the container, previous state isn't read by the gui.. so it looks like its a fresh start, but potential for name clashes of the things. Actually, we also need better handling of name clashes.

Create a KaldiPaths class to keep a local record of kaldi directory structure.

The Dataset, Model and Transcription objects all have common methods. Create a base class to modularize the code (and hopefully reduce chance of a bug occurring)

Currently using bash, with sort -u. This is problematic because the order is significant, so that for example, aa can take precedence over a in the example below:

aa a:
a a

Current script is:

cmd = f"grep -v '^#' < {self.pron_dict.l2s_path} | cut -d' ' -f2 | grep -v '^$' | sort -u"

Should be able to do it with python!

Can we determine the optimal n-gram size by corpus perplexity?

Design GUI and API methods to select Elan data by tier number rather than name.

Tried uploading corpus.txt but the words don't show in the wordlist.

These directories are empty except for python caches. From my understanding, they are artifacts from the old directory structure and all the files they once contained are now located under elpis/wrappers/.

should new_dataset and new_model be similar code structure?

    def new_model(self, mname):
        m = Model(parent_path=self.models_path, name=mname, logger=self.logger)
        models = self.config['models']
        models[mname] = m.hash
        self.config['models'] = models
        return m

Create an end-point that reveals the results of training.
Find the results in the Kaldi File Structure -> kaldi/exp/tri1/decode/<wer_x> where x is the highest number wer_ file in that directory

There are many references in the code that use absolute file paths, this is not good practice and must be fixed

Show an example of l2s on the upload page.

This would be a keyword argument like callback=<function>

There are numerous times variables with the same meaning have appeared with different names. Choose a coding style and conform the code base to it

Can we fix the paths in the scripts rather than doing path replacement eg in
elpis/wrappers/objects/transcription.py _cook_generate_infer_files

Where else does this occur?

Add unit tests for each object (again)