Issues like #37 show that, for scalability and durability, it would be wise to migrate to an ORM (object relational mapper) tool like sqlalchemy for simplicity and scalability. This will remove the need for raw SQL statements which are difficult to interpret and change.

When examining a particular transcription/composition, you might try several different search parameters. It would be useful to store these parameters by saving the logs to a file by an optional argument like save_logs_to_file in create_database.

See: https://stackoverflow.com/questions/6386698/how-to-write-to-a-file-using-the-logging-python-module

Not sure if this is a hash key issue or what, but definitely worth resolving. I use this kind of thing in the Path.score() function and need fast code.

A failure to reference the logger in the _make_paths_table helper function forces the paths table creation in create_database to fail early.

In the current system, if you call something like Decitala('Sama'), you actually get <decitala.Decitala 65_C_Kankala_Sama>. We can't revert to just id numbers due to subtalas. There may be a simple solution to this with Regex.

• should definitely be naming-specific Errors; currently, it spits out some scary stuff when it could instead just say "that name is a problem".
• I think it should accommodate for lowercase issues. I tried to create a Decitala('Kankala_khanda') object and it gave the error (above) because of the lowercase second k. This is an easy fix; just do name.split('_')[-1][0].upper(), or something similar to that.
• I'm not sure if this is an issue, but note the fact that talas with subtalas (e.g. 65) all have the same id num. But this is probably better since they're then all the same type. But this is a problem for histogram plotting. Maybe add a parameter called full_id that returns a string with the subtala index, if required?

Instead of having the confusing logic in the init to FragmentTree that checks for data/rep-type/frag-type, just have a class method for instantiating from frag_type, i.e. the default FragmentTree is made from a list of GeneralFragment/Decitala/GreekFoot objects and you'll have FragmentTree.from_frag_type("decitala"), for example.

This was a lingering issue because the quarter length of grace notes are 0.0 in music21.

Something wrong in the calculation is reversing the signs in create_database.

Rather than having a TimeoutError in the get_pareto_optimal_paths function, it'd probably be cleaner to just have a ceiling value for the number of possibilities (e.g. 10,000).

This is probably because of the ID situation. I should probably rename it and assign it an arbitrary value like 500. Update: although the ID is an issue (see the wiki), I think the actual problem is that the file has an .mxl extension rather than an .xml extension; there's something in the if name: part of the script that needs to be fixed. I added a temporary solution to the database.py file.

Currently, they can just handle paths which doesn't really make sense. It would be better to have,
t = GeneralFragment(fragment=[0.25, 0.25, 0.25], name='testing'

The following parameters were used for a query on Liturgie de Cristal:

create_database(
    db_path = "/Users/lukepoeppel/TISMIR_Messiaen_2021/db/liturgie_3.db",
    filepath = liturgie_path,
    part_num = 3,
    frag_types = ["decitala"],
    allowed_modifications = ["r", "rr", "d", "rd", "rsr"],
    windows = [4, 5, 6, 7, 8, 9],
    filter_found_single_anga_class = True,
    filter_found_sub_fragments = False,
    write_logs_to_file = True,
    verbose = True

It returned the following error:

It could be cool to explore the idea of a (rooted) dseg and reduced-dseg fragment tree (using a dseg rep-type).

The package is getting big enough that it would be useful (just for myself) to have a simple documentation page. Check out a few services like Sphinx.

This function is currently included, but meant to not be touched. Additionally, it's not extendable at all –– it's limited only to two representations of two fragment classes. But as I add more rhythmic datasets (e.g. the Astaara tala dataset), this has to be more malleable.

Maybe the following would work:

• Package includes built-in datasets+classes for the fragment types (e.g. Decitala, GreekFoot).
• If a new dataset is added, update some config file but use the GeneralFragment data type.

Having to use converter.parse on > 130 files each time, for instance, a RatioTree is created seems inefficient. Perhaps it would make more sense to create a decitala.db and prosodic_feet.db instead? The whole package would run much faster.

... but it is being found by get_by_ql_array, so it's odd that it isn't being displayed.

If this is not specified, the file still gets created, but the user has to add the extension themselves.

Need to pick a license before distributing on Pypi.

Currently only retrieves pitch information for individual pitches; but if there are chords, it flattens the data. I'm sure there's an easy fix, it's just a tricky script to read through.

The following tree has a size of 3 which is wrong (it should be 5).

test_tree = FragmentTree(data=[GeneralFragment([0.25, 0.25, 0.25], GeneralFragment([0.5, 1.0, 0.5]))])

Just learned you can enumerate from any start parameter :-). Fix all of those complicate lines in database.py.

I can just use --doctest-modules from pytest for the CI (Travis or Github actions). The file can still stay––useful for debugging––but not needed for CI.

Could potentially speed up by creating set of Decitala.ql_tuple that gets checked for each search. Worst case of x in set() is O(n). May get false-negatives for altered fragments –– could potentially add altered and original?

I think it would be a better idea if, rather than including static databases in the tests folder (which will never change, even if the package changes), we create the DBs in a tempfile and read them on the fly. This would make testing much more effective –– as is evidenced by v0.6.3.

The trees.py module is getting a bit big. Since I may end up implementing a KD tree with NN search, I think it makes more sense to take all of the searching functions (rolling_search and get_by_ql_array) and put them in a search.py module.

If you run a search on [0.5, 0.5, 0.5, 0.75, 0.75, 0.75, 0.25], it'll return Candrakala under a ratio representation of 1.0 –– but this is wrong; it's actually a ratio representation of 0.5.

FragmentTrees instantiated via from_frag_type don't have names –– could be standardized to frag_type_rep_type or something, e.g. decitala_ratio.

Got the following error when trying to create a database:

Write an additional class method that can take in multiple paths of data.

If a frame in rolling search also has a spanner, this should be noted in the database (and in the rolling_search output). The frames with these articulations generally correspond to prosodic feet.

It would be nice if in cases of an RSR modification being detected, we could give more precise details about what was subdivided.

In addition to searching for all matches in a dataset, it would also be nice to be able to explicitly search for a set of fragments. Something like:

rolling_search(
    filepath,
    part_num,
    search_for = [Decitala("Ragavardhana"), GeneralFragment([1.0, 2.0, 3.0], GreekFoot("Trochee")]
)

We get that "something is off" error in calculating the ratio. It's likely not related to the fact that there are rests since the other parts are working (seemingly) great and have rests, too...

The database.create_database the input parameters but it missing the allowed_windows.

This could be a nice feature for analysis. Return for either a single part or the full piece. Coolest idea: a plt.figure with subplots where each subplot is the histogram of tala use for a part. Note: There should probably be two options: either plot all talas found regardless of the overlapping paradigm, or restrict it to a path.

It would be useful to have an additional table created that stores relevant extracted rhythmic information like Valeur Ajoutées and measures that are non-retrogradable.

Create optional lambda function as input parameter (extra_filter) to create_database.

While the filtered Fragments table is normally the table of interest in analysis, for research purposes, it's also useful to have a complete, unfiltered set of extracted fragments.

Now that we are writing the logs to a file, we can add a parameter to create_database of something like parameters_from_logs which reads the logs and sets the parameters accordingly. If anything other than verbose, write_logs_to_file, and db_path are set, raise an Exception.

This would be useful for presentations and such. Just a simple function that takes in a few paths and displays them.

Was accidentally logging a dictionary that didn't exist in trees.py.

Replicate with:

create_database(
	db_path = liturgie_path_db,
	filepath = liturgie_path,
	part_num = 4,
	frag_types = ["decitala"],
	allowed_modifications = ["r", "rr", "d", "rd", "rsr"],
	windows = [4, 5, 6, 7, 8, 9, 10, 11, 12, 13],
	filter_found_single_anga_class = True,
	filter_found_sub_fragments = True,
	write_logs_to_file = True,
	verbose = True
)

Weirdly doesn't seem to replicate if part_num=3...