• split "formula" and "information"
• make formula shorter (e.g. by using fractions and abbreviations)

There are different types of silver standard (or incomplete ground truth matches). For example:

• specify which pairs are not duplicate
• specify which pairs are duplicates

Provide support for those.

filtering / sorting / limitting for

• matching solutions (including soft KPIs)
• experiments
• datasets

are managed on the frontend at the moment. When the number of entries gets larger this may lead to problems. We should therefore implement this functionality on the backend side.

For binary comparison, one of the two supplied experiments is regarded as a gold standard. We should make it clear in the UI which one the user wants to use as gold standard.

• provide support for running matching solutions directly from the tool
  • magellan
  • JedAI
  • BigGorilla
  • DeepMatcher
  • list of others
• add "debugging" capabilities for ML matching solutions by implementing XAI methods to show why pairs have (not) been declared as duplicate

Please include a list of my authors :)

The side menu should show the current page's menu item in color primary

We should consider how to use JedAI

• support for their result set format
• other integrations between snowman & jedAI

At the moment datasets, experiments and matching solutions can only be created and deleted. However, the API also supports changing them and this should be reflected in the frontend.

Speed up the metrics route by caching the result (or something else).

• we should come up with a coherent caching strategy we can apply to all routes.

When hovering or clicking on a dataset record in the benchmark section, show all records this one is related to (e.g. all true positives, false positives, false negatives and true negatives)

To reproduce (on branch main): Select magellan as experiment format

When visiting the metrics route directly in debug mode (npm start), the following error is thrown:

@phpfs Do we have any UX concept for that issue?

We use command line parameters in both electron and backend. Therefore, it makes sense to have a complete strategy on how these are implemented.

Additionally:

• accept port and host as command line argument to start backend server
• accept command line parameter to use an in memory database

When inspecting / modifying an experiment or when benchmarking add the possibility to preview the detected duplicates of a dataset as clusters.

Especially necessary for

• uploading files
• some benchmark routes

Implement by e.g. starting a worker thread and asynchronously waiting for it to finish so other routes can be handled in the same time.

• cheap solution: call (and await) setImmediate now and then
• thread library: https://www.npmjs.com/package/workerpool
  keep in mind only one writable connection per SQLite database (maybe spilt up databases more?)

Additional speed up by compiling to WASM?

Synchronize changes of a shared instance to all users. Otherwise if two users access the same system at the same time inconsistencies might occur.

Possible Implementation:

• connect clients and server via websocket
• the server informs all connected clients about all changes (e.g. experiment 5 has been updated). The client can then reload the relevant information

To reproduce (on branch main):

• upload the magellan songs dataset
• the total and uploaded counts are not updated after the dataset file finishes uploading

As we are adding more and more experiment formats the list of experiment formats gets too large to be comfortably usable. We should make it searchable in some way to improve usability.

It should be possible to add information about a matching solution like soft KPIs. The currently preferred way is using a questionnaire

The questionnaire for the soft KPIs should by dynamically rendered from a json definition file

Implement a dropdown (or similar) to select one of the following intersection modes:

• Pairs
• Clusters
• Investigative

Backend implementation, API spec and types will be merged with this PR #55

The SideMenu rerender correctly if the state and the current path changes. But it does not rerender automatically if only the current path changes.

Currently, the tooltip that should explain a metric does not work.

Some times the error message is too large to be completely visible. We should allow the user to expand the error message.

• API spec
• frontend
• backend

See this openAPI generator

As a data scientist I want to have a search field on the metrics viewer page to search for a specific metric.

Sometimes the view needs to be refreshed even if an error is thrown (as in this case):

We should add the following types of tests

• API tests (e.g. check correctness of routes)
• e2e / integration tests (e.g. via Cypress)
• performance tests (check that magellan dataset can be uploaded in less than x min)

Additionally the test coverage of our unit tests should be improved by testing edge cases like

1. upload dataset (no file)
2. upload experiment
3. upload dataset file with too few records and see if error is thrown

The user runs his matching solution on different datasets. He can now upload the different result sets (based on the different datasets) in our tool and see averaged metrics (--> calculate for each result set a metric and then calculate the average).
The user also has the possibility to investigate how the developed algorithm performs on each dataset seperately.

Adding new experiments is pretty easy. Therefore, it makes sense to include a special contribution guide just for this use case.

Should we put this in our docs or a README into the experiment formats folder?

Add button in frontend to "publish" matching solution + its soft KPIs. This should open a new GitHub issue with the matching solution configuration in its description (format of the example matching solutions)

Axios is not really needed. We could probably use the default request library as well.

Describe the bug
The new release includes a new preloaded dataset. When the user already has a snowman version installed and now wants to update this version, he does not see the newly added dataset because the database will not be updated. Up to now he has to delete the whole database in AppData.

Expected behaviour
Snowman should show every newly added dataset. For example, we should restart the database initialization process after installing a new version.

Currently it is only possible to compare 2 experiments. It should now be allowed to compare n experiments against one groundtruth.

This includes:

Frontend:

• visualising via a venn diagram
• save results sets after clicking on specific area in venn diagram

Backend:

• Update /benchmark/experiment-intersection/records and /benchmark/experiment-intersection/count routes to support n experiments
  • Bonus: Speed up those routes by adding in some fancy caching (see #21)
  • Idea for speeding up the calculation of the subclusterings (in getFalsePositives): Use a datastructure which maps from id to cluster id -> only have to find different cluster ids in cluster
• if possible show duplicate groups together

Refactor tests and provide

• mock datasets
• mock dataset files
• mock experiments
• mock experiment files
• utility functions

Most of them are already implemented and only need to be used. See wrapper/src/api/database/setup/examples

• add support for proprietary experiment formats
• add documentation on how to get to CSV formats from XLSX files

Is your feature request related to a problem? Please describe.
Analysing a duplicate detection mode should also be possible on a higher level: soft KPIs. It should be visualised in the frontend with some statistics or something like that

• add support for uploading a configuration file for each experiment
• make configuration files comparable (e.g. via git diff)

When deleting a used matching solution, two errors are thrown:

Expected: Only the second error should be thrown.

To reproduce: Reset the database and try to delete the Mock Solution

Problem: The current uploader can only handle files up to ~1GB.
Solution: Add support for partial (resumable) file upload.

Is your feature request related to a problem? Please describe.
It has to be possible to save the results of the soft KPIs in the database

We should think of a hosted version for better comparison?

We need to be able to "upgrade" databases from older versions of our tool (e.g. when adding a new column, a default value must be supplied and an old version of the table should automatically get the new column on system load).

• limit
• startAt

Idea to optimize performance for range query:

1. calculate the number of rows (including empty rows) / the number of groups
   • e.g. for the format clusters the count of groups in a subclustering with dimensions (d1,d2,d3) would be d1d2d3
2. for each cluster, save the accumulated count of rows / groups until this one
   • maybe cache this array
3. for a range query, only calculate the required subclusterings

Idea to optimize performance for sort query:

1. define that "sort" means
   • sort the values IN the clusters
   • sort the clusters by comparing their first element
2. join database table with experiment table
   • there are 2 ids to be joined on (?)
   • join 2 times (?)
3. sort the joined table by the required attribute
   • make sure id1 < id2 (regarding sorted column)
4. add the pairs in this order to the graph
5. when id1=4 is added -> all occurences of ids 0..3 are guaranteed to be in the graph

try to auto detect the experiment format when uploading experiment files by looking at the columns of the experiment.

we should also provide a preview of the experiment file (and dataset file if applicable).