See #13 for example

It is an option to return "Aus bus" and "Aus bus bus" when a search is done for one of them. The issue is related to gnames/gnmatcher#49 and was suggested by @abubelinha there.

I guess something similar to resolver score

Stemmed canonical will allow to alwas get names with different suffixes

When running gndiff -V shows no info about how to run as a server.
So I was assuming it is still work in progress.

But I have realized the documentation mentions it as an already implemented feature.

It is not working for me:

c:>gnames\gndiff -V

version: v0.2.1

build:   2023-09-06_18:45:19UTC

c:>gnames\gndiff --port 8080
Error: unknown flag: --port
Usage:
  gndiff source_file reference_file [flags]

Flags:
  -f, --format string   Sets output format. Can be one of:
                        'csv', 'tsv', 'compact', 'pretty'
                        default is 'csv'.
  -h, --help            help for gndiff
  -q, --quiet           Do not output info and warning logs.
  -V, --version         shows build version and date, ignores other flags.

Error: unknown flag: --port

Hi @dimus
This is more a question than an issue.

My initial idea of gndiff was comparing two files so its .csv input design is perfect for that:

• I call the executable from a Python script.
• That script has to generate reference.csv and source.csv files
• Then it calls gndiff command line executable passing those filenames
• It catches the output and processes its content from Python again.

That just works.

Now I am wondering about some other possible use cases with frequent repetitive gndiff calls (also from Python).
My concern is whether so many disk-write/disk-read of .csv files could/should be avoided.

Figure out my script is parsing a long list of new specimens to include in a museum collection.
I might prefer to gndiff-match them one by one, for whatever reason (my script could need to make other intermediate tasks in a certain order before processing next specimen name).
So I would be passing gndiff a small source.csv with just one row, but so many times.

In such scenario, would it make sense not creating a source.csv in disk (which means a Python file-write, plus a gndiff file-read), but somehow passing the source info as a parameter instead?
Maybe this is already possible somehow although I am not sure about what syntax should I try.
Or maybe this doesn't make sense at all because the script performance would be similar (i.e. the intermediate tasks are slower than gndiff call).

Of course, I can always design my script to process all gndiff-matching operations in advance.
I am just thinking before scripting and I am not a professional, so don't take me too serious.

Somehow related to this, in #13 I suggested the possibility of using gndiff as a server (so we can run gndiff in one machine and call it from others).
If that feature ever becomes possible, I wonder how such a server would work.

• I guess the idea is repeating exactly the same stuff (gndiff receiving two files, doing the work and returning the output as http response)
• But another possible scenario is running it as a server of a predefined reference list: so reference.csv is not passed in http requests, but defined at server start time ... so the requests only contain a list of source taxa (or just one taxon) to match against that reference.csv ... so again, the server could be receiving small but repetitive matching tasks.

Just wondering

First of all, thanks a lot for creating gndiff. It's gonna be so useful for me.

I have just tried with a small file, to get the feeling of how it works.
I already found some issues to comment:

• Unclear input file formats description, where it says "Prepare two files with names. There are 3 possible file formats:" ... but actually, only two formats are mentioned: (1) simple list, one name per line; (2) csv file, with some other fields (see below).
  Also, it is unclear to me if the CSV format applies only to the reference.csv file or also to source.csv:

  • Names to be matched (source.csv), might also contain their own ids, but it is unclear to me whether gndiff is suggesting user to provide them or not (i.e., for adding them as a new column in output, so it is easier for me to rejoin that output against my original database).
    I think that is not the idea, because output already provides an autonumeric index. So I understand source.csv would usually contain just one field, with names and nothing else (just one column). With one possible exception:
  • Except in case of using Family: I suppose in that case it should be present in both files. Correct? But I couldn't make it work properly in my tries (see below):

• I might have misunderstood input CSV format description above. But if Family and TaxonID are optional fields?, then JSON output contains errors sometimes:
  1.. If I don't provide a Family column in reference.csv, then json output referenceRecords[n].family contains the same value as name (the ScientificName field provided in my reference.csv file).
  2.. If I provide a Family column in reference.csv (even with empty values), then json output seems correct (referenceRecords[n].family contains those family values I provided).
  3.. But if I also provide a Family in source.csv, then json output includes a new sourceRecord.id which does contain the same value as sourceRecord.name.
  4.. If source.csv contains other columns (i.e., ScientificName + LifeForm) then json output produces sourceRecord.family=sourceRecord.id=sourceRecord.name (all containing the ScientificName provided in source.csv).

  So I am a bit confused. I think it would be worth to provide a couple of sample input files, and explicitly say if they can/should contain some other columns or not.

  Regarding family: A real example case of how "tricky homonyms where family helps to resolve taxa from each other" would be useful too (I think family is not going to solve anything in my case, but just to be sure). I wonder how this "use family" option affects speed: does it make matching faster or slower for large datasets?

• CSV/TSV outputs are missing column headers? This could seem irrelevant, but it makes a bit difficult to check if the output content is correct. Also, I cannot proceed with further tasks, like merging this output with other tabular data by means of column joins (I can try to figure out headers and add them myself ... but it would be safer if gndiff did it to avoid mistakes).

EDIT: I have just realized that some of the above suggestions were already addressed by @Adafede in a previous closed issue (#12).
Sorry about that. My comments are pretty verbose, so @dimus might still find some helpful feedback in some of them.
This is a new one:

• Shouldn't the output include a sort of calculated numeric similitude between the matched names? I have some cases where json produces several "Exact" matches (i.e. two referenceRecords for the same sourceRecord) because my reference.csv contains two similar versions of the same name (i.e. a subsp. rank vs. a var. rank, identical in everything else). But my source.csv only contains one (i.e. the subsp.). How can I make the decision to select the most similar in these cases?
  I will better post an example in a new comment to illustrate this.

Thanks a lot in advance !!

In case if current fuzzy is not enough to add other levels. May be something like

FuzinnesLevel 1, 2, 3

@abubelinha wrote

If I don't provide a Family column in reference.csv, then json output
referenceRecords[n].family contains the same value as name (the
ScientificName field provided in my reference.csv file).

I suspect it should be default, and an option should be to allow more than one row.

Hi, I tested quickly your tool and here is what I can say:

• Installation works fine
• Commands are clear and consistent with other GNtools
• Adding headers to csv/tsv format would be nice (found them via -f pretty)
• The family column in the output is quite cryptic for external users, a brief description in the README would probably help.
• The error message FATA[0000] the CSV file needs `scientifiName` field contains typos
• Globally, very nice even if I don't see applications at the moment on my side, adding somehow a more straightforward way for the user to do gnfinder and then gndiff on files might open possibilities. I am sure you already had this in mind but I see this combination as very powerful in the future.

Your work is amazing and useful to so many people out there!

Based on @abubelihna feedback in #13

See #13