Missing Files for Running COS Inference on OTT-QA about udt-qa HOT 5 CLOSED

Update: table_chunks_to_passages* has been added to HF repo, (data/OTT_table_to_pasg_links.zip)

from udt-qa.

Hi there,

Sorry for the late reply, the missing data are actually released under the CORE repo. You can download them here
https://huggingface.co/kaixinm/CORE/blob/main/data/knowledge.zip

from udt-qa.

Thank you for your response!
I have confirmed that ott_table_chunks_original.json and ott_wiki_passages.json are present in the link you shared.

However, it seems that table_chunks_to_passages*, ott_table_original*, ott_wiki_linker* do not exist. Could you please let me know where these files are located?

from udt-qa.

Due to the large file size, table_chunks_to_passages*, ott_table_original*, ott_wiki_linker* were not migrated to HF previously. You can obtain them by running inference using the model and data you have obtained:

ott_table_original*: using retrieval index expert (expert id 1) to compute embeddings for ott_table_chunks_original.json

ott_wiki_linker*: using entity index expert (expert id 3) to compute embeddings for ott_wiki_passages.json

table_chunks_to_passages*: using entity span expert (expert id 5) to identify entities in tables in ott_table_chunks_original.json, and then encode the entities with entity representation expert (expert id 2), and then run retrieval against ott_wiki_linker*

You can refer to the COS Readme on how to use each of the experts for inference. I will also try to migrate table_chunks_to_passages* in the next few days.

from udt-qa.

Thank you for your guidance on how to work with the files. It's clear now.
And it is great to hear that you are planning to migrate table_chunks_to_passages* in the coming days.
Please let me know once the migration is completed, as it will certainly be helpful.

from udt-qa.