yale-lily / folio Goto Github PK

Hi,

Thanks for your great work, I really like it!
But I want to ask that after the model generates the FOL, how should it be executed? You provided a link in the paper but it is broken somewhere.
Hope for your reply!

Best,
Tianbao

Appreciate your response.. Thanks.

Please let me know whether I've understood the paper correctly:

• Given a set of First-Order-Logic Premises, it's a ~mechanical task to resolve the True/False/Unknown for the given Conclusions
• So, the task that arises naturally from this dataset is a "natural language to first-order logic translation (NL-FOL translation)" task
  • Because, if we can do the NL-FOL translation well enough, then we can rely on a FOL solver to do the 'thinking' for us;

Naively, this seems like a simpler goal than build an NLP model that goes directly Natural Language input to Natural Language resolutions. But I'm guessing that doing the NL-FOL translation task is challenging to do at high enough accuracy rates to enable a solver to work on the outputs?

( Just trying to make sure I don't dive into building a translation tool, whereas the 'real problem' is to do with few-shot prompting, or more super-LLM training-based ).

I think that your translation of the sentences of the form 'Some As are Bs' is incorrect, e.g.:

Some musicians love music. > ∃x (Musician(x) → Love(x, music))
Some fish may sting. > ∃x ∃y (Fish(x) → Sting(x,y))

These should be translated with the help of the conjunction and not implication, e.g.

Some musicians love music. > ∃x (Musician(x) ∧ Love(x, music))
Some fish may sting. > ∃x ∃y (Fish(x) ∧ Sting(x,y))

Consult, for example, https://cse.buffalo.edu/~rapaport/191/S09/transtip-pnllogic.html#plsinglymonadic.

The reason is that the "implication" formal versions are semantically "too weak" - they are satisfied also when there are no musicians, no fish, etc.

Hello,

It's been since September that you didn't update the repo with the FOL convertor and evaluation codes.

Can you please add them?

number 107 of all propositions 'Customers who prefer TV series more and will not watch TV series in cinemas.' is mislabeled. Should be 'Customers who prefer TV series more will not watch TV series in cinemas.' if premises-FOL is

∀x (PreferTVSeries(x) → ¬WatchTVInCinema(x))

looking foward to the code, thank you

I only see the dataset, the paper seems to say that the code will be made available?