Code Monkey home page Code Monkey logo

bartscore's Introduction

Hi there ๐Ÿ‘‹

Twitter GitHub Linkedin

I am a 4th-year computer science Ph.D. student at the University of Toronto and a graduate student researcher at the Donnelly Centre for Cellular and Biomolecular Research and the Vector Institute for Artificial Intelligence. I previously interned at Ai2 (on the Semantic Scholar team) and at Semantic Health. My work centres on natural language processing (NLP) and natural language understanding (NLU) of scientific text, particularly biomedical literature, but I am broadly interested in all things ML/AI/NLP. I completed an undergraduate degree at the University of Ottawa, graduating with a B.Sc in biochemistry. I completed an M.Sc. in computer science at the University of Toronto. When I am not ๐Ÿ’ป I like to ๐Ÿšตโ€โ™‚๏ธ, ๐Ÿ‚ and ๐Ÿ‹๏ธโ€โ™‚๏ธ.

bartscore's People

Contributors

johngiorgi avatar pfliu-nlp avatar yyy-apple avatar

Watchers

 avatar  avatar

bartscore's Issues

Original BARTScore code skips documents for which all systems achieve the same automatic metric score

Original BARTScore code skips documents for which all systems achieve the same automatic metric score

BARTScore/analysis.py

Lines 42 to 43 in 614101d

if len(set(prediction_scores)) == 1 or len(set(target_scores)) == 1:
continue

This is likely because, if all automatic metric scores are the same, their variance is 0, and correlation metrics like Spearman's R are undefined when variance is 0. This is probably harmless b/c it seems rare, but, in the most extreme case it happens for 10% of instances in REALSumm when computing rouge1_p_src_hypo. Our option are:

  1. Leave it as is
  2. Use an implementation of these correlation metrics that can handle this case (see here).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.