arademaker / hs-conllu Goto Github PK

plus they can have the weird bracket-thing, which I don't know for what use it is.

8	قوللىرى	قول	NOUN	N	Case=Nom|Number=Plur|Number[psor]=Plur,Sing|Person[psor]=3	10	nsubj	_	Translit=qolliri

from the Uyghur UD test set.

[source]

For users of your parser who do not use it standalone, but integrate it with other code, two simpler functions would be beneficial:

parseConllu :: P.Parser [T.Sentence] -> Text -> ErrOrVal [T.Sentence] 
-- | parse a text (no IO) to sentences
parseConllu parser text = 
    case r of 
            Left err -> Left (s2t $ M.parseErrorPretty err)
            Right ss -> Right   ss
    where 
        r = M.parse parser "" (t2s text)   -- why a textname required ?? 

prettyPrintConlluSentence ::T.Sentence -> Text 
-- | prettyprint a single sentence
prettyPrintConlluSentence  = s2t . Pr.fromDiffList . Pr.printSent 

where ErrOrVal is Either Text and s2t is a conversion is pack from Data.Text.

In case you wonder - I use your package to parse output from coreNLP (udfeats) and stick them into a triplestore. Thank you for your effort!

• get confirmation email
• set up travis builds
• document code
  being done in the haddock branch.

Is there a plan to incorporate the same level of validation performed by the Universal Dependency tool, validate.py, found here https://github.com/UniversalDependencies/tools? Also Is there a plan to do a performance comparison between the official Universal Dependency validation python script and hs-conllu?

The universal dependency organization's validation software provides 5 levels of validation. Because it is written in python, I suspect it would be slower than written in Haskell or C++.

so that we can accumulate all errors and report them, instead of getting them one at a time.

it will parse cc first and then complain about a spurious 'o'

at least to check a few properties such as the identity on parsing/printing.

maybe use this library? https://github.com/mrkkrp/hspec-megaparsec

The function`` printSentseems to add a newline at the beginning of a sentence (and one at the end as well) which causes theprint . read` not to be identy. perhaps you remove the line which is commented out:

printSent :: Sentence -> DiffList Char
printSent ss =
  mconcat
    [ printComments (_meta ss)
--    , diffLSpace   -- causes an extra space initially  
    , printTks (_tokens ss)
    , diffLSpace
    ]

trigerred by #17 and UniversalDependencies/UD_English-EWT#60.

files here.

ideally, semantic validation would be separate from parsing (syntactic validation), but maybe this would require too much mapping between types?

I wrote the text below as an open reply to @arademaker for our conversation on #32 about plans for the library.

I'd like to change the structure of the library a bit: first have a really dumb parser that would accept anything remotely matching the conllu format, then do light validation on top of it according to user specification. This would mean not to hardcode deprels and other stuff, but read a file that lists the acceptable entities (these files already exist for the canonical validating script, but the user could tweak them if they wanted to).

I also think that the megaparsec library might be unnecessary since the conllu format is so simple, but its performance is not bad and the error-reporting facilities are great (are we using them as well as we could?), so maybe I'd leave that be. If there's a performance need, then we might think about it.

I don't think it's worth it to implement full conllu validation, for the reasons I said on #34.

At some point I had plans for a query interface like the one in http://match.grew.fr (see master...query), but honestly I don't think it's worth implementing it since just loading the data on a graph database would give better-performing queries and facilities for visualization for free :)

Finally, I started writing this library a long time ago when I first started learning Haskell, so I would also change the code quite a bit to reflect some of what I learned since then.

I used the parser on a small example and found that dobj is missing in the Dep enumerated type.
It might be better to move the tagset into separate modules to be imported qualified (to avoid construction like AUXpos and similar). and to publish the tagset independenly in a separate package for all to use and to improve (I have constructed one which I hope is more complete adn may submit a PR).

The error is produced by the code https://github.com/cpdoc/dhbb-nlp/blob/master/bin/CountLexicon.hs, I can't say if the problem is in the script or the library.

$ cat out.hs.orgs.error
countlex: Maybe.fromJust: Nothing
$ cat out.hs.people.error
countlex: Maybe.fromJust: Nothing

http://universaldependencies.org/u/overview/enhanced-syntax.html

including ref relation.