repent / acromatic Goto Github PK

View Code? Open in Web Editor NEW

1.0 2.0 0.0 23.74 MB

Online tool to extract a list of acronyms from a document for definition.

Ruby 78.84% JavaScript 2.79% HTML 2.99% CoffeeScript 0.69% SCSS 2.58% Haml 12.12%

acromatic's Introduction

acromatic

An online tool to extract a list of acronyms from a document for definition.

See https://acromatic.uk for a usable online instance.

Binary dependencies on Debian Buster

apt install docx2txt pandoc

acromatic's People

Contributors

Stargazers

Watchers

acromatic's Issues

Add blacklist to dictionary

Allow a dictionary to blacklist certain acronyms so they are ignored completedy and not added to the list

Doesn't automatically pick up definition if the term is in quotes

E.g. "Authorised Economic Operators" (AEO) is not understood.

Clean up plurals

Add a "singularize" button to the dictionary list view, which (tries to) operate on both acronym and meaning.

Think about trying to always list the singular in the acronym list, even if it doesn't appear in the text?

A plural definition does not match a plural acronym (only a singular dictionary definition will match a plural acronym) -- this is wrong (but would be less bad with a singularize button).

Auto find acronym definition

May be able to pick up a definition in the form compared to the North Atlantic Treaty Organisation (NATO), it has not been very successful.

Allow adding defs to dictionaries without pasting tab characters

The quick add screen is very cumbersome to type new defs into directly.

Infer meanings seems to attemp the same match several times.

I, [2021-06-16T09:22:06.111538 #7241]  INFO -- : Failed to match 'PNA' in ' by eight Parties to the Nauru Agreement'
I, [2021-06-16T09:22:06.113476 #7241]  INFO -- : Failed to match 'PNA' in ' by eight Parties to the Nauru Agreement'
I, [2021-06-16T09:22:06.127245 #7241]  INFO -- : Failed to match 'PNA' in ' by eight Parties to the Nauru Agreement'
I, [2021-06-16T09:22:06.128658 #7241]  INFO -- : Failed to match 'PNA' in ' by eight Parties to the Nauru Agreement'
I, [2021-06-16T09:22:06.130017 #7241]  INFO -- : Failed to match 'PNA' in ' by eight Parties to the Nauru Agreement'
I, [2021-06-16T09:22:06.131364 #7241]  INFO -- : Failed to match 'PNA' in ' by eight Parties to the Nauru Agreement'

Resolve current failure of IS-LM

Probably best to disallow internal hyphens/+s and remove the interface option.

Uppercase roman numerals are caught

Infer meanings when incidentals are present

Update get_meaning() to work if incidentals are present.

Add testing for this at least!

ArgumentError (invalid byte sequence in UTF-8)

I, [2017-04-12T09:34:41.695744 #12690] INFO -- : Completed 500 Internal Server Error in 102ms (ActiveRecord: 3.0ms)
F, [2017-04-12T09:34:41.696723 #12690] FATAL -- :
ArgumentError (invalid byte sequence in UTF-8):
app/models/document.rb:40:in scan' app/models/document.rb:40:in trawl'
app/controllers/documents_controller.rb:32:in block (2 levels) in create' app/controllers/documents_controller.rb:29:in create'

Allow post-definition

E.g. MOU (memorandum of understanding).

When finished editing an acronym should be returned to the dictionary not all acronyms

Handle duplicates somehow

Quickadd makes it easy to add duplicate definitions (same or different).

Easyest fix would be to have a "kill duplicates" button, highlighting but leaving conflicts?

See Google contacts merge.

Support .odt

Transition to mariadb

Regexp generation broken by superscript.

App broken by CS^2.

Error: premature end of char-class: /(CS[2)/

docx2txt spits out CS^2 as CS[2] which is converted to /\CS[2)/ as a regexp, which is invalid due to the unterminated [.

Create a difference between mergeable and non-mergeable duplicates

If acronyms are exactly duplicated, perhaps they should be automatically merged on batch upload, with only conflicting definitions appearing in red as now.

Drag and drop

Can't change dictionary after a document has been uploaded

Handle consolidation

Handle the case of a user wanting to create an acronym list from multiple files, or (equivalently-ish) a user wanting to add the acronyms from a new file to an existing acronym list (delete duplicates, realphebetise).

The first solution is probably easier to implement?

Crashes

D, [2018-09-06T12:09:13.893274 #30489] DEBUG -- :    (0.1ms)  SELECT COUNT(*) FROM "acronyms" WHERE "acronyms"."document_id" = ? AND "acronyms"."initialism" = ?  [["document_id", 502], ["initialism", "WTO"]]
I, [2018-09-06T12:09:13.905907 #30489]  INFO -- : Completed 401 Unauthorized in 54ms (ActiveRecord: 2.6ms)
F, [2018-09-06T12:09:13.907774 #30489] FATAL -- :
ActiveRecord::UnknownAttributeError (unknown attribute 'meaning' for Acronym.):
  app/models/document.rb:179:in `block in trawl'
  app/models/document.rb:94:in `scan'
  app/models/document.rb:94:in `trawl'
  app/controllers/documents_controller.rb:42:in `block (2 levels) in create'
  app/controllers/documents_controller.rb:39:in `create'

Add sentence-case button to dictionary listings

Add a button to the dictionary listing that would convert an acronym from, say, "Total Allowable Catch" to "Total allowable catch".

Trash collection

850MB of uploads collected by 2/18, need some periodic cleanup of /public/uploads/document/file.

RegexpError (premature end of char-class: /\(R[E\)/):

RegexpError (premature end of char-class: /\(R[E\)/):
  app/models/document.rb:280:in `location_of_bracketed'
  app/models/document.rb:304:in `bracketed?'
  app/models/document.rb:290:in `get_index'
  app/models/document.rb:308:in `get_context'
  app/models/document.rb:186:in `block in trawl'
  app/models/document.rb:100:in `scan'
  app/models/document.rb:100:in `trawl'
  app/controllers/documents_controller.rb:42:in `block (2 levels) in create'
  app/controllers/documents_controller.rb:39:in `create'

Quick-update current dictionary

Logged-in users could see an "bulk add to [current dictionary]" button underneath the main text area of acronyms after uploading a document, to speed up the learning loop.

Allow a trailing +

PACER+, TAF2+

Singularise plurals

Currently, if the plural appears and only the singular is in the dictionary, the acronym is treated as unknown.

Probably the best resolution is a "singularize" option that only lists the singular form (stripping Ss where necessary), which probably fits with most style guides.

Sort dictionaries with collisions first

Highlight meaning in context when found

Autodefine doesn't work if the definition is in quotes

E.g. if the text features '"skills for competitiveness" (SFC)' then the definition won't be picked up.

Punctuation symbols

Separate category?

e.g. S&DT TAF2+

Dictionary creation fails with new acronyms

 3 errors prohibited this dictionary from being saved:

Definitions is invalid
Definitions is invalid
Definitions is invalid

Add search link

Add a link for each acronym found to look it up in a search engine.

Crash on "PICs-"

Managed to crash it with "PICs-" -- not sure what type of dash, I think it was a hyphen.

Error:

D, [2021-04-02T21:15:39.598782 #15535] DEBUG -- : (1.3ms) commit transaction
D, [2021-04-02T21:15:39.598926 #15535] DEBUG -- : (0.0ms) begin transaction
D, [2021-04-02T21:15:39.600109 #15535] DEBUG -- : SQL (0.2ms) UPDATE "acronyms" SET "document_id" = ?, "updated_at" = ? WHERE "acronyms"."id" = ? [["document_id", 1347], ["updated_at", "2021-04-02 21:15:39.599245"], ["id", 119326]]
D, [2021-04-02T21:15:39.601668 #15535] DEBUG -- : (1.2ms) commit transaction
D, [2021-04-02T21:15:39.602444 #15535] DEBUG -- : Acronym Load (0.1ms) SELECT "acronyms".* FROM "acronyms" WHERE "acronyms"."document_id" = ? AND "acronyms"."initialism" = ? ORDER BY "acronyms"."id" ASC LIMIT 1 [["document_id", 1347], ["initialism", "STCW"]]
D, [2021-04-02T21:15:39.603524 #15535] DEBUG -- : (0.1ms) begin transaction
D, [2021-04-02T21:15:39.604163 #15535] DEBUG -- : (0.0ms) commit transaction
D, [2021-04-02T21:15:39.604689 #15535] DEBUG -- : CACHE (0.0ms) SELECT "acronyms".* FROM "acronyms" WHERE "acronyms"."document_id" = ? AND "acronyms"."initialism" = ? ORDER BY "acronyms"."id" ASC LIMIT 1 [["document_id", 1347], ["initialism", "STCW"]]
D, [2021-04-02T21:15:39.605734 #15535] DEBUG -- : (0.1ms) begin transaction
D, [2021-04-02T21:15:39.606255 #15535] DEBUG -- : (0.0ms) commit transaction
D, [2021-04-02T21:15:39.606908 #15535] DEBUG -- : Acronym Load (0.1ms) SELECT "acronyms".* FROM "acronyms" WHERE "acronyms"."document_id" = ? AND "acronyms"."initialism" = ? ORDER BY "acronyms"."id" ASC LIMIT 1 [["document_id", 1347], ["initialism", "PICs-"]]
I, [2021-04-02T21:15:39.608110 #15535] INFO -- : Completed 500 Internal Server Error in 1492ms (ActiveRecord: 366.6ms)
F, [2021-04-02T21:15:39.609079 #15535] FATAL -- :
RuntimeError (Acronym PICs- not re-found in the text document):
app/models/document.rb:310:in get_context' app/models/document.rb:186:in block in trawl'
app/models/document.rb:100:in scan' app/models/document.rb:100:in trawl'
app/controllers/documents_controller.rb:42:in block (2 levels) in create' app/controllers/documents_controller.rb:39:in create'