neutralitywtf / conceptreplacer Goto Github PK

The first example featured in the home page is currently Ada Lovelace, and right in the first sentence there's "Countess of Lovelace", which should have been switched to "Count of Lovelace".

First of all, it wasn't immediately clear where the code repository for this experiment was located, and second, after I came here I wasn't sure how to propose the change.

Should I edit men.json to change "singular": [ "king" ], into "singular": [ "king" , "count" ], then edit women.json to change "singular": [ "queen" ], into "singular": [ "queen" , "countess" ],? Or should I edit the royalty key instead? Should I create a new key, otherwise? And if so, what would be the appropriate nomenclature?

These questions make it hard for interested people to contribute and make the experiment less useful as a consequence. IMO some instructions in the README, or a CONTRIBUTING.md file (and a more prominent link in the website) would help.

/cc @slashme

There seem to be cases where the content of the <script> tag is not properly removed.

Example, see this article: http://www.neutrality.wtf/api/api.php?localize=1&url=http%3A%2F%2Fmoney.cnn.com%2F2017%2F08%2F21%2Fnews%2Feconomy%2Fgirls-who-code-saujani%2Findex.html

There's a block of script content (without the tags) in the middle. If you look at the original article's source (view-source:http://money.cnn.com/2017/08/21/news/economy/girls-who-code-saujani/index.html) it's clear that the script is inside a <script></script> tag. It's possible the DOM parser is confused because of the string "" that is immediately before that text.

This requires some investigation; it might be DOMDocument and xpath itself.

Part of the reason it fails right now is the inline styles that have changed. It shouldn't depend on those (we should resolve #3 ) but it should also pass consistently.

There's already phpunit that can run (though they need to be adjusted, they are failing at the moment, due to being written for a slightly earlier version) so we need to add phpcs and connect things to Travis to get CI going.

Since all methods are static, we constantly parse and output HTML for the next step. We should stop doing that. Parsing should happen once per dictionary-replacement, and the DomDocument object should be the one that's passed around.

There might be opportunity to consider whether the Replacer needs to work with static methods only, or if it just needs to be instantiated per replacement, pass around the document internally based on external calls. Or, we can keep its methods static but have all of them expect a DOMDocument instead of html, and add a one-time parsing method that runs ahead of all of them, then pass around the changed/saved DomDocument around.

Whatever we do, we need to consider that the Replacer should be flexible enough to also do future replacing methods where, perhaps, plain-text is passed in (it could be mocked into an HTML document, be parsed as if it's all one by paragraph, and then still be sent as DomDocument around, so we keep the wrapping of the output)

The replacements are already tagged with a class - we should attach a proper style at the to style them all rather than attach the style to each inline, like is being done right now.

Right now we're fetching all //text() nodes; it seems that also replaces content in <title> tags, which is not ideal.

The replacement should take both into account; split to 'formal' - "mother" / "father" vs familiar - "dad" / "mom"

There may be other words to consider splitting as well; this should also be taken into account when redoing/redesigning the dictionaries in general for #7

The dictionary categorization was born in the hackathon, and it's really bad.

The two dictionaries must match in categories of words, but the categories should really be organized properly.

Passing URLs through neutrality.wtf seems to display them without the localized fixed links. Not sure where this started.

Example: http://www.neutrality.wtf/api/api.php?localize=1&mobile=0&url=https://en.wikipedia.org/wiki/Silver_Jubilee_of_Queen_Elizabeth_II

The replacer removes <script> tags, but we should also safely remove inline scripts from nodes, if possible.