Code Monkey home page Code Monkey logo

loquor's Introduction

Loquor

CircleCI Heroku Project Status: WIP – Initial development is in progress, but there has not yet been a stable, usable release suitable for the public. Codacy Badge Codacy Badge Known Vulnerabilities

A translation script that parses strings and returns ranges of non-HTML/Templating syntax used by a corresponding Google Apps Script file to format strings as rich text on output. With the aim is to speed up complex translation requests by automatically highlighting translation tokens of correct strings to translate.

This project is made up of two parts, a Google Apps Script to handle interfacing with a Google Sheet, reading values and applying formatting to our output, and an API server that handle the parsing of the input data from the Google Sheet.

The overall goal being when provided with something like this:

<p>Please <a target="_top" href="%retry.url%">click here</a> to try your purchase again.</p>

Is formatted automatically to look like this:

<p>Please <a target="_top" href="%retry.url%">click here</a> to try your purchase again.</p>


The API portion of this application deploys automatically to https://loquor.herokuapp.com. It is running as a free Heroku dyno so might take some time to respond initially as it starts up.

Contents

Getting Started

  • Run npm i to install dependencies
  • Use npm start to run the node API server locally.
  • Use npm run develop to run the server with nodemon to automatically restarts the node server.

Developing locally

To work on formatting rules locally you can use any software or app to send HTTP POST requests to the local server (I'd recommend Postman). Start by sending a POST request to the /parse URL for your local server (localhost:3000/parse) passing a payload of the following:

{
    "entries": [
        ["<p>Hello world</p>"]
    ]
} 

This will then return the following:

[[[3, 13]]]

Ideally you set up a test in ./test.js then add your functionality to ensure the response is correct.

Please note - The parser only ever responds with an array of found translation strings that are safe to translate and should always fail safe and default to not being translatable.

Tunnel using Ngrok

You can use Ngrok to tunnel the Google Apps Script to your local machine during development. To get started follow the setup instructions and then run:

develop:ngrok

This script then exposes your local server from running npm start on a randomly generated public URL ( http://XXXXXXXXXXXX.ngrok.io ).

Changing the ENDPOINT constant at the top of the app.gs file (when added to your Google App Script installation) to your HTTPs Ngrok tunnel URL will route the script requests to your local server.

The Google App Script

The source code for this component can be found in the ./google-app-script directory. This interfaces with the Google Sheet of translations and handles the formatting of the cells. This script registers a custom menu, and a corresponding menu option to trigger the script processing.

This script then reads all the values from the B2-B999 range and sends that as a HTTP request to the server endpoint configured in the script. The API responds with a 2d array of results, first dimension representing the row, and subsequent arrays are substrings we're highlighting for translation with two numbers being their start and end offsets of their original string.

This component script fails safe. It will assume the whole string is not translatable unless a provided range of characters is provided denoting substrings that need translating.

Formatting rules

Below is a summary of the expected behaviour of the parser library in relation to the various templating and sample strings we're expecting.

Implementation status badges

  • Completed: ✅
  • In Progress: ⚠️
  • Pending: ⚪
Ruleset Rule Example Input Example Output Status
Handlebars Do not format {} wrapped strings {foo} {foo} ⚠️
Twig Do not format % wrapped strings %foo% %foo% ⚠️
HTML Do not format HTML tags <p>Foo<p/> <p>Foo</p> ⚠️
HTML Do not format encoded HTML entities Hello&nbsp;World Hello&nbsp;World ⚠️
URLs Do not format URLs (both relative and absolute) ../../foo/src.img ../../foo/src.img ⚠️

System Overview

Diagram

Tests

Test suite written with Mocha & Chai to ensure theparser() function correctly processes strings.

npm test

Tests are broken into separate "rule sets" that break out similar groups of assurances, such as plaintext vs HTML based tests.

Please note: These tests do not cover the Google App Script. Only the formatting logic of the API

Todo

  • Handle non-latin characters - Unsure of the requirement here as we're always using English as the base input and translating from there.
  • Expand script behavior to cover full sample

loquor's People

Contributors

dependabot[bot] avatar jamesrwilliams avatar snyk-bot avatar

Stargazers

 avatar

Watchers

 avatar  avatar

loquor's Issues

HTML Tag Parser: incorrect result for value that start with a tag pair.

HTML Parser unexpected result:

Description Data
Test Case <strong>Error:</strong> lorem ipsum.
Expected 111111110000001111111110000000000000
Actual 111111110000001111111111111111111111

Weirdly doesn't happen if the tag is at the end of the string. Or if the whole input starts with an element.

Add parser dialog.

Add UI dialog to allow some configuration changes prior to script runs:

  • Select option to change the API version / endpoint (ngrok vs Heroku)
  • Input range selection - The input range for the source data to be formatted
  • Output range selection - The output range for the formatted section (warning if overwriting input)
  • Ruleset selection (Requires API work - see #13)

Change encoding method

I have this idea to abstract the response encoding method a little during processing as a way to allow support for applying multiple rulesets against a string at once.

Currently, the parser returns [3,13] numerical arrays as results by default. While the AS script works like this I'd like to introduce a middle format to use before the responses are sent. See below:

Step Input Encoded
Default to unsafe <p>Hello World</p> 000000000000000000
Parse layer 1 <p>Hello World</p> 000111111111110000

This output is then encoded into the same numerical array of [3,13] but the output encoded string can be run through multiple rule sets removing possible issues finding overlaps in the numerical arrays.

Improve documentation

  • Clean up README - Its a bit messy at the moment, extract docs to separate repo/site
  • Add docs section - Adding a new validator
  • Adding a new test cases
  • Google Apps install instructions - (end users)

Handlebars Parser: Unexpected parsing result

handlebarsParser() returns an unexpected result for the following test case:

Description Data
Test Case {quantity, plural, one {%lp.currency.name%} other{%lp.currency.name.plural%}}
Expected 11111111111111111111111111111111111111111111111111111111111111111111111111111
Actual 11111111111111111111111111111111111111111110000001111111111111111111111111111

Add ruleset selection parameter to parse request

We should add a parameter to override what rulesets to be used by the request. Suggested syntax:

Current payload:

{
  "entries" : []
}

Proposed payload:

{
 "options": {
    "rules": ["html, handlebars, twig"]
  }
  "entries" : []
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.