lingua-libre / signit Goto Github PK

Lingualibre wikibase API may be in end of life.

Explore relying on Wikimedia Commons API queries :

One API query response on Commons has a limit of 500. To ensure code's resilience and handling of larger categories you need to recursively query the API using continue.cmcontinue if you query using list=categorymembers. Append &origin=* or it could fail due to xhr cross origin.

Clean ups ?

May require bot clean up or API query starting on parent category https://commons.wikimedia.org/wiki/Category:Videos_Langue_des_signes_française

See also

• Wikimedia Commons API queries
• https://mediawiki.org/wiki/API:Continue
• https://jsfiddle.net/dekz6Lbu/5/ Recursive queries using JS promises.
  • https://jsfiddle.net/90pL64f1/1/

Add-on handles several languages parameters :

• signLanguage
• relevantWiktionary, source for the definitions, to be readable by the user.
• relevantWiktinarySection = source for the definitions, the language of the written words clicked, written in uiLanguage (complexity therefore exploding).
• uiLanguage

Question

Is the relevant Wiktionary relevant to that sign language or to the add-on's user's language ?

signLanguage

As for uiLanguage, we currently store the following info :

{ wdQid: "Q99628", nativeName: "langue des signes française"}

We could store parentLanguage: [ 'ru', 'kh' ] so to support languages such as Russian-Kazakh Sign Language.

uiLanguage

As for uiLanguage, we currently store the following info :

{ wdQid: "Q1860", nativeName: "English", associatedWikt: "en", associatedAnchorId: "#English" },

See also this WDQS query.

Complexity arise

Spanish user studying American Sign Language, may want:

{ uiLanguage:"es", signLanguage:"ASL",relevantWiktionary:"es", relevantWiktinarySection:"#Inglès" }

Hindi user studying American Sign Language, will want:

{ uiLanguage:"hi", signLanguage:"ASL",relevantWiktionary:"hi", relevantWiktinarySection:"#अंग्रेज़ी" }

Policy needed

Must think about a better balance between hard coded data and ../i18n/

Seems logical to assume the learner of American Sign Language knows to read in ENGLISH, and entry to ASL must go via this parent language.

See also

About "Add language UI in settings": Consider https://github.com/wikimedia/jquery.uls .

Move toward a more Sign Languages specific visual branding.

Colors

• grey #474847 or #484848 (or variants) 17a39d2
• green #00a99d a1e40d0#diff-5f677133bd3f5a4dddc8497184199679c2c3b434018feaad5325405970cc3dde
• NOT TO USE. blue #36c

Phase 1

• Decide a base color : KEEP GREEN
• Create new logo with hands https://commons.wikimedia.org/wiki/File:Lingualibre_SignIt-logo.svg
• Adapt new logo with right color

Phase 2

• Integrate new logo to repository https://github.com/lingua-libre/SignIt/tree/master/icons
• regenerate derivative icons https://github.com/lingua-libre/SignIt/tree/master/icons
• commit to repository
• release on github https://github.com/lingua-libre/SignIt/releases 1.0.19

Current code has already several features making it nearly ready for cross-browser support.
But it got to be checked further.

Resources

• Building a cross-browser extension
• Chrome Developers > Migrating to Manifest V3

Status

See also

#21

Update manifest

• Update manifest.json with new urls

Improve documentation

• Expand readme.md : How to install on my computer ?
• Expand readme.md : add description from available resources (Firefox add on description)
• Add doc folder, add screenshots from available resources (Firefox add on description)

@edouard-lopez : do you actively use this project ? Could you share some visuals ?

Done in content_scripts/signit.js

• HTML/CSS coding 8b0b972
• hint-icon appears properly when text is selected 8b0b972

• Click on hint icon sends message which open the modal. 6ac4654
• Improve CSS
• Improve position while scrolling

As optional feature

• Hint icon toggle in settings and locaStorage value
• Hint icon conditional to setting and locaStorage value

Issue: A lot of time the language set by the user for the extension gets reset, this behavior is seen usually when we open a new tab or just navigate through various URLs in the website.

Reproduction: Change the default language of the extension to any language (English in my case), and just use the browser normally, and keep checking the extension popup's language. You will be very quickly able to see the language change.

My System:
OS: Windows 11 with WSL (Ubuntu 20.04)

Reported by @edouard-lopez

On Firefox I can't get a list of word nor a definition even for word from Category:Videos_Langue_des_signes_française

Reproducing the bug

• Open Firefox (NOT nightly) > Google Lingualibre SignIt Firefox > Go on mozilla apps store : Install app
• Open Wikipedia's article Changer :
  • right-click on word changer > SignIt
  • On top right of window : click on extension SignIt > enter changer.

FETCH INDEED FAILS.

Sur les wikis de WMF et Firefox, le module fait disparaître les deux icones de la barre utilisateur située tout en haut.

Having the wikitionary definition is a nice addition!

However, the links broke the pop UI as they are relative to the extension

moz-extension://cf0f4968-86f0-4896-9d0f-baeaa1a6a094/wiki/%C3%89tat

Solutions

1. They should be fixed by prefixing them with FQDN, e.g. https://…
2. render as text.

My preference goes to the first solution.

Half of current settings rely on large, double-line OO.ui.ToggleSwitchWidget :

• wpintegration
• twospeed
• hinticon
• coloredwords
• showvideo

See ./popup/popup.js.

We could migrate to single-line checkboxes elements such as

• OO.ui.CheckboxInputWidget
• OO.ui.CheckboxMultiselectWidget
• OO.ui.CheckboxMultiselectInputWidget

The differences of usage of those later 3 must be studied before implementation.

Also : should we do this ? This change would create 2 status in the setting, some larger 2-lines options, and some smaller single-line options.
Is this an acceptable UI/UX practice ?

See also : #36

Current	Suggested

Test

• Retest locally via npm run web-ext:test -> works
• Edit version number in package.json & /manifest.json
• Push to github

Publish

• Rebuild into 1.0.2
• Publish on Mozilla store: /lingua-libre-signit /versions

See also : #4 #5 .

Hello @kabir-afk , following your email I'm happy to grant you contributor rights.
Few rapid things to know 👍🏼

• Legacy: This web extension project is 5 years old, quite modest and now a bit messy, but still okish, and I'm around to answer questions.
• Manifest v2 : Please note this current code is v2, which will become deprecated on Extensions stores
• #21 : Being a junior developer handling legacy code, I have the way but I'm struggling to migrate to v3.
• Few codes coming : please wait a week or two to edit this repository, I got some local changes waiting to be pushed to github. A tiny new feature.
• It may be more time-efficient to study this version and recreate it from scratch using Manifest v3 boilerplate and ChatGPT's help. See #21 section V3 from scratch ? for leads.
• I actually look for someone to lead the development side so I may focus on coordinating partners to create more signed videos. While the project could receive all sign languages and the UI is translated in 30 langues, we only have video data for French Sign Language. Which is far too little.

You have the gist of it. The rest is coding and project management.

Some language communities may have and therefore need :

• only wiktionary
• both
• only the sign language videos.

Also, we would gain to make these blocks optional, based on the language ui used and user settings.

Actions

• clean and normalize SignItCoreContent.js and popup's classes, ids names.
• create proper panels, check for responsive html, css in SignItCoreContent.js
• create initial setting and customizable settings in background.js and popup.js

1.0.8

1.0.9

1.0.10

1.0.11 - start to fix

Add i18nsupport on the model of https://github.com/lingua-libre/RecordWizard/tree/master/i18n .

• Create /i18n/
• Add qqq.json
• Add en.json
• Add fr.json
• Add language UI in settings
• Load translation file
• Use translation file
• Works, need to clean the mess.
• Restore wpintegration and overlay

See also #29

Si on commence à communiquer sur l'extension, ça serait bien de rajouter le nom de Wikimédia à France à quelques endroits

When trying to manually install SignIt from chrome://extensions on Chrome (and from edge://extensions on Edge), with Developer Mode enabled, I get the following error message:

Failed to load extension
File 
~\Downloads\SignIt\SignIt-master
Error
'content_security_policy.extension_pages': Insecure CSP value "https://commons.wikimedia.org" in directive 'object-src'.
Could not load manifest.

Tested on Chrome latest version: 121.0.6167.185
Tested on Edge latest version: 121.0.2277.128

With SignIt latest commit on master: 75d09f3

Hello @pushkar707, @shashankiitbhu, @JatinSainiOO7, @LeoDupont . cc @kabir-afk

Understanding

In past few weeks I saw 4 new github accounts arrive on this open source Wikimedia project. I would like to understand better the reason of these arrivals and the possibilities it could open. Also may I ask you more about you and your interest in SignIt ?

1. How did you discovered SignIt's repository ?
2. Are you wikimedians editors ? -> which wiki projects
3. Are you developer ? -> which coding languages ?
4. What is your relation to sign languages ? -> practicing, interested, related people are practicing ?

2024

It's great to see this new activity and I strongly suspect it is linked with our recent GSOC exploration which is playing out this late March. I would like to understand your interest better so I may answer your needs better. Feel free to DM me if you prefer.

Following @saltykheera challenge, the choosepanels have been implemented.
Few steps before a new release.

Translations

• Wait one month for translate-wiki's translations to come in

Finish #36

• Clean Salty's in-modal toggle, either :
  • remove from SignItCoreContent.js + content_script/signit.css
  • rewrite with OO.ui.js in SignItCoreContent.js. Behavior should synch with localStoreage choosepanels.

Quality

Some clean ups to do :

• Distinguish better the setting popup and the on-click modal by renaming

Si l'extension commence à être pas mal utilisé, faudra sans doute ajouter un bouton de signalement. Mais faudra voir qui est capable de traiter ces signalements. Cet ajout peut attendre la phase où l'extension existera aussi sur Chrome.

Moved to T313348

Where should it lead ? -> Special:RecordWizard

URL: https://translatewiki.net/w/i.php?title=Special:Translate&showMessage=signit-si-overlay-coreContent-right-error&group=signit&language=hi

No defition found. should be No defintion found.

This actually implies to do a bunch of code migration, with async, etc.

Chrome's migration checklist

Chrome provides a migration checklist which points to more actionable resources :

• Update the manifest : manifest.json, fix syntactic error warnings.
• Update your code
  • Migrate to a service worker
  • Update API calls
  • Replace blocking web request listeners.
  • Improve extension security
• Publish your Manifest V3 extension

V3 from scratch ?

• Crxjs : versatile
• Templates: versatile
• React
• React + vite

Resources

• Firefox > Manifest V3 migration guide
• Chrome > Migrating to Manifest V3
• Extension Manifest Converter

Related

• #32 -- require v3 before to move forward.

Like fir WikiCheese, lot can be done with simple improvement to the photographic settings. See below.

• Write in #18
• Migrate to .md or .html file -- some screenshots are copyrighted and cannot be shared on meta.wikimedia.org

References

• (en-US) Amanda Morris, « How Sign Language Evolves as Our World Does », The New York Times,‎ 26 juillet 2022 (ISSN 0362-4331, read online
• La Bulle Elix par Signes de sens

Good light, soft-colored background

These videos settings seems ideal. The soft-color in the background allows the signers to fully benefit from provided, multi-sources and diffused lights. These videos screenshots also show casts visuals hints in white lines, which involves additional and costly post editing.

Good light, black background, black shirt

Good light, green screen, whitened

These pure white background are provided thanks to green screen and post editing. It integrates well with computer textual contents, readable webpages mostly having white background as well.

Zero Euro studio with correct lighting

Poor light

This video setting lacks both proper light and professional camera. Professional cameras have white balancing features which can partially neutralize yellowish backgrounds. The video quality also seems to be lacking, which could be due to the low light as well.

Poor light and irregular background

Similar to the previous setting, but the background field is also polluted by various visual parasites. This is to avoid.

Clothing

Prefer short, summer clothing showing arms and face clearly.

Framing

The video should at minima goes from the belly button to 10cm (3 inch) above the signer's head.
Framing should be consistent along a series.
Camera at shoulder or sternum level.

Conclusions

For best value for money investment, replicating the New York Times' settings seems the best option. It requires the following items :

• lights
• high level device holder
• proper camera lens (>= 8 Mpx)
• green screen background.

Icons have been recently updated (#39's phase 1 & phase 2).
The documentation can now be updated by redoing the screenshoots and spreading those to the relevant places.

Phase 3

The task below are good first issues.

• recreate documentation screenshots https://github.com/lingua-libre/SignIt/tree/master/doc
• update commons screenshots https://commons.wikimedia.org/wiki/Category:Lingua_Libre_SignIt
  • meta https://meta.wikimedia.org/wiki/Lingua_Libre/SignIt
  • wikidata https://www.wikidata.org/wiki/Q113354157

Phase4

This needs adminiship on Firefox store and is therefore assigned to @hugolpz

• on firefox addon store, replace illustrations on firefox addon store https://addons.mozilla.org/en-US/firefox/addon/lingua-libre-signit/

https://joss.theoj.org/papers/10.21105/joss.05811

Current situation

LinguaLibre SignIt is on mozilla addon store.
This addon is under 0x010C control and currently failling, likely due to migration from .fr to .org.
It is needed to get back control to redeploy.

Explore and learn

• Get account on Mozilla ➡️ hugo.lpz ...
• Explore developer tools, train, learn
• Upload an alternative version of SignIt.
• Settings page
• Published page
• Download that version, test.
• Edit, publish, test cycle.

Set up development suite

• Download FirefoxNightly - developers friendly
• Ubuntu Install FirefoxNightly. See here
• Allows addons from file : Open url about:config > search xpinstall.signatures.required : set to false
• Build SignIt addon : npm run webext
• Load SignIt addons : Open url about:addons > click the gear > click Install Add-on from File...
• Open url about:debugging#/runtime/this-firefox > Find SignIt : click "Inspect". It opens an extension inspecting tab.
• Test on Wikipedia
  • Couscous -> should show video
  • Changer -> should show videos
  • Chanter -> should not show a video because none recorded.

Get access

• Get a dev On wikimedia.fr account created, or similar. CC: @mickeybarber
• Message 0x010C
• Get access to Hugo ?

Thanks for adding this feature, it make it so easier to use!

Actual

I use other extension like signIt to translate selected text, and they are overlapping, below you can see La Bulle Elix is completely overlaping SignIt (we can see a part of the wing on the top left):

Expectation

The Simple Translate lets user choose where they want it display

Request

How feasible is it to be able to select the position?

Not sure that the right place to put it, feel free to copy/paste.

Comparison

Videos uploaded by Lingua Libre are approximately 5-10 times larger than the one I extracted and encoded from Laura Jauvert video (for the same duration and with a lower resolution).

Increase Quality

When I worked on the project I played a bit with FFMPEG options to get video as light and sharp as possible, i.e. with minimal blur on movement and small size as I targeted mobile usage.

My encoding script used the libvpx-vp9 codec and a bunch of other option for mobile (640x480) and desktop (1280x720).

Hope this could help

Experienced Extension dev's help required.

This is a Firefox/Chrome extension.
Its js scripts therefore works with special limitations.
This project creator is no more active.
A clarification of the overall flow of variables, scripts dependencies, scope access & limitations,
how to make script A available into script B, and how to declare those scripts with proper rights in manifest.json would help a lot.

Current usage

As a person that sign casually, when watching a new sign, I often find myself trying to decompose the movement, but at normal speed it's not easy.

Feature request

Having the control to slow down the video would be helpful. Either on a per video (inside the player) or for all videos

I noticed the following issues with the qqq.json file:

• Please remove the keys that don't have any content. Eg: "si-popup-settings-uilanguage-help":"",
• Remove key - "si-overlay-coreContent-right--wikt-section-id":"", that has no entry in the en.json file.

In french sign language, for example :

• j'aime/j'aime pas (i love/ I dislike)
• je veux/je veux pas (i want/ I don't want)

Here is a video, that introduce well LSF. Joël Chalude show the "irregular" verb in french sign language :
https://invidious.fdn.fr/b3v7o6Xi5Zs?t=104

So it bring another layer of difficulty as Sign It translate words by word. So i'm not sure if Sign It can detect a negative sentence, i don't know how it works on the technical side. For now, i can only think of a toggle above the video. Where we can switch to the negative verb. As for other sign language, i don't know if they have the same irregular verb.

@saltykheera , I encourage you to upload vector versions (svg) of your logo proposals to Wikimédia Commons. We can then store them in Category:Proposed_Lingua_Libre_logo. Let me know if I can assign you this ticket.

https://phabricator.wikimedia.org/T314253

Need to define issue better. It seems to work as intended within the top right popup. Search of accueil returns video of accueil (lieu). Maybe was just caching issue with the API.

Current intersection from (1) clicked-word via window.select() and (2) list of videos recordings is handled by function wordToFiles( word ).

This function is too restrictive and must be more inclusive.

May need edit 3 background-script.js sections :

• function wordToFiles
  • .hasOwnProperty usage may be to review
• var sparqlSignVideosQuery , to test in https://lingualibre.org/bigdata/#query
• function getAllRecords( signLanguage )

Additional cases to handle

Screenshot

As of Monday noon , it failed. (Recording on Saturday noon...)

Recorded video file exists, but SignIt don't find it !

In the 4th image, an other video (File:LL-Q33302_(fsl)-Taliba31-accueil.webm) is shown but only it. A second one (File:LL-Q33302_(fsl)-Hugo_en_résidence-accueil_(lieu).webm) should be available.

[EDIT]

As of Monday evening , it works. (Recording on Saturday noon...)

Participative listing

Other cases are observed to not work. You can add them in the table above to ease the work of the developer.

TLDR: Better support for homographs, words that write similarly but mean and sign differently, is needed.

Context :

Currently Lingua libre SignIt is limited to translation in sign language. It somewhat limit the tool to learning sign language for hearing people. I would like to make it useful for deaf people too.

Also there is lot words that will never have a translation : the, in, into, and...

Nor the correct sign language. Eg : To dive

We won't use the same sign depending of its meaning :

• To dive into a book : we will sign a book eating us
• To dive in a swimming pool : we will sign the movement
• To dive in the deaf world : We will sign the world eating us

Suggestion :

That's why i think the ability to add definitions in sign language can't be skipped. We also need an option for word that will never have sign, a image with "this word don't exist in sign language" and a link below that explain why. In fact, french sign language is very similar to chinese in some way.

In france, Elix has several definition translated in sign language. So most deaf people would use it daily.

Hence my suggestion, i would like to add several numberer tabs below the video with a subtitle as "Meaning n°"
So, when we click on those tabs, the definition translated in sign language would help a lot deaf people.

I hope it helped you.

[Edit by Hugolpz]

• Data : gather rich list with definitions
• Code : RecordWizard to handle rich data better 👉🏼 T212671
• Community: build collaboration to video record.

+ Testing requires: js skills ; physical web cams or digital cameras.

Problem

Videos uploaded to Wikimedia Commons seems limited to a default low quality : 640px and low quality / high compression. How to set higher standard ?

Tools

• Commons:Video#Video_formats https://commons.wikimedia.org/wiki/Commons:Video#Video_formats
• rw.VideoRecorder.js : where the code must be changed https://github.com/lingua-libre/RecordWizard/blob/master/modules/rw.VideoRecorder.js
• Plug in your web cam / digital camera, does the system :
  • recognize the new camera ? Tutorial: v4l2-ctl --list-devices (guvcview)
  • switch to that stream ?
• Test your camera capabilities https://webcamtests.com/resolution
• Test default and customized MediaRecord output https://jsfiddle.net/no9mfdc7/4/

Related to #11

• Enquire display bugs
• Redeploy.

Bug reported by @Pyb75

Sur lemonde.fr, le popup ne s'affiche pas correctement

Pyb75 bug suspected to be css related.

This is tricky and may require special training on extension debugging.

Given we work in the context of a Firefox/Chrome extension, some scripts don't have access to the i18n variables (banana) déclared in background-script.js, they are therefore hard coded in English or French. Most notably the file SignItCoreContent.js, which is called from popup/popup.js and content_scripts/signit.js can't access banana (is undefined), the i18n doesn't work there. I tried async + await loading as in popup.js, but switching the whole code to async create a result is non-structured-clonable data error. I was unsuccessful to provide banana . Maybe providing the banana as parameter would work.

This idea was floated by 0x010C.

• Add a toggle button in the setting
• In the page content, enlighten available words

https://stackoverflow.com/questions/67894682/how-to-add-style-for-specific-words-at-html-using-javascript
c6cf3ad - example of adding a setting

Current usage

When watching a new sign, I often play it several times to help me learn it.

Feature request

Let me replay video automatically so I can learn it more easily. Ideally, the 2nd play would be slower than the first one.

oojs.jquery.min.js removal in oojs v6.0

Oojs v.6.0.0 release note states states:

v6.0.0 / 2021-04-22
This release removes an (optional) dependency on jQuery.
[BREAKING CHANGE] release: Remove special "jquery" build (Timo Tijhof)

SignIt relies on npm oojs package and its oojs.jquery.min.js. Upgrading dependency to oojs 6.0.0 by Timo Tijhof removes this file, as declared.

Part of our build process is to copy oojs.jquery.min.js from oojs module to ./lib/.
But since oojs v6.0.0 this file is no more.
If oojs.jquery is missing from ./lib/ , the plugin fails to query contents.

Hack

I created and commited a js folder so the build pull oojs.jquery.min.js from there.
Few changes in the app js code should be sufficient to stop relying on this deprecate flavor of jquery and allow its removal from sign it.

Notes

I don't have the skills to inspect firefox plugins so I struggle to identify the code to change.