Accurate text bounds determination is slow about scenery HOT 50 CLOSED

Another option:

• Ship SVG font information for whatever locale is being used, construct the full text path from the glyph paths (this would require porting the Unicode bidi and other rendering algorithms), and render as a path. This would increase the file size and possibly include some startup overhead (and somewhat slower Text rendering), but would allow exact bounds, rendering Text in WebGL, we could handle multiline properly, and we would have the same text rendering across platforms. The implementation would be simpler if we handle English-style text only.

from scenery.

I instrumented Util.canvasAccurateBounds to track the time it is taking for Forces and Motion Basics. On my hardware, I have that it takes about 4 seconds on Win7/Chrome and about 11 seconds on iPad3.

from scenery.

It sounds like this is a problem that's not going to be solved immediately. What do you suggest for a workaround in the meantime? Beer's Law Lab is currently unusable on tablets due to this issue, and it's preventing me from moving forward with adequate testing/debugging/optimization.

from scenery.

I was hoping to have a call with you to discuss options. I'm planning on setting up flags for the following options, and I'm not yet sure what to name them, or whether to split them into multiple options:

1. Use SVG (or DOM) "bounds" for text, and disable canvas rendering
2. Use SVG (or DOM) "bounds" for text, allow canvas rendering, but disable dirty region repainting (repaint entire layer at once)
3. Use canvas bounds for text (with major efficiency improvements, probably 4x), and allow canvas rendering. This is currently the only option.

I'd like to performance test to determine whether (1) or (2) is the best default, and I'm considering having a separate flags for something like "boundsQuality" (or even allow selecting from canvas/dom/svg bounds methods) in addition to a flag that would control the rendering type (and dirty region handling).

@samreid found some good reference implementations for Unicode bidirectional text, so I'd like to look into using that to turn text into paths (and verifying that this performance would be an improvement, although the cross-platform pixel accuracy would be nice to have).

from scenery.

One possibility for a short term solution is to render to the DOM using scenery/nodes/DOM. For instance, this version seems speedy enough on iPad:

    var element = document.createElement( 'p' );
    element.className="concentrationMeterReadout";
    element.innerHTML = new Number( 1 ).toFixed( VALUE_DECIMALS );
    var valueNode = new DOM( element );

...

// displayed value
    meter.value.addObserver( function ( value ) {
      if ( isNaN( value ) ) {
        element.innerHTML = NO_VALUE;
        valueNode.centerX = imageNode.centerX; // centered
      }
      else {
        element.innerHTML = value.toFixed( VALUE_DECIMALS );
        valueNode.right = imageNode.right - VALUE_X_MARGIN; // right justified
      }
    } );

with css

.concentrationMeterReadout{
  position: absolute;
  width:50px;
}

from scenery.

I am interested in spending a bit of time pursuing rendering our own text using svg style shapes. However, my main concern is how to create those shapes for every glyph in every language that we will ever want to support, and how to deliver in the sim downloads.

from scenery.

I'm imagining a build step that would identify every glyph needed, and pull the necessary info to render those glyphs in a bidirectional stream from http://www.unicode.org/Public/UNIDATA/ (probably from a local copy). We would then ship this combined information?

Alternatively, we could have pre-built files for each language that could simplify handling, possibly be dynamically loaded, and would allow better non-PhET usage of text.

from scenery.

I think we should keep in mind the possible future project of an interactive translation creator/editor. It is difficult to image how the "just bundle in the needed glyphs" strategy would work for an interactive translation creator since the translation author will be adding new glyphs that were not bundled into the sim.

from scenery.

For handling the glyphs, we may also need to handle different fonts for different languages as we currently do for simulations. Temporarily we can go with routes that will ship more font information, and optimize it later.

from scenery.

SR wrote:

I am interested in spending a bit of time pursuing rendering our own text using svg style shapes.

This sounds like a "last resort" solution to me, and I can envision a whole range of associated headaches. Hard to believe that we're developing for a platform where we need to invent our own text-rendering solution, but here we are. Someone needs to push hard for decent text-bounds support.

from scenery.

SR wrote:

One possibility for a short term solution is to render to the DOM using scenery/nodes/DOM.

Not a viable workaround until scenery #13 is resolved.

from scenery.

I tried the code from my comment above and it was very speedy on the iPad. It looked like a viable workaround to me. Did you try it on the iPad? Perhaps it is unrelated to #13

from scenery.

It may work in specific cases, but it is not a general workaround for text. I'm not going to convert all of my Text nodes to DOM nodes when I know that DOM nodes split text across lines and can't be dragged without lag time.

from scenery.

SR wrote:

Did you try it on the iPad?

Yes, I did. Performance is usable. But positioning of the DOM node is off on all platforms, "valueNode.centerX = imageNode.centerX" does not center-justify the value.

from scenery.

Decided in meeting to use option (1) as default, with selection between the three by a single parameter.

Will also add DOM support to Text, and a property of htmlText will be added to support direct HTML text. Additionally, performance for the Canvas-based bounds will be improved.

Multiline support is planned for a 'node.multiline' boolean flag (to be designed later), use htmlText for now with styling.

from scenery.

Text.boundsMethod flag is completed.

from scenery.

Text.isHtml has been added, along with Text DOM support.

It basically supports explicit newlines (br elements), but not implicit newlines (wrapping). I'm considering a flag like textWidth or multilineWidth that would cause wrapping (and would also force the DOM renderer).

It is now technically possible to set a renderer explicitly and then later change options to make that renderer unsupported (this currently will throw an Error on the change). Additionally, it's possible to combine isHtml and text strokes or incompatible (gradient/pattern) fills such that no renderer is supported (isHtml forces DOM, but DOM renderer doesn't support stroked text or advanced fills). WebKit has text-stroke, but it's not standardized anywhere, so I'm leaving out due to partial browser support.

I'd love to hear suggestions on the API.

from scenery.

The API for Text.isHtml looks good to me, after reviewing the sample usage here http://phetsims.github.io/scenery/doc/#text-isHtml and your notes above. I guess I tend to type isHTML instead of isHtml, but I don't feel too strongly about it--just brought it up in case other agree and wish to change it.

from scenery.

I'm fine with isHTML, especially if there is a consensus. It's a bit harder to type, but easier to read.

from scenery.

The best practice for how to handle acronyms in names is typically to follow the convention used by other APIs in your programming environment. JavaScript APIs are very inconsistent about how they handle acronyms. One of my favorites is XMLHttpRequest, 2 acronyms handled differently. (WTF? or wtf?) It does seem like they typically uppercase "HTML" (eg HTMLElement), so perhaps we should stick with that.

from scenery.

JO wrote:

I'm considering a flag like textWidth or multilineWidth that would cause wrapping (and would also force the DOM renderer).

I wouldn't bother with that feature. If you do, then you're going to have to address justification, too. If someone wants control over mutliline and justification, they can use HTML. No need to duplicate this functionality in Scenery.

from scenery.

If someone wants control over mutliline and justification, they can use HTML.

Yes, they could use a DOM node, set up a div, give it a specific width, set the CSS font to be the desired font, set the CSS color, and call invalidateDOM any time the text changes. It seems simpler to add a flag to control the wrapping width.

from scenery.

I'd like to express a personal preference for the "isHtml" style for acronyms. Since, as CM pointed out, JavaScript is inconsistent on this front, we should be able to choose our preference. The reason I prefer this approach is that it is easier to visually parse when the acronym borders another word, e.g. isHtmlRequest vs isHTMLRequest. And it's easier to type.

from scenery.

Not sure who made the comment about "isHtml" (you're logged in as phetsims). My point was that JS is inconstent in general with how it handles acronyms. But it's consistent about how it handles HTML, XML, Http, etc. And I think we should follow that.

from scenery.

Ah, so we should be consistent with the inconsistent acronym treatment seen in JavaScript (the previous phetsims comment was from me). Seems like a perpetuation of a bad decision to me, but I don't really care that much. I can live with either isHtml and isHTML, and all that falls out of the decision.

from scenery.

isHtml flag is now inaccessible from Text, it's called isHTML internally, and is only set from the type HTMLText (which should be used for that purpose from now on).

Still leaving this issue open for the speedup of the accurate canvas bounds, and possibly text wrapping support

from scenery.

Was fed up with text bounds today, decided to try a proof-of-concept based on Emscripten (see http://www.colorado.edu/physics/phet/dev/olsonjb/text-bounds-test-1/test.html, takes a few seconds to load due to download. Then try typing in the text box on the upper-left).

Although it's not optimized (and is lacking ICU and --with-harfbuzz for FreeType), it runs fairly quickly. Currently, the non-font data takes up 2.5 MB, so that would need to be significantly cut down. I didn't run -O2 on the libraries, so that + dead code elimination should probably shave off a large majority.

The font data (all of the basic Arial, but not the unicode-complete 22MB version) is taking ~2.2MB due to inefficient encoding (should be closer to ~700kb for the font). If we can strip out unused glyphs, this could be knocked down to probably under ~100kb.

from scenery.

The bounds test above looks good, I'd like to hear more about how it was implemented and more about the tradeoffs it implies.

from scenery.

See the new branch https://github.com/phetsims/scenery/tree/harfbuzz-test (it forces Canvas, since I only implemented the glyph shape display). It un-bolds some things (since it doesn't have the bold face embedded). Performance takes a bit of a hit since we aren't rasterizing them before displaying, but with image down-scaling quality across browsers, we'd probably have to mip-map things (don't have time to test that).

Text display seems to be better while animating (no hinting issues), and it seems better than Chrome's current hinting issues (that cause the jiggle). Hinting would be possible, but difficult and slower than the current non-hinting prototype.

Chrome SVG:

Chrome Harfbuzz (looks better to me):

Firefox SVG:

Firefox Harfbuzz (hinting could help here, since it's not moving):

from scenery.

For something with wide language support, we may want to consider http://www.google.com/get/noto/#/

from scenery.

Also mentioned from a thread on the subject about why there is a font per language (good general information):

There are technical reasons for this. For example m, OpenType only supports 64k glyphs within a font. Which is enough for the BMP but nothing else (and counting ligatures that are necessary for, e.g. Arabic, it might not even suffice for the BMP).
Then there are practical considerations. While Latin, Greek and Cyrillic are similar enough to warrant the same styles (serif, sans-serif, script, italic, and various weights) not all of them make very much sense for, say, CJK or a variety of other scripts. So having different fonts for different scripts that are still designed to go together is actually not that bad a solution.

ygra from https://news.ycombinator.com/item?id=8040226

from scenery.

Deferring, memory/startup-time/font handling implications are big, and accurate bounds aren't needed as often as originally anticipated.

from scenery.

Today I saw https://github.com/devongovett/fontkit which purports to load font files, render as glyphs, provide accurate bounds, etc.

Shipping desired fonts is one path to getting reproducible playbacks across platforms.

from scenery.

Shipping fonts sounds like a nightmare for i18n.

from scenery.

Today I saw https://github.com/devongovett/fontkit which purports to load font files, render as glyphs, provide accurate bounds, etc.

Looks nice, like a more fleshed-out version of the Scenery harfbuzz branch that is capable of loading fonts, rendering and doing exact metrics. Much less hacky.

Shipping fonts sounds like a nightmare for i18n.

Yup, that's really the reason I haven't further investigated this.

from scenery.

Shipping fonts sounds like a nightmare for i18n.

Yup, that's really the reason I haven't further investigated this.

Yes, I agree that "nightmare" is the appropriate word here. But on the other hand, Noto sans supports 581 languages. On the other hand, it weighs 449KB.

from scenery.

Pushed an improvement for 'accurate' bounds for Text above, browser notes show it being compatible. I'm kicking off testing with http://localhost/scenery/tests/sandbox.html?script=const%20flags%20%3D%20%7B%0A%20%20approximateSVGBounds%3A%20false%2C%0A%20%20approximateDOMBounds%3A%20false%2C%0A%20%20accurateCanvasBoundsFallback%3A%20false%2C%0A%20%20approximateImprovedDOMBounds%3A%20false%2C%0A%20%20accurateCanvasBounds%3A%20true%2C%0A%20%20approximateHybridBounds%3A%20true%0A%7D%3B%0A%0Aconst%20unicodeTestStrings%20%3D%20%5B%20%22A%22%2C%20%22b%22%2C%20%22m%22%2C%20%22x%22%2C%20%27Q%27%2C%20%22A%5Cn%5C%5Cn%22%2C%20%22A%5Cr%5Cn%5C%5Cr%5C%5Cn%22%2C%20%22A%5Ct%5C%5Ct%22%2C%20%22%5Cu222b%22%2C%20%22%5Cufdfa%22%2C%20%22%5Cu00a7%22%2C%20%22%5Cu00C1%22%2C%20%22%5Cu00FF%22%2C%20%22%5Cu03A9%22%2C%20%22%5Cu0906%22%2C%20%22%5Cu79C1%22%2C%20%22%5Cu9054%22%2C%20%22A%5Cu030a%5Cu0352%5Cu0333%5Cu0325%5Cu0353%5Cu035a%5Cu035e%5Cu035e%22%2C%20%220%5Cu0489%22%2C%20%22%5Cu2588%22%20%5D%3B%0Aconst%20styles%20%3D%20%5B%20%7B%20fontSize%3A%2060%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27serif%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27monospace%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27cursive%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27fantasy%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27Georgia%2C%20serif%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27%22Palatino%20Linotype%22%2C%20%22Book%20Antiqua%22%2C%20Palatino%2C%20serif%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27%22Times%20New%20Roman%22%2C%20Times%2C%20serif%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27Arial%2C%20Helvetica%2C%20sans-serif%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27%22Arial%20Black%22%2C%20Gadget%2C%20sans-serif%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27%22Comic%20Sans%20MS%22%2C%20cursive%2C%20sans-serif%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27Impact%2C%20Charcoal%2C%20sans-serif%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27%22Lucida%20Sans%20Unicode%22%2C%20%22Lucida%20Grande%22%2C%20sans-serif%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27Tahoma%2C%20Geneva%2C%20sans-serif%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27%22Trebuchet%20MS%22%2C%20Helvetica%2C%20sans-serif%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27Verdana%2C%20Geneva%2C%20sans-serif%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27%22Courier%20New%22%2C%20Courier%2C%20monospace%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27%22Lucida%20Console%22%2C%20Monaco%2C%20monospace%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontWeight%3A%20%27900%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontWeight%3A%20%27900%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontStretch%3A%20%27ultra-condensed%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontStretch%3A%20%27ultra-expanded%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontStyle%3A%20%27italic%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontStyle%3A%20%27oblique%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20fontFamily%3A%20%27serif%27%2C%20textAlign%3A%20%27end%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20direction%3A%20%27rtl%27%20%7D%2C%20%7B%20fontSize%3A%2060%2C%20direction%3A%20%27rtl%27%20%7D%20%5D%3B%0A%0Aconst%20getTextDemo%20%3D%20(%20string%2C%20textOptions%2C%20scale%20%3D%201%20)%20%3D%3E%20%7B%0A%20%20const%20text%20%3D%20new%20Text(%20string%2C%20textOptions%20)%3B%0A%20%20%0A%20%20return%20new%20Node(%20%7B%0A%20%20%20%20scale%3A%20scale%2C%0A%20%20%20%20children%3A%20%5B%0A%20%20%20%20%20%20...(%20flags.approximateSVGBounds%20%3F%20%5B%20Rectangle.bounds(%20TextBounds.approximateSVGBounds(%20text._font%2C%20text.renderedText%20)%2C%20%7B%20fill%3A%20%27rgba(0%2C0%2C255%2C0.3)%27%20%7D%20)%20%5D%20%3A%20%5B%5D%20)%2C%0A%20%20%20%20%20%20...(%20flags.approximateDOMBounds%20%3F%20%5B%20Rectangle.bounds(%20TextBounds.approximateDOMBounds(%20text._font%2C%20text.getDOMTextNode()%20)%2C%20%7B%20fill%3A%20%27rgba(255%2C0%2C255%2C0.3)%27%20%7D%20)%20%5D%20%3A%20%5B%5D%20)%2C%0A%20%20%20%20%20%20...(%20flags.accurateCanvasBoundsFallback%20%3F%20%5B%20Rectangle.bounds(%20TextBounds.accurateCanvasBoundsFallback(%20text%20)%2C%20%7B%20fill%3A%20%27rgba(255%2C255%2C0%2C0.3)%27%20%7D%20)%20%5D%20%3A%20%5B%5D%20)%2C%0A%20%20%20%20%20%20...(%20flags.approximateImprovedDOMBounds%20%3F%20%5B%20Rectangle.bounds(%20TextBounds.approximateImprovedDOMBounds(%20text._font%2C%20text.getDOMTextNode()%20)%2C%20%7B%20fill%3A%20%27rgba(0%2C255%2C0%2C0.3)%27%20%7D%20)%20%5D%20%3A%20%5B%5D%20)%2C%0A%20%20%20%20%20%20...(%20flags.approximateHybridBounds%20%3F%20%5B%20Rectangle.bounds(%20TextBounds.approximateHybridBounds(%20text._font%2C%20text.renderedText%20)%2C%20%7B%20fill%3A%20%27rgba(0%2C0%2C255%2C0.3)%27%20%7D%20)%20%5D%20%3A%20%5B%5D%20)%2C%0A%20%20%20%20%20%20...(%20flags.accurateCanvasBounds%20%3F%20%5B%20Rectangle.bounds(%20TextBounds.accurateCanvasBounds(%20text%20)%2C%20%7B%20fill%3A%20%27rgba(255%2C0%2C0%2C0.3)%27%20%7D%20)%20%5D%20%3A%20%5B%5D%20)%2C%0A%20%20%20%20%20%20text%0A%20%20%20%20%5D%0A%20%20%7D%20)%3B%0A%7D%3B%0A%0A%2F%2F%20Allow%20scrolling%0Aif%20(%20display._input%20)%20%7B%0A%20%20display.detachEvents()%3B%0A%7D%0A%0Ascene.addChild(%20new%20VBox(%20%7B%0A%20%20spacing%3A%2010%2C%0A%20%20align%3A%20%27left%27%2C%0A%20%20children%3A%20styles.map(%20style%20%3D%3E%20new%20HBox(%20%7B%0A%20%20%20%20spacing%3A%203%2C%0A%20%20%20%20children%3A%20unicodeTestStrings.map(%20string%20%3D%3E%20getTextDemo(%20string%2C%20style%2C%201%20)%20)%2C%0A%20%20%20%20align%3A%20%27origin%27%0A%20%20%7D%20)%20)%0A%7D%20)%20)%3B%0A.

I believe it's more accurate than the alternative on everything (although Safari is a bit less-than-ideal on crazy non-realistic things).

from scenery.

from scenery.

from scenery.

from scenery.

Windows 10 Chrome (one accent goes slightly out but it is quite accurate).

from scenery.

Windows 10 Firefox (similar thing)

from scenery.

Windows 10 Edge

from scenery.

@zepumph do you mind reviewing, and seeing if the boundsMethod: 'accurate' works for your use case?

from scenery.

Wonderful! I'll note over in phetsims/friction#330. Could you please note why it isn't worth turning this on by default? Is this potentially fast enough to be our default bounds operation for TextBounds?

from scenery.

Usually for layout purposes, we'll care more about the consistent typical ascender/descender box (so that the baseline of the text is at the same place). We'd need to change a lot of layout code if that became the default (lots of align: 'origin') just to align text properly. Additionally, we usually care about centering vertically the "common" part of the text, and not having the baseline jump around.

There's a slight argument for adjusting the horizontal dimensions of the text to use this method, although some code might depend on x=0 being the left of the default text bounds.

@zepumph thoughts?

from scenery.

I am seeing some broken unit tests on CT that may be related to this issue:

scenery : top-level-unit-tests : unbuilt
https://bayes.colorado.edu/continuous-testing/ct-snapshots/1675257245304/scenery/scenery-tests.html
389 out of 390 tests passed. 1 failed.
Trail: Consistent and precise bounds range on Text failed:
[x:(-4,10),y:(-11,3)]
at Object.<anonymous> (https://bayes.colorado.edu/continuous-testing/ct-snapshots/1675257245304/chipper/dist/js/scenery/js/util/TrailTests.js:411:10)

Trail: Consistent and precise bounds range on Text failed:
precision: undefined
at Object.<anonymous> (https://bayes.colorado.edu/continuous-testing/ct-snapshots/1675257245304/chipper/dist/js/scenery/js/util/TrailTests.js:413:10)

id: Bayes Puppeteer
Snapshot from 2/1/2023, 8:14:05 AM

from scenery.

#1527 indicates the above was fixed.

from scenery.

There's a slight argument for adjusting the horizontal dimensions of the text to use this method, although some code might depend on x=0 being the left of the default text bounds.

I'd say this is worth a side issue that can do some pixel comparison testing ONLY if you think so. Feel free to close.

from scenery.

I think I'd like to keep as-is. Closing!

from scenery.