xability / maidr Goto Github PK

When help menu is displayed, "document" role must work as tagged to activate screen readers' virtual cursor (AKA browse mode). However, currently it does not take any effects. Usually, assigning tabindex 0 with "document" does the trick, but it does not work this time. Probably, it inherits from "dialog" ARIA role, but it was not the issue b/c removing the "dialog" did not make it work.

I know there is a tradeoff, but please bind all the keydown events to svg units instead of the whole document. Because we will likely embed more than one graph on the same page and the document-level key bindings can cause some issues. We may want to isolate them within an SVG object.

• Boxplot definition: https://www.khanacademy.org/math/statistics-probability/summarizing-quantitative-data/box-whisker-plots/a/box-plot-review

• Up and down arrow keys move between different boxplot groups.

• Left and right arrow keys move section by section within selected boxplot.

I will add thorough description on the system design soon.

Reproducible Steps

1. Open scatterplot.html.

2. Focus in the graph area.

3. Hit page-up key to go to the second layer (the smoothed line).

4. Press Ctrl+Shift+Left-Arrow to autoplay.

5. While the autoplay is working, press Ctrl+Shift+Right-Arrow to play the other direction: two autoplays get played at the same time.

Expected Behavior

The last autoplay needs to kill the previous autoplay job.

1. When the number of outliers is 1, the current method is just omitting the number; however, please indicate the number like below:

1 Lower Outlier 126.18

1. For empty outliers, please do the following:

0 Out Lier

While addresing #61, please also fix the following issue.

Reproducible Steps

1. Open boxplot.html

2. Press arrow keys to go to upper outlier area of suv.

3. Hit down-arrow key: You will see:

2 subcompact Upper Outliers 590.97, 639.05

This needs to be fixed as:

subcompact, 2 Upper Outliers 590.97, 639.05

1. In the current location (upper outliers of subcompact), hit up-arrow key: You will see:

5 suv Upper Outliers 302.48, 318.51, 334.53, 350.56, 366.59

This needs to be fixed as:

suv, 5 Upper Outliers 302.48, 318.51, 334.53, 350.56, 366.59

After multiple experiment on my end, I have found that double lines are more pronounced. The following includes my suggested logics. I will share more ideas on some thresholds.

left_boxside = median - Q1

right_boxside = Q3 - median

if (left_boxside == right_boxside) {
    box_brl = "⠿⠸⠇⠿"
} else if (left_boxside < right_boxside) {
    box_brl = "⠿⠸⠇⠿⠿"
} else {
    box_brl = "⠿⠿⠸⠇⠿"
}

lower_whisker_length = Q1 - min
upper_whisker_length = max - Q3

if (lower_whisker_length == upper_whisker_length) {
    "⠒" + box_brl + "⠒"
} else if (lower_whisker_length < upper_whisker_length) {
    "⠒" + box_brl + "⠒⠒"
} else {
    "⠒⠒" + box_brl + "⠒"
}

In boxplot, changing group refreshes the cursor location, defaulting to min area instead of preserving the current stat area. For example, if the current cursor is at 25% of group A, users may expect to land in 25% of group B when they press down arrow key.

1. Alt+Shift+Left-Arrow: From the beginning to the current position.

2. Alt+Shift+Right-Arrow: From the end to the current position.

In barplot and other graphs, autoplay (ctrl+shift+arrows) does not work with braille mode being turned on.

In boxplot.html, application role is not activated in the svg area.

Boxplot empty cell sound is not well pronounced because it uses low pitch and may be confusing. Let's keep using the same empty sound as the one used in heatmap.

We currently use min frequency, and sometimes custom tweak the oscillator type, but we should have a unique repeatable null sound used everywhere that we can tweak to improve UX.

For lack of better word, boxplot empty cell sound is a little bit annoying. Could we explore other smoother sound going forward?

While hosting our project on GitHub, we may want to publish our solution to NPM as an open-source library at some point so that people can download and import our library.

When landed in either lower or upper outlier region, it would be great to announce the total number of outliers like below:

{total_number} upper outliers: . . .

I have pushed a heatmap svg sample file at accessible-graphs/static/heatmap.svg.

With this sample file, please make a heatmap.html file just like accessible-graphs/barplot.html:

• Visual highlight moves along with left/right/up/down arrows.

• Play audio tone that addresses the following

  1. left and right stereo panning for x-axis.

  2. Similar to left and right panning mode, make sound moves along up and down (not the pitches, just the spatial audio) along with y-axis.

  3. different pitches for values (higher pitch for higher value; lower pitch for lower value).

For the log tracker feature, we need to host a server. I will take care of this.

Reproducible Steps

1. Load NVDA or any screen reader.
2. Launch boxplot.html.
3. Tab to focus on the svg area.
4. In terse mode, move your point by pressing left and right arrows: You don't hear group label, which is correct.
5. This time, however, press b to activate braille mode and do the same thing (pressing left or right arrows): You would hear redundant group label even though it's still in terse mode.BB

[JooYoung will take care of this issue]

It would be great if we could bypass some screen reader's virtual cursor conflicting keys. See yihui/xaringan#262

CC @ellvix and @yilinxia

I suggest we reword the following terms in boxplot:

1. Minimum -> Low

2. Maximum -> High

This is because these terms are misleading. The ending points of left and right wiskers actually are not matched with data min and max values.

Here is a formula how low and high (i.e., the ending point of both wiskers) values are calculated:

• High: $(IQR * 1.5) + Q3$

• Low: $(IQR * 1.5) - Q1$

Outliers are any values outside either low or high values.

It seems like the braille cursor position is one column behind the visual highlight.

The braille display works nicely. Great job! The only issue I have found is that the braille cursor locations are not precisely coupled with its corresponding sound pitches. When I press left and right arrows, cursor location moves along with braille columns. However, this cursor location is one column after its corresponding sound. For example, The highest tone/y_value in our sample svg file is at the far right. This pattern is also represented in braille unicode, but I here the highest tone when my braille cursor is one column after the target location.

As we discussed today, please implement an event-driven log tracker.

• Log tracker needs to get started as a page is loaded.

• We may want to bind log tracker to each SVG object.

• Default log file format: JSON

• Use F10 to bring up "Save As" dialog box for exporting log file.

• Use sessions and cookies to address multi user access.

• In our user studies, we will embed each plot in separate pages, which means we may need to append multiple log data from the same user agent until they explicitly press F10.

• Structure (please feel free to add more if any):

1. graph type

2. timestamp

3. keystroke

4. current modes (BTS)

5. misc settings (like custom frequency range, volume, braille display size, etc)

6. current position on chart

7. user OS platform info

8. user browser and version

Reproducible Steps

1. Open heatmap.html.
2. Tab to focus on svg area.
3. Turn on braille mode by pressing b.
4. Place your cursor in the second column in any row.
5. Press up arrow to go to the row 1 and column 2.
6. Press up arrow again to see if the cursor moving is blocked or not: currently, the cursor goes to the row 1 and column 1 instead of stopping there. This issue can be found when you reach the row 3 and column 2 and press down arrow key. Instead of stopping there, your cursor will move to the right (row 3 and column 3 location).

Also CC'ing @ellvix

We need Ctrl+Home key for going to the very first element and Ctrl+End key for going to the very last element in graphs. This is useful for two-dimensional graphs, such as heatmap and boxplot. However, please make these keys work for other types of graphs including barplot and scatterplot as a general solution. I know, in barplot and scatterplot, these keys' behaviors would be exactly the same as Ctrl+LeftArrow and Ctrl+RightArrow, but we need to make this generalizable so that we can be prepared for any types of graphs going forward.

@yujuny2 -- I have just uploaded the following scatterplot SVG:

1. scatterplot/scatterplot_no_jitter_point_only.svg

2. scatterplot/scatterplot_no_jitter_with_bestfit.svg

Please work on these files instead of the old one because these did not use jitter noise. The only difference between 1. and 2. is whether there is an added best line or not.

FYI, the backend raw data that I previously provided in json format as well as HTML are still valid because I created them without jitter noise at that time. I guess this no jittered SVGs would be more similar to the raw data.

It would make more sense that we turn off braille mode when it loses its focus. Could you please add event handler to the braille div for "focusout" event?

Currently, oscillator play is triggered via spacebar keydown event. This would cause some issues when there are multiple SVG plots included on the same page. After addressing #2, please modify this in the following ways:

1. Add tabindex="0" attribute to each svg objects via setAttribute() method.

2. Make oscillator event handler work against focus event (when the current SVG object gets focused) instead of spacebar keydown event.

I have finally been able to do reverse-engineering jobs to obtain the real data for both point and smoothed line layers within scatter plot. Please use the following data for scatter plot.

1. Point layer (I know you already used the raw data for point values, but this is the exact one. Please replace): scatterplot/layers/point_layer.json

2. Smoothed line layer: scatterplot/layers/smooth_layer.json

Please note: if there are multiple values on the same spot, use the larger volume to indicate the weight. For example, let's say there are x: 2, y: 4 and x: 2, y: 4 (the same values on the same spot). Sighted people would see them thicker, but blind people cannot see them. Therefore, please use larger volume parameter as a way of weighted value.

Toggle focus between braille output div and user's previous location (on the chart) with B key.

I think this is an inadvertent regression after we patch aria role bound to SVG element.

Reproducible Steps

1. Open any graphs, such as boxplot.html or barplot.html

2. Tab to focus in SVG object.

3. Press arrow left and right to move around.

4. Press B to turn on braille mode.

5. Turn off braille mode by pressing B key.

6. See if your focus goes back to your original position: it does not.

• Notes: we still need to bind application role and tabindex to SVG object instead of its parent div container because chances are we may embed more than one SVG object on the same web page.

For Mac / PC switcher: use navigator.platform or scrape from navigator.userAgent, because navigator.userAgentData.platform does not exist on all browsers (only exists for Chrome and Edge).

Investigate smooth audio for scatterplot curve. Try AudioParam.setValueCurveAtTime()

When navigating a stat vertically across groups, sometimes focus goes to another stat category.

Reproducible Steps

1. Open boxplot.html

2. Focus on low (AKA minimum) of "suv"

3. Keep pressing down-arrow keys (autoplay does not have this issue so please test with manual arrow keys).

4. If you reached the low of "2seater" without any issues, keep pressing up-arrow multiple times to go back to the suv low.

5. Repeat this up and down round trip multiple times: you may notice that your focus sometimes moves to lower outlier group out of the blue.

Currently, all the keydown events and their corresponding event handlerss are bound to window object. This is a great start. However, what if there are more than one svg objects contained in the current web page? addEventListener() will be triggered against when the whole browser window is loaded. Probably, a better approach would be binding these events and handlers to svg object unit instead of the window object. This way would allow users to interact with any number of svg objects in the current window.

1. Singular and plural are flipped around.

If you turn on verbose mode and navigate "suv" lower and upper outliers, you would hear/see the following:

Lower Outliers are 126.18

5 Upper Outlier is 302.48, 318.51, 334.53, 350.56, 366.59

First, the plural and singular verbs are unmatched with the the number of outliers.

Second, no information on the number of "lower outlier". Currently, it displays as:

Lower Outliers are 126.18

However, the expected outcome would be:

1 Lower Outlier is 126.18

2. The number of outliers are incorrectly appended in verbose mode

1. In verbose mode, go to upper outlier area in "suv"

2. Arrow down to focus on the upper outlier of "subcompact": you would hear and see the following:

class is subcompact, Upper Outlier2 is 590.97, 639.05

However, the expected outcome is as follows:

class is subcompact, 2 Upper Outlier are 590.97, 639.05

Autoplay is not currently supported in boxplot. Go to the first and last location (Ctrl+arrows) does not work, either.

Currently, the autoplay speed controls (faster and slower) only work for scatterplot; however, please make this work for all types of graphs we support.

Also, please replace the current shortcut keys with the following:

• Faster: period key (.)

• Slower: comma key (,)

At some point in the near future, we need to integrate our interface with Chart2Music. To do this, we need:

• Create a JSON-looking general API like C2M that does not necessarily depend on visual formats.

• Allow users to fall back to C2M for any graph types we don't support, such as candlestick plot, band plot, etc.

Currently, we obtain boxplot stat values from svg object. However, we may want to use exact values exported from R back end: boxplot_data.json

In any graphs and in any position, press CMD+Shift+LeftArrow or CMD+Shift+RightArrow to start autoplay.

When autoplay is on, text mode needs to be muted so that our users only hear the sonification sound. However, autoplay triggers the current position in text output. Please turn off the text mode as soon as the autoplay key is pressed and turn it on again right after the autoplay is finished.

It would be nicer if ESC key event can close the help menu dialog.

Reproducible Steps

1. Load NVDA or any screen reader.
2. Launch boxplot.html.
3. Tab to focus on the svg area.
4. In terse mode, move your point by pressing left and right arrows: You don't hear group label, which is correct.
5. This time, however, press b to activate braille mode and do the same thing (pressing left or right arrows): You would hear redundant group label even though it's still in terse mode.BBeeee

Currently, pressing arrow keys or any other keys trigger help menu.

CC'ing @ellvix as well.

This is a known issue, but we need to track the progress. Currently, autoplay doesn't work in heatmap while braille mode is turned on.

Autoplay response speed is a little bit slow. I think there is about one second lag to respond after pressing autoplay keys (Ctrl+Shift+LeftArrow or Ctrl+Shift+RightArrow). We may want to make the response rate faster.

Add Command + key for mac, mirror current Ctrl + key for PC controls.

Try using navigator.userAgent or navigator.platform to detect OS and switch both controls for all plots, and key control info on help menu.

Currently, the first landing point is the very bottom gorup instead of the top one. Given that the screen reader's reading order is top-to-bottom and left-to-right, it would be more natural to set the first focus to the first top group and at the lower outlier.

In the long run, we need to switch the current project into Typescript for better scalability, compatibility, and maintenance.