unhcr / dataviz-streamgraph-explorer Goto Github PK

Avoid this abrupt area ending:

This is the single area at the bottom of the page that shows the full extent of the data.

Tasks

• Add the context stream area, spanning all time
• Add a brush to the context stream
• Make the brushing zoom in the src and dest streams

Called contextStream here:

It would be great to avoid labels going off the edge like this:

The red areas here need to be added as SVG elements:

These SVG elements should change in size to respect the Semantic UI grid layout. I'm not sure exactly what kind of resize behavior we want. Perhaps for Desktop the SVG elements should reach exactly to the bottom of the display, and for mobile they should be a fixed aspect ratio and fill in the available width.

As mentioned by @vidonne , this is part of the recommendataions for print, but doesn't quite fit in with the Web version. Perhaps this part can be removed.

Currently, no selected src or dest leads to the field still being present in the URL, for example the default view http://localhost:8080/#dest&src&types=0-1-2-3-4-5-6.

Original Sketch

How do you best want to handle the mobile vs desktop logic in the code? In previous projects, I have used a function to detect mobile and then set a variable isMobile, which can be used as a conditional in other parts of the code, such as SVG resizing. Do you have another approach you would recommend?

This bar chart should update as the selected year updates.

This should get us the themed radio buttons and button style.

We can import it the same way this page does https://unhcr.github.io/dataviz-somalia-prmn/index.html

Called typeSelector in this diagram:

This should be a "component", meaning the DOM structure can be created via JavaScript, and there should be JavaScript hooks for

• Setting the set of toggled types
• Listening for changes to the selected types
• Listening for when the "Reset selection" button is clicked

This will provide the axes for the two StreamGraphs, and also will show the selected year and allow changing the selected year. For this we can use d3-boxes

Called timePanel here:

After adding labels, the performance slowed way down. The algorithm for positioning labels is time consuming. One thing we can do is first draw the StreamGraph areas, then after a second or so of no updates, then we do the label placement.

Let's adopt ReactiveModel as a dependency, and refactor what we have now to leverage it.

only show "top 20" if there were more than 20 originally. This label doesn't make sense if there are fewer than 20.

Ideally we'd pull the data we need when we need it, so we don't need to download a 2MB file on page load.

It might be a nice interaction to add - hovering on the bars cause the corresponding stream to pop out, and vice versa.

The type selection should feed into the query that goes to the API simulation, so that the filter gets applied to the data when the selection changes.

Currently the colors is nondeterministic - each color will be assigned to a country based on its order with all filters applied when the page is loaded, then those colors will be used as filters changed.

The colors should be consistently mapped to countries, no matter what filter state is present on page load.

Currently, areas are shown for all source and destination countries no matter what, even if they have all zero values or are too thin to be visible.

We can improve performance by not rendering areas that are too thin to be seen.

There are two cases to be addressed:

• When all sources and destinations are shown, we should limit the number of areas shown, and take, say, the top N (e.g. 100) areas in terms of their largest height (across all time points).
• When filters are applied (source, destination, types) and there are very few areas to show, we should not render areas that have a maximum height below a certain threshold of visibility. This means that sometimes, we may display even fewer than the top N areas.

Might it make sense to color the bars the same as the stream areas they correspond to?

It may improve readability.

Currently it's difficult to track down, this one https://unhcr.github.io/dataviz-streamgraph-explorer/

This should be front and center on the README.

It may be better if these areas expand, so you can read the labels:

called srcStream, destStream in this image:

when you click on a stream to drill-down, and then click to go back, it is a bit slow. could it be improved by keeping the default state in memory and avoiding re-calculations?

This issue is about setting up the infrastructure for JavaScript, and agreeing/deciding on code style.

I propose we use Webpack for bundling JavaScript from many source files into a single file that gets loaded on the page. There are many advantages of this over just using <script> tags, such as:

• We can import files as modules into other files, and the bundler will take care of combining them. So, we don't need to think about the order in which we include <script> tags.
• We only need to include a single <script> tag in our HTML, rather than many. This improves load time, and also makes it so we don't need to change the HTML each time we add a new JS file.
• We can import modules from NPM, such as the D3 modules, and ReactiveModel.
• We can use ES6 Language Features safely, by telling Webpack to output ES5 (old JavaScript standard defined in 2009), which runs in all browsers. These language features allow us to take advantage of incredibly useful and widely used advances in the JavaScript language, such as modules, template strings, arrow functions, and destructuring, and enhanced object literals which make it possible to produce more expressive, clean, and well organized code. While some of these things may be unfamiliar to our team, their benefits are immense, especially for a project like this one that will have a moderately complex JavaScript component.

Disadvantages of using WebPack:

• Developers will need to have NPM running on their machine.
• Deployment will depend on the generated JavaScript bundle, so to deploy we'll need to first run the build step, then deploy the files, rather than just deploy the content of the repository directly.

Depends on #41

I propose we work with reactive-model so we can code in terms of data flow, and not need to worry about the logistics of when different parts of our code execute.

Given that we'll use the packed 2MB data for now, and we'll use an API later, we need to design the data flow graph such that we can cleanly swap out the data source, and everything will still function correctly.

This is approximately how it might look:

Then we can write the visualization code to solely depend on srcDataAggregated and destDataAggregated, and make sure they get the same values no matter which data backend is used.

We need to think about the design for all cases:

• multiple src, multiple dest
• single src, multiple dest
• multiple src, single dest
• single src, single dest

Sketch for (multiple src, multiple dest) case:

Sketch for (single src, single dest) case:

Currently, hovering over areas does not update the selected year. This is a bit confusing, and perhaps a better interaction would be to have hovering over the areas set the selected year as well.

The concept of the details panel is to show details for a selected year. To select a year, we can add the following:

• Add year property to the data flow, representing the selected year
• Render a vertical line that shows the selected year
• Add hover interaction, so hovering in the timePanel sets the selected year
• Add hover interaction, so hovering in the background of the StreamGraph component sets the selected year
• Display the selected year as text in the details panel

Clicking on an area in the source StreamGraph should filter by that source, likewise for destination.

As seeing 0 in the list may be confusing for non-programmers

We need to get the data from data/data.json into the browser and unpacked into a flat table structure (array of objects).

Get the full timeseries data from http://popstats.unhcr.org/en/time_series , and preprocess it such that the file is small enough to load in the browser.

This should contain

• Year
• Population Type
• Origin (we'll call this "Source", or "src", in the code)
• Destination (we'll call this "dest" in the code)

The full data is available at http://popstats.unhcr.org/en/time_series.csv

• Add interaction where clicking "pins" the selected year (similar to the interaction in http://unhcr.github.io/dataviz-population-change/ )

Some changes to get watching to work left us in an unfortunate state of having built files in dist/dist rather than just dist.

There are a number of TODOs sprinkled throughout the code. At some point these all should be addressed by a refactoring effort.

• The URL should update to contain the zoomed region
• The zoomed region should update from the URL
• The brush selection should update from the URL
• The brush selection should be cleared if zoom is removed from the URL

So we don't need to run npm run build all the time.

We want to eventually pull aggregated data down from an API via queries, which will be much smaller than the 2.3MB file we have now. However, that API is not ready quite yet, so the best approach might be to simulate that API. That way, we can drop in the API easily later.

Approach idea:

• Use a Web Worker to load the full 2MB file.
• Treat the Web Worker as though it were the API - passing a query in, getting the aggregated data out.

In prototyping, the on-the-fly filtering and aggregation was quite expensive, causing the page to freeze briefly. Putting the computation inside a Web Worker will cause it to execute in a separate thread, so the UI will not freeze.

Semantic UI Default:

Desired:

Currently the list of selected types is bloating the URL. We can try to think of a more concise representation, maybe a list of array indices for example.

The wide bars are a bit awkward. For example when there are only 3-4 rows, it should be just empty space below the bars, rather than expanding the bars to fill the space available.

The URL of the page should contain the current selections to support deep linking, where you can share a link to a specific state.

This should eventually contain:

• types The array of selected population types
• src The selected origin
• dest The selected destination
• zoom The selected zoomed time interval
• year The selected year, which feeds into the details view

As one of the first things, let's set up a layout of nested boxes that changes depending on whether we're on a mobile device or not, and also responds to resize.

Here's a first pass from Trello by @vidonne

It should look like in this design mockup:

Deploy WIP version to GitHub pages or other solution so that we can share development progress with end-users.