@thoat started this.

Add most recent work to eslintignore and fix it gradually.

Questions:

• Where is the lookup to country to datasets

When saving a map, a user should be able to determine if the map should be auth protected or if it is available to the public. As a bonus, build a check or flag that prevents a user from saving a map as publicly accessible if it points to a dataset that is marked as sensitive.

The user should also be able to see a list of their saved maps. The list might the name of the map and/or include a search feature. From this list, it should be easy to share the URL.

Test datasets and sample maps will be made available on Azure storage or via a drive

moved here

Why do it?

• On load, kepler should fetch config.json from public dir and load into map.

References

• https://github.com/unicef/magicbox-maps
• https://medium.freecodecamp.org/how-to-make-create-react-app-work-with-a-node-backend-api-7c5c48acb1b0
• https://github.com/NewGyu/react-tutorial-with-create-react-app

Use docker-compose to run Auth app, Kepler client, and backend.

When user logs in to AAD app, then the kepler app is beamed in through an iframe. Currently the back-end serves the config.json, but we might switch out backend for direct data exchange with Azure and external APIs. Let's discuss tomorrow.

"Add data to Map" to should not display by default on load.

Log scale for anything to do with aggregation of networks to polygons.

Magic box currently serves

• admin boundary files with population - topojson
• healthsites.io facilities - CSV

When user saves config, the entire config, with data, gets saved on server. Haven't decided what to do with files user drops into kepler, but above data sets shouldn't be copied.

Need to save pointer to dataset, and recreate when config is fetched on front-end.

What type of data to be saved?

Maps - kepler exports maps into a file called config.json. It has three nodes:

• datasets: an array of datasets
• config: the layer configurations which contain dataset names (dataId to be exact, which references the id value inside the datasets node. See Kepler.gl docs here and here).
• info: metadata about the file itself.

Where should the data be stored?

UNICEF uses Azure, so many static files are in Azure blob or file storage. Here is an npm module to interact with Azure.

What functionality should the feature have?

When a user logs in to create a map, a function should check to see if they have a storage folder already in Azure storage. If not, one should be created with their username. The user should be able to create a visualization and then name and save the map to this folder when it is completed.

As a bonus, create an autosave feature that saves an untitled map upon creation and periodically throughout (every x seconds or minutes, for example) and/or create an alert that would remind user to save before closing out the browser window.

Test datasets and sample maps will be made available on Azure storage or via a drive

When user selects borderfile, try to fetch healthsites for that country by default, and load as separate dataset.

Add this feature by creating a feature branch off dev.

Currently, the auth wrapper is the out-of-the-box create-react-app App.js. We would like to update this to make it part of the Magic Box Maps product. This would serve as the log in page for UNICEF staff and the public-facing interface of the product.

Expected outcome

• Log in (currently just for UNICEF staff; possibly part of a sticky header so it is accessible at all times)
• Scroll for content explaining the context of Magic Box and about the maps tool
• Contact form/footer so interested individuals/partners can contact us

In ./client/config.js, add item to indicate whether 'show data table' for all data sets should be included in UI.

In modal, make "Add Magic Box data" the first frame, and "Drop in data" the second.

Link in first modal to second might read, "Add your own datasets"

When "display-share-button" is set to false in ./client/config.js, it should not be displayed in kepler app.

What types of data?

• Shapefiles
  • We have GADM shapefiles for each country and at each admin level in Azure blob storage here:
    • Topojson instructions

Where is the data stored

UNICEF uses Azure, so many static files are in Azure blob or file storage. Here is an npm module to interact with Azure.

• address dependencies/vulnerabilities

We're creating datasets for all countries for school mapping and EPI.

Can we do something like add somecountry-school-mapping to fetch relevant map?

This needs to be fleshed out before being implemented.

What

Clone https://github.com/unicef/magicbox-aad-app as new projectlet in root.

Context

Right now, our app does not have an authorization layer yet. The authorization protocol we desire lives in a separate project (listed above). What we want is to integrate this auth service into our demo app, which, we think, will smooth out our deployment process.

We see two potential ways to integrate auth:

1. Add the auth functionality directly to the demo application code
2. Use containers/docker compose to run as two separate but connected projects, on different ports, and operate as microservices, with Kepler possibly beamed in through an iframe on the page where the user gets redirected to on login.

Outcome

The ideal outcome would be to have a functioning, deployed version of the app (in its current iteration) with an auth layer. If you have an alternate suggestion for how this could be done, we are open to creative solutions different than the suggestions above that include a write up and code demo.

Sometimes, we need to query external databases using only SQL. One of the common use cases when this occurs is calculating multi-hop mobility.

What is multi-hop mobility?

To better understand multi-hop, imagine 3 users (u1, u2, and u3) and 3 cellphone towers that they might ping off of (A, B, and C).

Over the course of a day, the first user may travel in such a way, that when using their phone, they ping the following towers:
AAAA BBB C A

Additional user trips might be:
u2: C A BB
u3: CC BBB A C

How do you calculate multi-hop mobility?

To calculate the mobility for each user user, we need to calculate all trips between areas, but not count "self-loops", or when the start and end towers are the same (i.e. A -> A).

First, we would simplify the trips to remove self-loops, finding their basic path in chronological order. For example, u1: A, B, C, A

Next, we need to calculate all permutations of trips among these locations:

• Single-hop trips: A -> B, B -> C, C -> A
• Two-hop trips: A -> C, B -> A
• Three-hop trips: A -> A (this can be thrown out, as it is a self loop)

If there were more trips, perhaps if the path was: A, B, C, A, C, A, B, we would continue until all hops are recorded.

SQL query should ultimately return either a table with only the hops, and no user information, such as:

1. AB
2. BC
3. CA
4. AC
5. BA
   and so on for each user, or a total, aggregated count of each possible hop, for example:

• AB - 52
• CB - 23
• BC - 10
• AC - 15
• CA - 139

The database could be hundreds of thousands to millions of lines, so the query should be efficient. It could be broken down into parts and use temporary tables, for example, so portions of the query can be run at a time to minimize chances of timing out. Variable names should be clear, so that this can be easily adapted to multiple different databases.

For testing purposes:

The columns are
sim_id : identifier of sim card
timestamp : datetime dd-mm-yyy hh:mm:ss, where hh is given in 24 hour format
event : the type of event that generated this row, can be sms, call, data
site_id : antenna identifier

The activity sequences are:
u1: AAAA BBB C A
u2: C A BB
u3: CC BBB A C

The simplified sequences are:
U1: A B C A
U2: C A B
U3: C B A C

The mobility results should be:
AB: 2
AC: 2
BA: 2
BC: 2
CA: 3
CB: 2

Resources

SQL online sandbox: http://rextester.com/l/sql_server_online_compiler

Moved here

This might work: https://github.com/helmetjs/nocache#readme

A clipped Voronoi diagram is one that is confined to a compact domain (i.e. a bounded and closed 2D region or a 3D volume). In the example above, a normally generated Voronoi diagram will have some cells infinite or partially outside of the country boundary. However, only the cells that lie within the country boundary are of our interests.

Admin 1 geojson: http://myjson.com/wvt2s

The output of this task could be used as input to #15.

When user clicks save, the data sets and layer configuration should be saved to the back end. When user returns to url in new browser, the updated config.json should load.

Make "share" and "save config" buttons configurable in ./client/config.js

Problem statement

Deploy Magicbox Kepler app on Kubernetes. That includes both componentes Client and Server, and expose them for external consume.

Expected outcome

Input:

• The source code working well.

Output:

• The deployment configuration files.
• Steps to deploy on Kubernetes.

Resources

• An example to deploy a react app on Kubernetes (link).
• How to run kubernetes locally with minikube (link).

Problem description

When working with region-based datasets, we usually encounter the situation where the polygons in one dataset do not geospatially align with the polygons in another.

Examples:

• Cell tower's coverage areas VS. administrative divisions
• Voronoi diagram of Ebola outbreak points VS. administrative divisions
• Health zones VS. population administrative divisions

One feature we want for the MagicBox Platform is the ability to aggregate the values for polygons of one shapefile/GeoJSON/csv based on the polygons of another shapefile/GeoJSON/csv. Hence, corresponding with the examples above:

• Cell tower's coverage areas determine where cellphone calls originate. To know mobility i.e. how people move from one place to another, we need to know where people are at a given time and the call locations will tell us that. But we are not interested in the human movement from one coverage area to another; rather, we want human movement from one administrative division ("admin" for short) to another. To generate this "mobility by admin" data, we have to aggregate the mobility values from the "coverage area" shapefile/GeoJSON/csv to the "admins" shapefile/GeoJSON/csv. The output will be a new column in the "admins" file: the mobility_value column. Summarized problem statement: Aggregate mobility values by admins, based on mobility values by cell tower's coverage areas.
• From the Ebola outbreak points, we can generate a Voronoi diagram showing the probability of new contract cases. These Voronoi polygons of course won't fall squarely on the admins polygons. What we want, then, is to assign the new_case_probability value to the admins, so that we know which admins are more vulnerable than others and thus need more attention and resources from the government/aid agencies. Summarized problem statement: Aggregate new case probabilities by admins, based on Voronoi diagram of Ebola outbreak points.
• We have the number of Zika cases per health zone, but those values will be more helpful to MagicBox data scientists if they are by population area. An derived use case is to connect this case-per-admin data with the mobility-per-admin data above and plot the (potential) spread of disease. Alternatively, we might have data on the medical capacity per health zone (e.g. the number of doctors/nurses/community health workers/vaccines). If we can aggregate these values by population admin, governments and policy-makers will be informed of how well or uderrserved each admin is. Summarized problem statement: Aggregate [a numeric attribute] by admins, based on [that attribute's values] by health zones.

The pattern is spotted:

• Input: geofile_1, geofile_2, an attribute_of_interest that exists in geofile_1
• Output: geofile_2 augmented with attribute_of_interest

An example use case

Here you see a Voronoi diagram over a country shapefile. The generating points are arbitrary, but we can assume they are cell tower locations (pairs of coordinates to be exact).

This Medium article discusses in-depth this use case, though it goes one step further with the output. Going beyond (origin_admin, destination_admin, mobility_value) for each row/tuple in the resulting geofile, it asks for a mobility matrix. The latter is easily generated from the former, though.

User should be able to add description of visualization. This description should be visible to audience via modal or something.

Would think this would be something all kepler users want. Not sure if exists as issue here: https://github.com/uber/kepler.gl/issues

Each user should have a personal directory where their map configs are stored.

@mikefab including here some sketch/diagram for our app architecture would be helpful.