Code Monkey home page Code Monkey logo

portic-storymaps-2021's Introduction

Portic storymaps 2021 : "Commerce multi-échelles autour du port de La Rochelle au XVIIIe siècle"

Screenshot of the website

DOI

This repository hosts the source code of PORTIC research program's first case study (see PORTIC homepage for more information). Built by an interdisciplinary team of historians, engineers and designers, it proposes a detailed study of the economic history of the region of La Rochelle (France) circa 1789.

Through a series of three "storymaps" combining text and visualization, this publication tells the story of the decline of La Rochelle region's ports after France's loss of Canada during the seven years war, and the consequences of this event on the region's trade structure at the dawn of french revolution. It also features an atlas allowing to browse and share individually all the visualizations crafted during this research.

This project rests on the shoulders of two existing digital history projects : Toflit18 and navigo.

Installation

Prerequisites:

Then in a shell terminal, copy the following commands and hit enter:

git clone https://github.com/medialab/portic-storymaps-2021
cd portic-storymaps-2021
yarn install
pip install -r requirements.txt
yarn data:update

Development

yarn start

Useful scripts

The project is provided with diverse scripts aiming at updating data and contents.

Data-related scripts

yarn data:load # fetches data from toflit18 and navigo sources and stores them in the `data` folder (ignored from this git repo)
yarn data:build # builds derivated datasets from complete data given by data:load script. It stores them in public/data
yarn data:update # orchestrates data fetching, building, then deleting of the temp `data` folder containing complete datasets

Updating the thumbnails

Thumbnails making for the atlas view is automated. To update thumbnails, in two different terminal tabs, run :

yarn start

Then run:

yarn run thumbnails:make

Warning: thumbnails building can be capricious on some machines. Backup existing screenshots from public/thumbnails before re-running this script.

Contributing

The project is open to contribution through pull requests.

Suggested guidelines for commiting to the repository

  • the main branch is the principal branch for the website version under development. Suggested workflow for contributing to the code is : for project members, to develop new features in a separated branch, then to merge it in main branch when it is ready ; for external person, to clone it then to submit a pull request with your modifications.

  • it is suggested to use imperative tense verbs and explicit features/bugs mentions in commit messages (see this guide for optimal commit messages)

  • it is suggested to reference related issue in commit messages (example of commit message : improve radar viz #8) in order to keep track of commits related to an issue in particular.

Technical architecture of this repo in relation to data sources

Schema of the project architecture

Detailed presentation of the organization of the repository

- .github # contains automated deployment workflow
- datascripts # contains python scripts that fetch and build web-oriented datasets and contents, putting them in public/data and src/contents folder
- public # contains website's html, icons, datasets and other static assets
- src # the source code of the website
  - components # reusable components
  - contents # mdx files for website's content in 2 languages
  - helpers # various functions and utils
  - pages # page containers used directly for routes rendering
  - visualizations # components directly used in website's pages and atlas
  - App.js # entrypoint of the react application
  - App.scss # entrypoint of the scss code
  - colorPalettes.js # color palettes to be used accross visulizations
  - ...
  - summary.js # summary of routes and their titles in french and english
  - visualizationsList.json # dynamically generated list of visualizations to be used in atlas and pages to retrieve titles, captions, & co.

Guidelines concerning code development and modification

  • reusable components should go into src/components folder. Each component should have its own folder with an index.js file, plus as many files as you want (js subcomponent files, scss files, component-specific assets, ...)

  • components aimed at being directly used for specific visualizations should go in the src/visualizations folder. They should use reusable components from src/components as much as possible.

  • style is managed through scss files. It is suggested to use existing variables in src/variables.scss as much as possible, and to add a .scss file specific to each new component with its non-reusable styling rules (if any).

Deployment

Deployment used to be automated to happen every day and each time a commit is pushed to the prod branch. The published website was then pushed on the gh-pages branch, which serves the site at https://medialab.github.io/portic-storymaps-2021/. It is now only updated when pushing on the prod branch.

Therefore :

  • Contents and data are updated every day
  • To deploy a new version of the website code, it has to be pushed to the prod branch.

This project was bootstrapped with Create React App.

portic-storymaps-2021's People

Contributors

c-assl avatar cplumejeaud avatar myllaume avatar paulgirard avatar robindemourat avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

ksachikonye

portic-storymaps-2021's Issues

Internationalization review

  • [refactoring] use es6 key-value shortcut ({start: start} -> {start})
  • translation regex: dash/underdash-proof ?
  • translation regex: handle if args[key] undefined
  • src/visualizations/PrincipalVisualizationPart2/PrincipalVisualizationPart2.js l92 -> some parts not translated in english ?
  • PrincipalVisualizationPart2.js -> issue at line 127 (double function), issue with value at line 138-139 (always undefined)

home view

Visualizations and data preparation

  • workflow script for retrieving the contents of the intro
  • boats animation
  • map 1 province highlights
  • map 2 centered on region and showing ports
  • map 3 centered on region and showing bureau de fermes

Integration

  • color ports, customs offices, ... according to colorpalette
  • header with centered title and boat animation
  • two column intro
  • full width nav menu at the bottom

Home

in pages visualizations, add the possibility to open each visualization in atlas mode

... in order to be able to read the howto

It would require to:

  • update scrollypage by adding an optional lightbox
  • update routes map for chapters routes to be able to add an optional :visualizationId param to them
  • update all visualizations components to be able to optionnally add a callback for this, and then update the design to display it (a "?" button ?)

part 3

Data preparation and visualizations

State 1 of the viz

  • script => write csv like so:
port nb_pointcalls_out mean_tonnage cumulated_tonnage longitude latitude
  • rotate and center
  • position ports
  • render triangle objects and labels (rotated)
  • make the full svg fit into the viz column
  • develop custom GeoComponent => merge geo and triangle components into 1 SVG
  • introduce a threshold => not to represent all ports ? (will require o change nbOfCols based on data.length)
  • color ports ?
  • display lines between objects and map (then turn those into curves)

State 2 of the viz

  • write gexf for the region @robindemourat
  • write centrality comparison @robindemourat (parameterizable histogram component)
  • how to handle scroll ? (have to click ? limit the scroll ?)
  • use color variables for nodes' colors

State 3 of the viz

  • script => write 1 csv like so: (if too complicated I produce 3 csvs)
type_of_object name cumulated_in_region cumulated_out_region nb_objects_taken_into_account customs_office customs_region
  • reusable double triangle and circle component (beziers)
  • integrate component in viz
  • what do we do with Oléron as ferme_bureau ?? + Navigo data taken in 87 or 89 ??
  • gestion of the comparison with other regions
  • bonus : add animations (could be starting animation with the map turning)

Integration

  • legend, captions, labelling ...

Partie 3

part 1

Visualization and data preparation

  • set up principal viz
  • timeframe prop + enter/exit transitions
  • tooltips

Integration

  • move ProductsDistributionChart call to main component

  • include colors

  • handle different container heights

  • allow to switch between timespans

  • improve lisibility : colors and labels

  • improve understandability (captions)

  • see if customed width size for text column is needed

Partie 1

Viz "partie-2-carte-direction-bateaux-de-la-rochelle"

Guillaume:

Souci avec ce graphique : l’importance de la Traite est minimisée, car les deux flèches vers la côte de l’Or et la côte de l’Angola sont superposées. Suggestion : pointer la flèche vers le golfe de Guinée plus à l’Ouest, du côté du Ghana.
En plus, la Côte de l’Or, c’est effectivement le Ghana......

Guillaume:

En fait, je ne comprends pas ce graphique...

  1. Quelle règle détermine la couleur des flux si ni le port de départ ni celui d’arrivée sont La Rochelle ?
  2. Quel est l’univers des relevés de Navigo 1787 ? Que les congés français je suppose ?
  3. Lorsque je passe le curseur sur les flux jaunes vers Saint-Pierre et Miquelon, j’obtiens les ports de Saint-Martin-de-Ré ou les Sables-d’Olonnes.

Réponse Silvia:

  1. le port d'attache = La Rochelle (j'ai reformulé pour que ce soit plus explicite)
  2. j'ai précisé
  3. oui car tu as des navires appartenant à des négociants de La Rochelle qui partent de ces autres ports (on ne représente que les navires dont le port d'attache est = La Rochelle

Réponse Guillaume:

  1. et 2) c’est clair maintenant..
  2. ce n’est pas clair. Voir email avec capture d’écran.

Capture d’écran 2021-10-04 à 16 32 08

Réponse Silvia:

Je pense que le toponyme St Martin est juste à la gauche du point concerné et donc à sa place et pas à Terre-Neuve. Ou alors je n'ai pas compris? Le souci de cette carte est qu'il est difficile de distingue rou se termine une carte et où commence l'autre. Au pire on regarde ça tous devant un écran le 14?

Réponse Guillaume:

Oui, mais le problème vient de la position de la souris à ce moment-là. Si on passe la souris sur la flèche trajet Saint-Pierre et Miquelon — New York dans la carte du bas à droite, c’est « Les Sables d’Olonne—Rochefort » qui apparaît. Pour avoir Saint-Pierre et Miquelon — New York, il faut avoir la souris sur les Grands Lacs (ou l’Ohio… ces coins-là).

Update contents parser

  • integrate introduction
  • integrate about
  • integrate references
  • normalize document's hyperlinks

LineGraph component

Props synopsis:

  • x: {field, title, tickInterval?, tickFormat?} : settings for x variable

  • y: {field, title, tickInterval?, tickFormat?} : settings for y variable

  • color: {field, title, tickFormat?} : settings for color variable

  • fillGaps: whether to fill gaps for missing data

  • animated : whether to animate

  • tooltip : function with datum as parameter to display the tooltips

  • highlights

  • annotations

Viz #3 finetuning

  • improve networks size
  • "pourquoi est-ce que Les Sables-d’Olonnes sont tellement au sud ? "

[at the end] typographic cleaning

At some point we should clean text typography (french quotation marks, ...). We could integrate that in the automated workflow or do it at the level of the doc. To keep in mind for the end of the project !

part 2

Visualization and data preparation circular alluvial diagram (@robindemourat @c-assl)

  • prepare data and basic layout
  • aggregate flows better (currently several flows for the same modalities)
  • stabilize labels
  • improve understandability (arrows, explanations, ...)
  • improve interaction (highlight on click instead of hover ?)

Visualization and data preparation radar plot (@gegedenice @cplumejeaud)

Data preparation

  • adding a script in datascripts folder writing a csv in public/data (aggregated by ports), like so:
port customs_office destination tonnage_cumule nb_outs

Visualization

  • build radar visualisation component in src/components folder
  • integrate the component in visualizations/principalVisualizationPart2

Radar component features wishlist:

  • highlight object: prop to be able to highlight a particular spider net (by cutom office or by port --> example: highlight object = Marennes)
  • add a way to toggle aggregation value (--> example: aggregate by nb_outs, or by tonnage_cumule)
  • bonus: animate shape change with react-spring

Partie 2

Refined Geocomponent

API GeoComponent : change design

to let : width, height

MVP

  • projection : { object }
  • projection.template { string } [ 'Poitou', 'France', 'World']
  • layers { array }
  • layers[n].data (on donne data à chaque layer)
  • layers[n].type { string } [ 'flow', 'choroplète (zones)', 'point', (customs)]
  • color (field, title, label)

API par type

  • points et flows : gérer size
  • choroplète : 2 data (backgound et data ? ou alors jointer en amont)

bonus

  • projection.rotation, .centerX, .centerY, .scale, ...

For Introduction / Atlas

renderObject function

  • object that handle their position themselves (projection function in parameters)
  • create renderFlow function : being able to give multiple object as param (to represent a flow for instance, and construct this map : https://kepler.gl/demo/map?mapUrl=https://dl.dropboxusercontent.com/s/nncyyk8e3op7b0c/keplergl_gj3hx1b.json a secondary viz)
  • position geographically small multiples ?? (maybe more intelligent to align it to baseline)
  • objects that can separate (reduce size or move centers)
  • handle colors (introduce a loop to determine if colorpalette should be chosen, or other computation) : should be a param of GeoComponent ??
  • implement layers ? to handle choroplète with other objects (such as circles for ports)

Specific GeoComponent for part 3

  • delete useless parts for this component ??

Reusable BarChart component

Props:

  • data -> data to display
  • layout ?-> ['stack', 'groups'] -> if color is defined, sets the bars agregation layout
  • orientation -> vertical or horizontal
  • x -> x variable (field, title, tickFormat, fillGaps, domain)
  • y -> y variable (field, title, tickFormat, fillGaps, domain)
  • color -> {field, title, palette}
  • highlights
  • annotations

TODO:

  • show numbers on bars when not too small

The Tonnay-Charente alignment mystery

A remainder for later investigation:

Navigo ports alignments with customs office is done at the DB level and corrected in some of our scripts. Tonnay-Charentes seems to be attached to Marennes, when it should be attached to Tonnay-Charente.

The problem was solved for viz 3, but we just discovered it was not for viz 2.

Two things to do:

  • short-term: understand the displacement, find if other visualizations should be changed
  • long-term/optional: have a discussion with relevant parties to fix these alignment issues once and for all in the data

Improve map: improve flows layer

  • make arrows end at the end of line
  • allow for controlled hover when displaying multiple linked maps
  • add a "hide out of bounds flows" option
  • when arrows overflow, shorten them so that they end at the limit of the visualization box

How to shorten the arrows - scans from brainstorming:

DOC061021-06102021102954-1

Or with trig:

DOC061021-06102021103032-1

fix pointcalls data problems

Silvia 2021-07-19:

Il y a un souci quelque part sur la manip d'extraction des données de Navigo (la visu avec les triangles dans la partie 3 et la nouvelle carte de l'intro de la partie 1 (et peut-être ailleurs aussi): je pense que vous avez pris le pointcall_rank = 1 (ou le pointcall_action = OUT?) comme équivalent du port de départ, alors qu'il faudrait prendre le pointcall du datablock leader marker = A car dans certains ports on nous dit aussi que le navire qui sort du port (IN-OUT) vient de la mer des pertuis (OUT) et va à XXX (IN), ce qui fait 3 pointcall pour le même enregistrement, et dans ce cas le port de départ qui nous intéresse sort du pointcall n° 2 IN-OUT

Christine 2021-08-19:

Silvia, depuis Juillet, je prend les valeurs in-out en départ aussi donc ce souci va être réglé

`datascripts` folder: better organization/documentation ?

Should we, at some point in the future, refactor datascripts scripts to have one folder per datafile, and a possibly readme describing the data?

This question depends on the number of "secondary" visualizations we will have, because if we have a lot of them it could quickly become a mess !

Bug with scroll on atlas view

Silvia:

je suis en train de regarder l'Atlas, je te signale à toute fin utile un petit bug (chrome et EDGE): l'ascenseur gris sur la barre latérale droite ne marche pas (il bouge bien, mais la page ne descend pas, du coup pour lire la suite il faut se positionner sur le texte blanc sur noir et descendre. Pas mortel mais je te signale quand même)

Misc bugs from Silvia's contacts

1

  • Dans la première rubrique "Le déclin de la région de la Rochelle", la cinquième visualisation, celle qui commence avec le paragraphe suivant : "La direction des Fermes de La Rochelle s’est donc [...]", se met à "sauter" lorsqu'on descend ou remonte la page. Je précise que je suis sur un mac avec le navigateur google chrome.

2

  • Lors que le navigateur n’est pas en plein écran (ex : écran scindé), seul Edge affiche les onglets, qui disparaissent sur les autres navigateurs (il faut remettre en plein écran ou cliquer sur la boussole pour y avoir accès à nouveau)

Partie 1:

  • Pour la visualisation intitulée « Comparaison des parts des produits exportés par la direction des fermes de La Rochelle totalisant plus de 90% du commerce en 1750 et en 1789 (ports francs non pris en compte) », majorité des produits illisibles (peut-être est-ce le zoom ?)

  • Pour la visualisation intitulée « Produits dont les valeurs d'exports sont les plus importantes en 1789 : comparaison de La Rochelle à la moyenne française », le graphique en barres s’inverse lorsque l’on scrolle vers le bas (Données pour la France d’abord en vert, puis passent en orange)

Partie 2:

  • Pour la visualisation intitulée « Part de la navigation française dans la région », majorité des pourcentages en camembert illisibles (peut-être est-ce le zoom ?)

Partie 3:

  • Latence relativement importante sur tous les navigateurs pour l’affichage des visualisations (peut-être sont-elles plus « lourdes » que les autres ?)

3

Partie deux une forte spécialisation portuaire

Par ailleurs vous verrez sur les deux photos que je joins ( en fin de message) qu il y a un problème d’apparition ou pas des intitulés au-dessus des deux graphiques ; quand on arrive sur la page on ne voit pas les intitulés mais on voit les graphiques ; il faut descendre dans la page pour que les intitulés apparaissent au dessus des graphiques

1

4

  • dans la partie 1 le repère 1790 sur l’axe du bas ne s'affiche pas bien . Il est sur un iPad, en dernière version

1-1

Atlas

Integration

List view

  • display grid ou flex
  • vignettes à génerer pour chaque visualisation

Lightbox @c-assl

Atlas des visualisations

Atlas des visualisations (avec lightbox)

Cleaning/documentation

  • reorder python scripts
  • attach readme files to CSV #17
  • radarplot: genericize API
  • improve README's technical presentation
  • delete unused modules, components and dependencies
  • comment utils
  • comment components
  • centralize scss and clean file

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.