This looks like a good component to go with: https://github.com/mhnpd/react-loader-spinner

Rather than our own implementations of metrics, it makes sense to use those from fairlearn

This is a tracking issue for things to do to clean up the code.

• favour fig over fig.show() in notebooks so that browser tabs aren't spawned when building locally
• run black on all source, black-nb on all notebooks
• check for unused imports
• sort imports
• Use f-strings rather than old school string interpolation with %
• Consistent use of fairness terminology (e.g. demographic parity rather than independence)
• Format floating point numbers to 3 decimal places

Likely could have a page (per subtopic) with all binder links and context, to help people understand what is expected to happen when the click those links, and also highlight the chance to do that, otherwise in the text it might be too subtle.

When do the different flows need to run? (PR opened, PR approved, merged in already?)

Collapsibles are waiting for other JS to be loaded which might take some time, and they cannot be opened while that is ongoing. This is not great UX.

These notebooks are automatically generated, blah

On first view, the plots are displayed with the right size, but if navigating to another page, and then back, it can be squashed. One stark example is the baseline pages hours worked per week plot, which is quite long, but then squashed to a standard small size when navigated away & then back.

The console shows an error like this:

Uncaught (in promise) Error: Resize must be passed a displayed plot div element.
    at plotly.js:141605
    at new Promise (<anonymous>)
    at Object.plots.resize (plotly.js:141603)
    at resizeHandler (factory.js:206)

Currently it's handcrafted, should be automatic.

I don't think the way we measure fairness, nor the way we visualise it is quite right for Pleiss experiments. @baba-mpe perhaps you could take a look at this tomorrow?

Currently it's backwards / forwards link text, but it is usually ellipsized and that doesn't look great. Look for alternatives, or fall back to Previous / Next.

Left over from the experimentation

Currently we have data/ getting generated inside src/, but artifacts/ in the root directory. Should we keep them in the same place. Or perhaps data/ even belongs inside artifacts/?

Doesn't apply to all, but applies to some, such as linting, which should be carried out not just when the last push had relevant file changes, but when the entire PR anywhere had such changes.

I think it would be helpful to have Actions build the site without deploying it on all branches. Just merged something and only found out on master that I had broken something 😳 (all fixed, don't worry)

Currently we install requirements.txt in the linting workflow, but I think this is actually not necessary. The requirements are pretty overbearing and take a good few minutes to install. I think black, flake8 etc. can analyse code without running it, so perhaps we can remove this step.

Because everything needs a license

Things need to happen to design before handover, add new items as they come:

• Header logo replacement with CDEI one
• Licenses update #6
• Author update in files
• Updated favicon #5
• New social preview image
• Loader colour
• Fonts review
• Dealing with "Theme by Rocketseat" item in the bottom (repurpose it for copyright?)
• Plot theming #24
• Styling of 404 page
• Small screen plots #91
• Soften code snippet styling (it's too dark)
• Unresponsive collapsibles on page load #92

I think it doesn't look great, looking for options to do away with that highlight.

There's react-collapse that we use at the moment.

An alternative is react-component/collapse, and there I see the highlight in in their demo already, though a bit less jarring: http://react-component.github.io/collapse/examples/custom-icon.html (also the demo already does the tabbing and expand-on-keypress, that we added manually, so it seems better equipped out of the box?)

Also looking at react-collapsible which might be an alternative (and seems to need less code? but maybe that's just how they write their readme. Tried it out through the JSFiddle link in the repo). Less nice looking though.

Opening this issue to look at these for a quick facelift, if that's possible.

somehow the recruiting data generation notebook got reverted to an old version. need to fish the up to date one out of the commit history.

Connects to #9 somewhat, but extra as well. Eg. deploy into an S3 bucket. But we'll see, and it might be just during development time. We'll see.

So people know where they should add files, what tags to use in the notebook, etc.

Notebook output to be stripped.

Currently we have only pinned a subset of the dependencies we're using. Might be worth installing contents of requirements.txt in a clean virtual environment and saving the output of pip list / pip freeze or something?

We can set some custom defaults with this: https://plotly.com/python/templates/

Would give figures a uniform look without much boilerplate.

Better than showing horrible graphs.

To speed up site builds, hopefully.

So that people can test better remotely.

Add to replace the default Gatsby one.

Github can host docker images, could publish our precompiled image there. But this presumes that Binder doesn't cache images, which needs validation. Right now the master branch is moving too quick, so couldn't tell.

So that it's not in the root

better name to that library?

Currently we check dp, cdp, eo, eopp. Should add a calibration check also.

It seems its not possible to save the predjudice remover model directly, so instead we should save the outputs for reproducibility.