Code Monkey home page Code Monkey logo

mediature's Introduction

Mediature

Technical setup

AlwaysData

This helps for DNS delegation (to configure domains, emails...). Configuration steps will be specified below for each service that needs specific DNS records.

GitHub

Apps & Actions

  • CodeCov: code coverage reports (we have CodeQL in addition in the CI/CD... pick just one in the future)
  • Lighthouse: accessibility reports (we also have an accessibility plugin for Storybook, but this one only helps while developping)

Environments

You must configure 2 environments in the CI/CD settings:

  • global (to restrict to dev and main branches only)
  • dev (to restrict to dev branch only)
  • prod (to restrict to main branch only)

Secrets

The following ones must be repository secrets (not environment ones):

  • CHROMATIC_PROJECT_TOKEN: [SECRET]
  • LHCI_GITHUB_APP_TOKEN: [SECRET]
  • SENTRY_URL: [SECRET] (format https://xxx.yyy.zzz/)
  • SENTRY_AUTH_TOKEN: [SECRET]
  • SENTRY_ORG: [SECRET]
  • SENTRY_PROJECT: [SECRET]

Default branch

The default branch is dev.

Branch protection rules

  1. Pattern: main Checked:

    • Require status checks to pass before merging
    • Do not allow bypassing the above settings
  2. Pattern: dev Checked:

    • Require linear history
    • Do not allow bypassing the above settings
    • Allow force pushes (+ "Specify who can force push" and leave for administrators)

Sentry

Upload sourcemaps

To upload sourcemaps to Sentry you need a specific "auth token", it must have these scopes:

  • project:releases
  • org:read

You can create this token at https://${SENTRY_URL}/settings/account/api/auth-tokens/ ;)

Scalingo

Global

We enabled the Postgres addon and the "review apps" feature.

Domains

For each environment you need to configure the domain DNS records to target the Scalingo instance when accessing the domain.

For the www subdomain:

  • Type: CNAME
  • Hostname: www
  • Value: xxxxxxxxxx.scalingo.io (depending on the environment)

And since the root domain cannnot use a CNAME we have to use fixed IPs:

  • Type: A
  • Hostname: ``
  • Value: xxx.xxx.xxx.xxx (depending on the environment)

(you can find those IPs at https://doc.scalingo.com/platform/app/domain#configure-your-domain-name . There is a low risk they change so we should be fine... another dynamic solution would have been to use an ALIAS record if the DNS provider is compatible but it disables the DNSSEC and we are not confident yet of the real underlying risks so we stick we A records for now)

Postgres

Extensions
  • uuid-ossp
Tooling

When using Docker you will be able to use pgAdmin 4, and for your local workspace just choose the software that fits your need. Preferences for:

  • pgAdmin 4 (specific to Postgres)
  • DBeaver (RDBMS-agnostic)

Environment variables

Scalingo is used as a PaaS to host our builds.

For each build and runtime (since they are shared), you should have set some environment variables.

For the "main" app
  • BUILD_APP_NAME: main (it would be docs if you wanted to deploy the other app)
  • APP_MODE: prod or dev (depending on the instance you deploy)
  • DATABASE_URL: $SCALINGO_POSTGRESQL_URL (filled by Scalingo automatically when adding a database)
  • FILE_AUTH_SECRET: [SECRET] (random string that can be generated with openssl rand -base64 32. Note this token is just for the short-lived read permission of private attachments)
  • NEXT_AUTH_SECRET: [SECRET] (random string that can be generated with openssl rand -base64 32. Note that if this secret is lost, all users will have to log in again)
  • NEXT_PUBLIC_APP_BASE_URL: [TO_DEFINE] (must be the root URL to access the application, format https://xxx.yyy.zzz)
  • CRISP_SIGNING_SECRET_KEY: [SECRET] (this secret is generated from your Crisp account and depends on the development or production environment)
  • NEXT_PUBLIC_CRISP_WEBSITE_ID: [TO_DEFINE] (this ID is defined in your Crisp account and depends on the development or production environment)
  • NEXT_PUBLIC_SENTRY_DSN: [SECRET] (format https://xxx.yyy.zzz/nn)
  • MAILER_DEFAULT_DOMAIN: [TO_DEFINE] (format xxx.yyy.zzz depending on the environment application URL)
  • MAILER_DOMAINS_TO_CATCH: domain.demo (this should only be set in the development environment)
  • MAILER_SMTP_HOST: [SECRET]
  • MAILER_SMTP_PORT: [SECRET]
  • MAILER_SMTP_USER: [SECRET]
  • MAILER_SMTP_PASSWORD: [SECRET]
  • MAILER_FALLBACK_SMTP_HOST: [SECRET]
  • MAILER_FALLBACK_SMTP_PORT: [SECRET]
  • MAILER_FALLBACK_SMTP_USER: [SECRET]
  • MAILER_FALLBACK_SMTP_PASSWORD: [SECRET]

Review apps

Those are temporary environments, different than dev and prod. Since they have their own random generated IDs, we use the scalingo.json file to:

  • adjust environment variables
  • seed the database to have some data to test on

Monitoring

It's important to be aware of some events, for this we decided to monitor:

  • Scalingo (responsible for the application and database)
  • Sentry (responsible for gathering runtime issues)

Note that doing it after the full setup will avoid flooding your inbox :)

IMPORTANT: keep in mind webhook URL tokens can be invalidated if someone leaves the project while he created them.

Scalingo

For both the development and production environments you need to create notifiers by following the following steps.

First, create a Slack notifier named tech to keep an eye on the global activity, specify the webhook URL to use and enable those events:

  • addon_db_upgraded
  • addon_deleted
  • addon_plan_changed
  • addon_provisioned
  • addon_resumed
  • addon_suspended
  • alert_added
  • alert_deleted
  • app_alert_triggered
  • app_command_ran
  • app_crashed
  • app_crashed_repeated
  • app_deleted
  • app_deployed (only in production)
  • app_edited
  • app_region_migration_started
  • app_renamed
  • app_restarted
  • app_scaled
  • app_stopped
  • app_transferred
  • collaborator_accepted
  • collaborator_invited
  • collaborator_removed
  • domain_added
  • domain_edited
  • domain_removed
  • github_link
  • github_unlink
  • notifier_added
  • notifier_edited
  • notifier_removed
  • variable_added
  • variable_bulk_edited
  • variable_edited
  • variable_removed

And in some cases you don't want to miss the notification, because it's a runtime failure, a sensitive setting that changed... For this create an email notifier named tech-important and add appropriate people to be notified. Enable those events:

  • addon_deleted
  • addon_suspended
  • app_alert_triggered
  • app_crashed
  • app_crashed_repeated
  • app_deleted
  • app_region_migration_started
  • app_renamed
  • app_transferred
  • collaborator_accepted
  • collaborator_invited
  • collaborator_removed
  • domain_added
  • domain_edited
  • domain_removed
  • github_link
  • github_unlink
  • notifier_edited (only in production)
  • notifier_removed (only in production)
  • variable_bulk_edited (only in production)
  • variable_edited (only in production)
  • variable_removed (only in production)

Find more context on those events on https://doc.scalingo.com/platform/app/notification

Debug the runtime

To debug scalingo apps you may prefer using their CLI with some commands like:

scalingo login --api-token ${SCALINGO_API_TOKEN}
scalingo -a ${SCALINGO_APP_NAME} addons
scalingo -a ${SCALINGO_APP_NAME} logs
scalingo -a ${SCALINGO_APP_NAME} logs --addon ${SCALING_ADDON_ID}
scalingo -a ${SCALINGO_APP_NAME} run bash
scalingo -a ${SCALINGO_APP_NAME} pgsql-console
scalingo -a ${SCALINGO_APP_NAME} db-tunnel ${SCALINGO_DATABASE_URL}

Note that:

  • pgsql-console command will logs you onto the database but it's shell only (it uses the default Scalingo user)
  • db-tunnel command sets up a SSH tunnel first, so you need to configure your public SSH key in your Scalingo account settings, but then you will be able to use local software to navigate through the database, or even migrating the schema with custom scripts.

To debug a remote database we advise creating a specific user (because you are not suppose to store credentials of super users). Make sure the user created has been granted needed roles on business tables (through the psql-console command), something like GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA $SCHEMA TO $USERNAME; that you can customize as needed ;) . (If you still see no table that's probably because you logged into the wrong database)

Debug the CI/CD pipeline

Scalingo uses the Heroku technology of buildpacks to embed your application at runtime and it's quite complicated to replicate the logic locally. It does not bring the flexibility of a custom Docker image, nor the new "standard" tool to manage buildpacks (https://buildpacks.io/).

In case you have an edge pipeline error, it may help to try using herokuish for mimicing the build embedding your application in the wanted buildpack. It's not a miror of the Scalingo pipeline but it can help. The easier thing we found for now is to use a custom Dockerfile:

FROM gliderlabs/herokuish:latest

COPY . /app

ENV BUILD_APP_NAME=main

On the other side if you are looking for investigating a built image, Scalingo provides a paid addon to access the registry of images that passed their pipeline with success (those are Docker images). It helped us in the past to optimize excessive size of remaining dependencies for example.

Sentry

TODO: to write once it's compatible with Next.js 13 a new time

Emails

We use 2 providers to send emails:

  • the main one (Mailjet)
  • the fallback one (SendInBlue) in case the main one is not reachable it keeps our delivary reactive

Sending verified emails must be taken seriously so they don't end into the spam inbox. Keeping a good reputation by sending necessary content so users don't flag you as spam.

Also you need to configure your DNS records to handle from both providers on the 2 environments (development and production): DMARC/DKIM/SPF. It's well explained when adding sending domains on their interface. It will make your emails signed according to your domain.

When creating SMTP credentials make sure sure to use different ones between the development and the production environment.

Crisp

Crisp is used as a livechat both for visitors and users.

From their interface we create 2 websites:

  • Production: use the production domain
  • Development: use the development domain

Set the name:

  • Production: Médiature
  • Development: Médiature [DEV]

And upload as the icon the one used for the website (usually apple-touch-icon.png has enough quality).

Add to the team the people you need (without giving too many rights depending on their role).

Into the Chatbox & Email settings section go to Chat Appearance and set:

  • Color theme (chatbot color): Red
  • Chatbox background (message texture): Default (No background)

Then go to Chatbox Security and enable Lock the chatbox to website domain (and subdomains) (not need to enable it inside the development environment).

And inside Advanced configuration, enable Verify user emails with cryptographic signatures. This will help making sure someone named "John Doe" engaging the conversation in the livechat is really our agent "John Doe" of the platform and not someone trying to impersonate his identity. Since Crisp messaging is initialized from the frontend, without the cryptographic trick it would be impossible to certify a user asking for sensitive operation is the expected one.

It will give you a secret you that you need to serve to your backend as CRISP_SIGNING_SECRET_KEY (it should not be available on the frontend side contrarily to the website ID). (for now, we only use it in production since it requires ugprading the Crisp plan)

On the other site, the public "website ID" will be used as NEXT_PUBLIC_CRISP_WEBSITE_ID.

IDE

Since the most used IDE as of today is Visual Studio Code we decided to go we it. Using it as well will make you benefit from all the settings we set for this project.

Manual steps

Every settings should work directly when opening the project with vscode, except for TypeScript.

Even if your project uses a TypeScript program located inside your node_modules, the IDE generally uses its own. Which may imply differences since it's not the same version. We do recommend using the exact same, for this it's simple:

  1. Open a project TypeScript file
  2. Open the IDE command input
  3. Type TypeScript and click on the item TypeScript: Select TypeScript Version...
  4. Then select Use Workspace Version

In addition, using the workspace TypeScript will load compilerOptions.plugins specified in your tsconfig.json files, which is not the case otherwise. Those plugins will bring more confort while developing!

Tips

Frontend development

i18n

Currently we only use i18n to help displaying ENUM values. We use the i18n Ally VSCode extension to improve a bit the usage but everything can be written manually in .json files :)

Storybook
Use it first

Developing a UI component by launching the whole application is annoying because you may have to do specific interactions to end viewing the right state of the component you are building.

We advise when doing some UI to only run Storybook locally, so you do not worry about other stuff. You can also mock your API calls! At the end splitting your component into different stories makes easy for non-technical people of the team to view deepest details. They will be able to review the visual changes with Chromatic, and also from the Storybook they will save too their time avoiding complex interactions to see specific state of the application.

It's not magical, it does not replace unit and end-to-end testing, but it helps :)

Advantages
  • Accessibility reports
  • See almost all tests of your application
  • Helps architecturing your components split
  • Their rendering is tested in the CI/CD, so it's very likely your components are valid at runtime
How we test stories

You can do UI testing scoped to components (I mean, not full end-to-end), and if so, it's recommended to reuse the stories to benefit from their mocking set up.

A decision we took to keep in mind: it's common to create a .test.js file to test that a component renders correctly (thanks to Jest, Playwright or Cypress), but since we have "Storybook test runners" already in place that pop each component, we feel it's better to use the story play function concept (also used by the interaction addon) to make sure they are rendering correctly.

  • It avoids rendering each component one more time
  • It saves 1 file and it's clearer since from Storybook you can see in the "Interactions" tab if the expected behavior is matched

Those kind of tests may be useful to:

  • Make sure for an atomic component the data is displayed correctly (but it's likely the work of your team to review visual changes)
  • Guarantee the sequences when mocking (e.g. first a loader, then the form is displayed), it helps also the "Storybook test runners" to wait the right moment to take a screenshot/snapshot of the final state (through Chromatic in this case), not intermediate ones since the runner doesn't know when the component is fully loaded otherwise

(in case you have specific needs of testing that should not pollute the Storybook stories, go with a .test.js file, see examples here)

Tip: during the testing you could findByText but it's recommended to findByRole. It helps thinking and building for accessibility from scratch (because no, accessibility is not done/advised by an accessibility automated check unfortunately).

Testing performance

During the test we render the story, we test the light theme accessibility and then we switch to the dark theme to test it too. The re-rendering for color change over the entire DOM is unpredictable, it depends on the CPU saturation. We had some leakage of previous theme rendered over the new one.

We evaluated 2 solutions:

  • limit the number of Jest workers with --maxWorkers
  • increase the amount of time we wait after triggering a theme change

After dozens of tests it appears the most reliable and the fastest is by keeping parallelism (no worker limitation), but increase the amount of time. But this latter depends on the environment:

  • when testing headless: there is less work done, the delay is shorter
  • when testing with a browser rendering: it is in higher demand, to avoid color style leakage we need to increase the delay (tests duration is +50%, which is fine)
Hydratation issue

When developing a frontend it's likely you will have client hydratation according to the server content. It will fail if some browser extensions are enabled and modify the DOM. You need to identify the source of the issue and then, either disable the extension, or request it to not modify the DOM when developing on http://localhost:xxxx/.

From our experience, this can be caused by:

  • Password managers (make sure to have no credentials that match your development URL)
  • Cookie banner automatic rejection (in their settings you're likely to be able to exclude your development URL from being analyzed)

(in React the error was Extra attributes from the server: xxxxx)

Cannot fetch specific files

As for any hydratation issue it worths taking a look at your browser extensions, some may block outgoing requests.

For example:

  • Ad blockers (whitelist the blocked URL in your extension)

Jest not working in VSCode

Sometimes it appears Jest VSCode extension will be stuck and will keep throwing:

env: node: No such file or directory

We found no fix. The only solution is to relaunch VSCode, and if it still doesn't work, try to close entirely VSCode and open it through your terminal with a something like:

code /Users/xxxxx/yyyyy/mediature

mediature's People

Contributors

sneko avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.