Code Monkey home page Code Monkey logo

local-dev-env's Introduction

This repo was archived since the relevant code was migrated to serlo/api.serlo.org

serlo.org - Local Development Environment

Scripts for having the main backend services of the serlo.org environment in a local machine.

Requirements:

  • UNIX System
  • Docker and Docker Compose
  • Nodejs and Yarn
  • Python 3 and pip
yarn
yarn start
# or `yarn start:detach` to have control of the terminal

Take a look package.json for useful scripts!

Integrating local Serlo frontend

  1. cd $YOUR_FRONTEND_DIR
  2. Make sure to use the local environment in .env
NEXT_PUBLIC_ENV=local
# NEXT_PUBLIC_ENV=staging
  1. yarn dev Note: every time you change .env, you have to rerun yarn dev

  2. cd back to this repo directory and cp localhost-graphql-fetch.ts "$YOUR_FRONTEND_DIR/src/pages/api/frontend/localhost-graphql-fetch.ts" <- important for doing authenticated actions, but do not commit this change in frontend!, in case you are also working there.

In case of registering new user head to localhost:4436 to get the verification link.

Developing with Ory Kratos

Integrating with Keycloak

First of all add nbp as host
sudo bash -c "echo '127.0.0.1 nbp'" >> /etc/hosts

why do I need it? Kratos makes a request to the url of the oauth2 provider, but since its running inside a container, it can't easily use host port. nbp is a dns that is discoverable for the kratos container, so the host can use it also.

Run yarn start:nbp.

Keycloak UI is available on nbp:11111 (username: admin, pw: admin).
There you have to configure Serlo as a client.

Client -> Create Client

id: serlo
home and root url: http://localhost:3000
redirect uri: http://localhost:4433/self-service/methods/oidc/callback/nbp

Get the credentials and go to kratos/config.yml:

selfservice:
  methods:
    oidc:
      enabled: true
      config:
        providers:
          - id: nbp
            provider: generic
            client_id: serlo
            client_secret: <put secret here>

Run the local frontend (not forgetting to change in .env to local) to test.

Documentation:

Email templates

Kratos has to be rebuilt every time you change an email template. Use the following workflow:

  1. Edit templates.
  2. Run yarn kratos:rebuild
  3. Test the verification or the recovery email at localhost:4436. Repeat the process.

Writing an import script

We are still in the phase of importing data from the legacy database into the kratos one.
From now on let's prefer imports using python for the simple reason that we want to use the very same script in terraform and terraform template files the $ may interfere with javascript $ in string interpolation.

$ python3 -m venv .venv
$ source .venv/bin/activate
(.venv) $ pip install -r requirements.txt
(.venv) $ python3 path/to/import/script
...
# when you are done
(.venv) $ deactivate
$

Developing with Rocket Chat

$ yarn start:chat -d # or start:chat:kratos:db-layer if you're developing in api at the same time
# wait +- 1 minute
$ yarn prepare:rocket-chat
Registering rocket chat as client in hydra

CLIENT ID a6c3e143-****** # It will be different every time
CLIENT SECRET rocket.chat

Copy the client Id.
2. Rocket chat will be available in localhost:3030. Log in as admin using the username dev and password 123456.
3. Go to three dots -> Administration -> Workspace -> Settings -> OAuth2 -> Serlo and paste the client id that you got earlier in the id.
4. sudo bash -c "echo '127.0.0.1 hydra' >> /etc/hosts". This step is optional, but quite handy. Downside: at the end you are not going to see yourself inside of the rocket chat, because of dns name clash with the container.
5. Logout and click on Serlo. You are going to be redirected to http://hydra:4444.... If you haven't done step 4, just change hydra with localhost.

Note that while developing you may want to change the links to chat in some files of frontend (https://community.serlo.org to http://localhost:3030).

local-dev-env's People

Contributors

hugotiburtino avatar dependabot[bot] avatar kulla avatar andreashuber avatar eliflores avatar elbotho avatar

Stargazers

Mikey Stengel avatar  avatar

Watchers

 avatar  avatar hackerman avatar Richard Mörbitz avatar Wayne avatar Benjamin Knorr avatar  avatar

local-dev-env's Issues

Split e2e fixtures from anonymized dump

We shoud probably split the fixtures intended for testing from the dump for a couple of reasons:

  • We can't update the anonymized dump easily
  • We can't run the e2e tests in staging (since they don't have that stuff)
  • It's hard to update the test data

To achieve this, we should IMHO:

  • Add another database (with minimal schema / data) that is intended for automated testing, generated via migrations. This way, the anonymized dump can be exchanged / kept up to date. And the testing database can be rolled back whenever (maybe even similar to django after every test), should also be easily applicable to staging / production deployments

kratos: upgrade sdks

The image of Ory Kratos was updated to the newest state (March 7, 6d83dc9). Let's test it out and upgrade the sdks, as a preparation for implementing SSO.

  • First, test all auth flow (just to be sure it's working in your machine)
  • Open branch in frontend, upgrade ory sdk packages there
  • Open branch in api, upgrade ory sdk packages there
  • Test thoroughly all auth flows
  • If everything ok, open PRs in frontend and api

Latest version of SDK
https://www.npmjs.com/package/@ory/client 1.1.21
Breaking changes
https://www.ory.sh/docs/guides/upgrade/sdk-v1

All auth flows (test please only happy paths)

  • registration
  • verification (go to localhost:4436 to see verification email)
  • password reset
  • login
  • logout

Important Note: No need to test SSO for now. Comment out lines 25-34. We have to improve the frontend code first, since clicking on Anmelden (normal login) leads us to SSO Login.

Initial improvements

  • Make sure rocket chat dump has already basic configuration
  • Script for importing user job
  • Make use of API and DB Layer configurable (image from .env or even if we want to start them)
  • Add formatter
  • Better smtp server image (use oficial mailslurper or a still maintained one)
  • Add basic Actions (v.g. for formatting)

kratos: test all scenarios with SSO and documentate

SSO has to be fully tested, that means:
A. Happy paths

  1. New user clicks login with SSO and gets an account
  2. Existing SSO user clicks login with SSO and gets logged in

B. Sad paths

  1. User register but has a missing attribute (v.g. change jsonnet mapper to only accept claims.email if email exists and try to register user without email). (It is expected that user is prompted to fill the missing fields. After that it should work.)
  2. User unique identifier already exists in serlo (v.g. email is already registered at serlo.org and try to register again with same email)
  3. SSO user tries to login with password now

Please documentate here the results.
Add also other scenarios and unexpected behaviors.

`graphql-fetch-cloudflare-auth.ts`

  • currently jsonwebtoken is not installed in the frontend so the import fails
  • could we the env.local setting in the frontend repo so we don't need to patch this file? (should not be a lot of code right?)

kratos: Add webhook for updating lastLogin

Description

The user last_login from legacy wasn't migrated (see serlo/frontend#2019): API still gets info from a column that doesn't get updated anymore (and this column should stay so).
We want to migrate it to kratos and for that it sounds simpler just to use a non blocking webhook each time someone logs in, updating metadata_public.lastLogin. Of course, it is here just a dry run, after that we should do the changes in cluster.

What should be changed

For testing

  • Run yarn start:without:api
  • In api repo run yarn start and do your changes there
  • Clone frontend repo, follow the readme there, but change .env to use local environment. yarn dev
  • Head to localhost:3000 and login as one user.
  • Check db if lastLogin is set and with correct date.

Nice to have

  • Make a script for importing the last login of existing users to the metadata_public.lastLogin

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.