Code Monkey home page Code Monkey logo

safe-client-gateway's Introduction

Safe Client Gateway

Coverage Status

Motivation

The Safe Client Gateway serves as a bridge for the Safe{Wallet} clients (Android, iOS, Web).

It provides UI-oriented mappings and data structures for easier integration with several Safe{Core} services. In essence, it works as a bridge between the frontend and backend, ensuring smooth, efficient data exchange.

Documentation

Requirements

Installation

Optional: If you have NVM installed, you can run nvm use in the root folder of the project to use the recommended Node version set for this project.

We use Yarn as the package manager for this project. Yarn is bundled with the project so to use it run:

corepack enable && yarn install

The project requires some ABIs that are generated after install. In order to manually generate them, run:

yarn generate-abis

Running the app

  1. Start Redis instance. By default, it will start on port 6379 of localhost.
docker compose up -d redis
  1. Start the Safe Client Gateway
# development
yarn run start

# watch mode
yarn run start:dev

# production mode
yarn run start:prod

Test

The unit test suite contains tests that require a database connection. This project provides a db-test container which also validates the support for SSL connections. To start the container, make sure that the key for the self-signed certificate has the right permissions.

# disallow any access to world or group
chmod 0600 db_config/test/server.key

With the right permissions set on the server.key file we can now start the db-test container:

# start the db-test container
docker compose up -d db-test

# unit tests
yarn run test

# e2e tests
docker-compose up -d redis rabbitmq && yarn run test:e2e

# test coverage
yarn run test:cov

Linter and Style Guide

We use ESLint as a linter and Prettier as a code formatter. You can run yarn run lint to execute ESLint and yarn run format to execute Prettier.

These checks can be automatically executed using Git hooks. If you wish to install the provided git hooks:

yarn install
yarn husky install

safe-client-gateway's People

Contributors

dependabot[bot] avatar fmrsabino avatar hectorgomezv avatar iamacook avatar katspaugh avatar moisses89 avatar schmanu avatar usame-algan avatar uxio0 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

safe-client-gateway's Issues

Use NetworkModule to abstract http call in all services and tests

Summary
Now we have a NetworkModule which exposes a NetworkService via global modules. Lets use this instead of depending directly on axios in the services and tests.

Acceptance criteria
All services and tests should depend on NetworkService instead of axios.

HttpErrorFactory leaks Axios dependency

The current HttpErrorFactory assumes that axios is being used in the project. While this setup works in production it doesn't work in a testing scenario where we want to return an error response.

NetworkResponse should be used instead in HttpErrorFactory.

HTTP Logger format decision/implementation

More context here

Summary
We need to decide and implement a common logger messages format for incoming HTTP calls to the service.

Acceptance criteria

  • Should be as similar as possible to the current log format. (TBD)
  • Should add a requestId which will make possible to link request and response for a specific HTTP call.

Use an automatic fake data generator on tests

Summary
Currently we are generating our test data manually per test. It would be nice to integrate a tool to generate this fake data to favor tests robustness. Faker seems a good candidate for this.

Acceptance criteria

  • An automatic tool should be added as development-time dependency only.
  • Current and further test classes should use this tool instead of manually generate test/fake data.

Implement /chains/<chain_id>/about/backbone

  • Implement /chains/<chain_id>/about/backbone with the same schema as the current Safe Client Gateway.
  • Caching is not required for this ticket.
  • Unit tests should be added for this route.

Add POST Delegates

  • Add POST /v1/chains/<chain_id>/delegates route
  • Tests should be added
  • OpenApi specification should be added

Use automatic API versioning

Summary
We could take advantage of Nest automatic API versioning feature. By using this, route prefixing (v1, v2, etc,) is handled automatically by the framework, by prepending the adequate v to the URLs this way.

After configuring this, we could simply pass a { version: 'X' } configuration object to each controller, or even to each particular route.

This feature is pretty flexible, it can apply multiple versions to each controller/route, and it also allows "version-neutral" controller/routes.

Validate external service calls payloads

Summary
Currently we consume two Safe services: Config Service and Transaction Service. To assure this services are providing the expected payloads, it would be nice to validate those into the Client Gateway.

Technical details
We have several options to implement this:

We need to do a brief research of the pros and cons of each alternative.

Acceptance criteria

  • Define a validation layer which will validate payloads coming from external services against some object interfaces.
  • Throw an error when the validation doesn't pass.

Add unit tests to ExchangeRepository

Add unit testing to ExchangeRepository. Something similar to the tests for other repositories we have should work. These tests should check all the logic contained in the repository, including validation logic.

Improve validation for paginated resource coming from external services

Dependencies

  • Centralise schemas and data validation (TODO: link to issue)

Context here.

Summary
We'd like to validate not only the contents of a Page but the Page itself. This implies we need to implement additional schemas for verifying a Page<?> so probably some composition pattern could be used when writing down the JSONSchemas.

Validate paginated data coming from datasources

More context here.

Currently we are validating data coming in a paginated fashion from Safe Transaction Service and Safe Config Service, but we aren't validating the page format itself. Instead of iterating the array of elements and we can define a Page<T> schema for each T and validate the complete payload using composed JSONSchemas.

Filter endpoints not returning `DATE_LABEL`s

Describe the bug
The following endpoints only return TRANSACTION elements:

  • /v1/chains/{chainId}/safes/{address}/incoming-transfers/
  • /v1/chains/{chainId}/safes/{address}/module-transactions/
  • /v1/chains/{chainId}/safes/{address}/multisig-transactions/

As they are all 'historical' transactions, they should match that of the /v1/chains/{chainId}/safes/{safe_address}/transactions/history endpoint and return DATE_LABELs nested between differing days (relevant to the timezone_offset, see #660.

To Reproduce
Steps to reproduce the behavior:

  • Call each of the endpoints with a filter query and observe that no DATE_LABEL elements are present.

Expected behavior

  • A list of TRANSACTION elements matching that of the desired filter with DATE_LABEL elements between those on different days (relevant to the provided timezone_offset). (Example client side implementation (that was removed) can be found here)

Environment (please complete the following information):

  • Staging and production
  • All chains

Balance schema validation fails on valid balances response

The validation of balances fails on a valid payload. Eg.: GET http://localhost:3000/v1/chains/1/safes/0xA976eA51b9ba3232706aF125a92e32788Dc08Ddc/balances/USD

The error is as follows:

[
    {
      "instancePath": "/token",
      "schemaPath": "#/properties/token/anyOf/0/type",
      "message": "must be null"
    },
    {
      "instancePath": "/token",
      "schemaPath": "balanceToken/required",
      "message": "must have required property 'logo_uri'"
    },
    {
      "instancePath": "/token",
      "schemaPath": "#/properties/token/anyOf",
      "message": "must match a schema in anyOf"
    }
  ]

Implement /balances/supported-fiat-codes

  • Implement /v1/balances/supported-fiat-codes with the same schema as the current Safe Client Gateway.
  • Caching is not required for this ticket
  • Unit tests should be added for this route

Include new threshold in method name

Is your feature request related to a problem? Please describe.
Changing the threshold when adding/removing an owner does not have a clear method name (e.g. addOwnerWithThreshold). Adding an explicit threshold would clarify this.

Current titles are inconsistent:

removing an owner -> removeOwner
adding an owner -> addOwnerWithThreshold
removing an owner with changing the threshold -> removeOwner
adding and owner with changing the threshold -> addOwnerWithThreshold
increasing the threshold -> changeThreshold
decreasing the threshold -> changeThreshold

Provide references to the feature you are implementing that requires this change

Describe the solution you'd like
The following method names are proposed:

  • changeThreshold ({{newThreshold}}/n)
  • addOwner
  • addOwner + changeThreshold ({{newThreshold}}/n)
  • addOwner
  • addOwner + changeThreshold ({{newThreshold}}/n)

If the previous threshold could be included in the method name it would be an added bonus, but {{threshold}}/n is sufficiently clear.

image

Describe alternatives you've considered
It is possible to map these on the client side but it would cause disparity across web/mobile that could lead to confusion.

Dockerize the service

Summary
We need to provide a Docker image to deploy the service in our Kubernetes clusters.

Technical details

  • The provided image should contain production code only, so a multi-stage build is probably the way to go.
  • We use yarn during the development, so it would be nice to use it as well for building the image dependencies.
  • (TBD) Sync with DevOps team about image security (Github/Synk checks, etc.)

Add GET Delegates

  • Add /v1/chains/<chain_id>/delegates?<safe>&<delegate>&<delegator>&<label> route
  • Tests should be added
  • OpenApi specification should be added

Complete Chain model schema

  • Currently our domain Chain entity only has a subset of what is required by the clients
  • We should add the remaining data with the appropriate schemas in place

Refactor external data providers naming

Summary
Currently we're using the term service to refer to the external data providers (Safe Transaction Service, Safe Config Service). Now we we also have an Exchange data provider in place which provides currencies data.

We adopted Nest (an other frameworks as well) convention to place our internal business logic into a services layer. Due to this, we have a name collision between two concepts which should be differentiated: external data providers and internal business logic.

This conflict is even worse thinking about Transaction, because at some point we'll have a Transaction internal service (to manage transaction endpoints/entities) which directly collides with the external Transaction data provider (Safe Transaction Service).

Proposal
The idea here is to drop a proposal and comment/iterate over it:

  • Rename folder src/services to src/datasources (other options: src/external, src/providers).
  • Rename folder config-service to config-api (and it's internal classes accordingly).
  • Rename folder transaction-service to transaction-api (and it's internal classes accordingly).
  • Rename folder exchange to exchange-api (and it's internal classes accordingly).

Integrate caching solution

Similar to the the Rust implementation, a caching solution should be introduced.

  • Redis is already deployed in production so we should keep using the same solution.
  • The implementation used (for caching) should be easily replaceable by a mock/fake for testing
  • Caching should only be done for external requests done by the service

Caching should be done in a more effective manner: see #849

Add GET /safes/{safe_address} route

Transaction Service dependencies:

  • /v1/safes/{}/ #116
  • /v1/about/master-copies/ #98
  • /v1/contracts/{}/ #100
  • /v1/safes/{}/transfers/ #125
  • /v1/safes/{}/multisig-transactions/ #126
  • /v1/safes/{}/all-transactions/ #127

Config Service dependencies:

  • /v1/chains/{}/

Centralise datasource data validations logic

More context here.

Summary
We want to centralise data validation on the domain layer in a single place per repository.

Currently we are using a façade class for AJV (our validation utility): JsonSchemaService, which exposes compile and addSchema functions to the repositories. This forces the callers to store references to each ValidateFunction returned from compile internally as private members. This pollution will increase in the future as we will be adding more repository methods with more schemas.

Acceptance criteria
We need to hide all the interactions with AJV from callers, so we need to create a Validator per repository, which:

  • Encapsulates all AJV interactions, all its types and interfaces. (no more import whatever from 'ajv' outside these classes)
  • Compiles all the schemas with their dependencies regarding the repository, and stores a reference to the associated ValidateFunctions.
  • Links each ValidateFunction to a specific Typescript class (if possible).
  • (Thanks to the previous point, if feasible) It has just one public function: validate(data, Type): void where data is an arbitrary object and Type is a Typescript interface. This function would just return void or use ValidationErrorFactory to create and throw an exception.
  • By having ValidationErrorFactory injected, the caller don't have to catch errors, just fire & forget about validation.

Implement /safes/<safe_address>/balances/<fiat> route

  • Implement /v1/chains/<chain_id>/safes/<safe_address>/balances/<fiat>?<trusted>&<exclude_spam> with the same schema as the current Safe Client Gateway.
  • Caching is not required for this ticket
  • Unit tests should be added for this route

Migrate validation logic to domain layer

Summary
Currently we're doing validation into the datasources layer. Since we're moving all business logic and entities to the domain layer, this will be also the place to perform all validation against our schemas for the data coming through datasources to domain.

Acceptance criteria

  • Validation functions should be hold and used by domain repositories.
  • Data schemas should also live near to the entities they guard, so they should be moved to the domain layer as well.

Add BaseUrl to SafeConfigService

The BaseURL is hardcoded for the SafeConfigService.

This value should be injected instead of being hardcoded. On an initial phase it doesn't need to be read from a configuration/environment variable

Configure Jest code coverage tooling

Summary
We're not keeping track of the code coverage while running tests right now. We need to collect this coverage and output proper reports so the developers could be aware of blind spots in the service.

Acceptance criteria

  • A coverage report can be obtained using a yarn script in the local environment.
  • A coverage report is visible during the CI pipeline execution.

Add support for cursor query parameter

  • Current paginated endpoints should have a cursor parameter (as implemented in the current version of the Safe Client Gateway).
  • We should add support for it while mapping the contents of the cursor to the correct pagination query parameters of the Safe Transaction Service and the Safe Config Service.

Reference implementation:

Implement serialisation and output DTOs

Dependencies

  • Centralise schemas and data validation (TODO: link to issue)

Context here.

Summary
Currently, as shown in the context link, we have some inconsistencies between the current Client Gateway implementation responses and this new service ones. The idea here is, instead of doing ad hoc mapping in the services to the expected formats, having DTOs providing the expected payload format, and implement a serialisation layer which takes "service objects" and converts them to "presentation objects" to fulfil client's requirements.

Note
We could do some additional research here to see what's the best solution, but we're already using AJV to validate incoming payloads so we could use it for serialisation as well.

Add incoming requests logging

Summary
It would be nice to have a minimal logging for every request that the service handles.

Technical details
In Nest.js we could achieve this easily by applying a middleware to the modules we want.

Acceptance criteria

  • Every incoming request the services handles should be logged in.
  • Basic http metadata should be logged in: timestamp, method, route, (more TBD).

Add support for Swagger

  • Swagger support should be added
  • The Json Schema for the response types should be correctly generated

Client gateway should return ENS names in AddressInfo records

Is your feature request related to a problem? Please describe.
This is related to safe-global/safe-pm#39

The client gateway already returns AddressInfo records for known addresses. It would be great if it added reverse ENS lookup results to the name if there is no known address name already.

Provide references to the feature you are implementing that requires this change
Provide at least one of the following:

Describe the solution you'd like
The client gateway already returns AddressInfo records for known addresses. It would be great if it set the name field to reverse ENS lookup results if there is no known address name already. To prevent imposters the CGW would also need to do the forward lookup and check that the address returned is the same. The CGW could use caching as described in https://docs.ens.domains/dapp-developer-guide/front-end-design-guidelines#caching-and-updating-ens-names to minimise requests.

Describe alternatives you've considered
We considered doing the lookup in the client. Doing it in the CGW has (at least) the following advantages:

Caching
But the client gateway would be able to do better caching. As I understand it the client would need to do at least 2 requests for each address. One for the reverse lookup and one for the forward lookup to prevent imposters(, if it is done for the first time and doesn't match the previous result).

Logic in one place
What name to use if there is several (server) sources could be handled in one place and affect mobile and Web.

Less implementation effort in the mobile clients
Doing this in the CGW would mean no changes are necessary to display these names in the mobile clients. (It would just work on Android)

Additional context

  • This can also be used by the web client.
  • Additionally the AddressInfo.logoUri() could contain the ENS avatar if there is one

Tx history table - Date labels can be wrong in some timezones

Bug description

When there is no filter applied, the date label that groups tx in table is 1 day off from the date of the tx; A tx created.
It might have to do with the timezone. This was tested in the GMT -3.00

Environment

  • Browser: Chrome
  • Wallet: MetaMask
  • Chain: Any

Steps to reproduce

  1. Go to this safe that has a bunch of tx's https://web-core.pages.dev/rin:0x9913B9180C20C6b0F21B6480c84422F6ebc4B808/transactions/history
  2. Search for the tx on Sept 6 (the label in the table)
  3. Hover the date in the tx

Expected result

It should also be at some time of the day on the sept 6

Obtained result

The tx are on Sept 7.

Screenshots

image

NOTE:
There is a case where the date in the tx and the label in the history is fine, I assume because the tx was done near midnight:
image

Missing or incorrect index of `DATE_LABEL`s when timezone offset

Describe the bug
It is possible to adjust the timestamp of DATE_LABELs in the transaction history via the timestamp_offset param, however, they can be missing or at the wrong index. They are returned relative to the UTC timestamps of the TRANSACTIONs.

Missing DATE_LABELs

Here is an example transaction list (in UTC):

[0] `DATE_LABEL`
[1] `TRANSACTION` (02/01 @ 02:00),
[2] `DATE_LABEL`
[3] `TRANSACTION` (03/01 @ 02:00),
[4] `TRANSACTION` (03/01 @ 04:00)

When converted to UTC -3 via the timezone_offset param:

[0] `DATE_LABEL`
[1] `TRANSACTION` (01/01 @ 23:00),
[2] `DATE_LABEL`
[3] `TRANSACTION` (02/01 @ 23:00),
 <= missing `DATE_LABEL`
[4] `TRANSACTION` (03/01 @ 01:00)

Incorrect DATE_LABEL indexes

Here is an example transaction list (in UTC):

[0] `DATE_LABEL`
[1] `TRANSACTION` (02/01 @ 12:00),
[2] `DATE_LABEL`
[3] `TRANSACTION` (03/01 @ 02:00),
[4] `TRANSACTION` (03/01 @ 04:00)

After being offset, the DATE_LABEL at [2] in at the incorrect index. It should be exchanged with the TRANSACTION at [3]:

[0] `DATE_LABEL`
[1] `TRANSACTION` (02/01 @ 7:00),
[2] `DATE_LABEL` <= wrong index `DATE_LABEL`, should be at [3]
[3] `TRANSACTION` (02/01 @ 23:00),
[4] `TRANSACTION` (02/01 @ 01:00)

To Reproduce

  • Observe no index change between /v1/chains/4/safes/0x9913B9180C20C6b0F21B6480c84422F6ebc4B808/transactions/history and /v1/chains/4/safes/0x9913B9180C20C6b0F21B6480c84422F6ebc4B808/transactions/history?timezone_offset=39600.
  • Set machine timezone to be be -n UTC. Open rin:0x9913B9180C20C6b0F21B6480c84422F6ebc4B808/transactions/history in the Safe and observe days spanning across one date label (07/09).

image

Expected behavior

  • DATE_LABELs are nested between transactions on different days, relevant to the timezone offset. (Example client side implementation (that was removed) can be found here)

Environment (please complete the following information):

  • Staging and production
  • All chains

Create wrapper around ConfigModule

A wrapper should be created around Nest's ConfigModule.

The reason is that this module reads from the local environment which is external to the service and in order to provide a better testing setup we should be able to replace the ConfigModule with a fake one.

Shutdown hook not working while trying to connect to Redis

When Redis is not available the service retries to connect repeatedly. After sending a Control-C (SIGTERM) the process still running and trying to connect to the Redis server.

We are currently using Nest hooks, we need to investigate why they is not shutting down the process when SIGTERM happens.

  // Starts listening for shutdown hooks
  app.enableShutdownHooks();

Image

Use a logger instead of the console

Summary
We need to use a production-grade logging tool in the service, so the idea is to replace the current usage of console.log/console.error/etc. with a proper tool.

Technical details
We have the option to use the logger integrated in the Nest.js framework, which could be the first option. Another options would be to use a logger provided by an external npm package like Pino or Winston.

We need to evaluate if the integrated Nest.js logger covers our needs during the implementation of this issue, and take a decision on which logger fits better.

Acceptance criteria

  • All the usages of the embedded JS console are replaced by a proper logger.

Extract logic for http error mapping

Currently we don't have a centralised place to handle errors coming from the Config/Transaction service. A common handler can be implemented to manage these errors and provide a common/normalised behaviour when they happen, so we can also provide a standard response to the clients.

Acceptance criteria

  • Logic for handling Config/Transaction service errors will be extracted to a common handler.
  • The handler should provide a standard payload format to inform errors to the clients.
  • Error codes and messages coming from Config/Transaction services should be provided to the clients.
  • On unexpected/unknown error the service should return http 503 to the clients.

Research how to abstract the HttpClient for testing

The current endpoints are exposed to the HttpModule provided by NestJS. Even though this works it makes testing difficult as this is one component that we'd like to mock/provide fake data (remote http calls).

An alternative solution should be considered where we avoid mocking the axios client directly (which ties our tests directly with the http client being used)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.