compasjs / compas Goto Github PK

Think about ways of 'fuzz' testing code-gen. Some way could be to define the structure of the core types and plugins as mocks and then generate random schema's and verify those with Typescript and via validators

Depends on: testing-library/react-hooks-testing-library#68

Tests will fail if another instance is using the 'normal' database. This happens because Postgres can only use a database as a template when it has no connections.
There should be a way to froce drop these connections.

Current stubs generation is pretty unreadable even after running Prettier.
Minimize empty lines in a single Jsdoc block

Check if we can support importing & transforming OpenAPI documents in to LBU structures

It is an experimental package without too much updates yet. It also contains some unnecessary dependencies.
Which results in things like #103

Caching small files from S3 in memory & bigger files on local disk will improve response times for static files.

At least an easy to use handler in @compas/server

We can then leave the current getMigrationsToBeApplied internal only and expose the friendly variant

For a small project the generated validator file is around 4000 lines. Check if we can reuse anonymous validator functions

When an Error is thrown during a (tape) test, the error stack of the test runner is printed instead of the location of the source code

Minio doesn't support bucket names with an _ and for postgres we need to escape the database name everywhere

We have an opinionated prettier, now take the ESLint rules to enforce modern and fast JS
e.g.
https://github.com/benmosher/eslint-plugin-import/blob/master/docs/rules/order.md

See https://github.com/porsager/postgres/releases/tag/v2.0.0-beta.0

With the new relation type we should be able to at least generate the different permutations for simple left joins.
From there we can also look at combining or even nesting them

Currently the apiClient code-gen will use any for the params if it detected param usage in the route path.
We can easily do that while building a RouteBuilder

Generate a api client that is using react-query. Most likely going to depend on the existing api client, as to not duplicate efforts, and correct managing of the mocks, X-Request-Id etc.

The options property in the typescript definition for the session middleware (@lbu/server) is incomplete.
Found that sameSite is missing.

Should be in sync with; https://github.com/koajs/session#example

TypeBuilders are not processed when calling app.add(), but only when calling either app.generate()or app.generateStubs().

The options.group validation for generateStubs happens before calling the internal processing, so it will always throw.
Current workaround is to call app.generate() with an ignored output directory

Changed...

Add opt-in functionality to generate queries for row-level versioned tables.
Another way is to add some generic triggers in store so the user can enable it via migrations

Typescript doesn't want to resolve the types when not imported.
So add import("./types"). before every use of a type.
Applicable for apiClient and validator plugins

Currently probably ignored when publishing to npm

Generators have some overlap in in how they work through the structure, and build up a file.
Some parts that need improvement:

• Check if we can drop the @lbu/stdlib templates and use something else like EJS or more minimalist. Alternatively we can also take a look at just using template literals.
• Common way of specifying imports
• Useful error handling, more isNil checks

Just some things again, there should be some ways to improve DX here

TODO's from #142 (comment)

• Accept isClient and isServer properties and try to infer them in App.generate
• Add better type generation based on these properties and their possible usages
  A good example of this is the file type in combination with the apiClient and router generators
  • If the apiClient is generated for isNode the resulting response should be ReadableStream
  • If the router is generated, most of the times we want the custom object type with a path, name etc properties
    However both of these case use the same non-_Input type currently
• Validate more phases in the generation by using self hosted validators
• Consistent type naming in templates

Add a koa-session compatible, PostgreSQL backed, session store.

See koa-session docs

When passing an array instead of a spread array, the v const in the typebuilder build function is of type array instead of object. So v.build() fails.

Bad example object;

app.add(
    T.object("Object", {
        property: T.anyOf().values([
        T.object("object_1", {
            id: T.uuid().primary(),
        }),
        T.object("object_2", {
            id: T.uuid().primary(),
        }),
        T.object("object_3", {
            id: T.uuid().primary(),
        }),
    ]),
});

There are a few big sources of overhead

• a new logger on every request
• koa-compose adds overhead

Check if we can improve the current situation or if we can compose more in line of:

const middleware = fn => async ctx => {
  await fn(ctx);
  const body = await ctx.getBody();
  ctx.foo = "bar";
}

We can remove koa-compose overhead while also removing some unexpected 'magic' from the router

Issue with inline params, instead of binded/prepared once. This was due an issue in the vendor postgres library. (porsager/postgres#58)

Should be done right (packages/code-gen/src/generators/sql/templates/sqlWhereClauseUnsafe.tmpl):
query += ANY (ARRAY['${where.{{= it.name }}.join("\', \'")}']::uuid[]); L22

Currently uuid & date type rely on the string type for basic checks and parsing. However errors thrown by the generated stringValidator uses validator.string.x. This is not clear for the api consumer.

We can solve this with something like:

function stringValidator11(value, propertyPath, parentType = "string") {
 // ...
}

function uuidValidator10(value, propertyPath, parentType = "uuid") {
  const validString  = stringValidator11(value, propertyPath, parentType);
  // throws validator.uuid.min instead of validator.string.min
}

Allow milliseconds / seconds since unix epoch?

Usage:

const T = new TypeCreator();

T.date()
  .ISOOnly() // Don't allow milliseconds since epoch
  .defaultToNow() // Set default("new Date()")

An overview of documentation that needs to be written:

@lbu/*

• What a feature should be to be added to these packages

@lbu/cli

• General overview of how it works, where it finds scripts and how it executes those
• Possible edge cases for passing arguments
• How the lint command works
• How the test command works
• How the coverage command works
• How the provided t;emplate is setup

@lbu/lint-config

• Dependencies used
• Arguments that are passed to eslint & prettier
• How to overwrite it's config

@lbu/code-gen

• A complete getting started for backend developers
  • Should contain App.add, addRaw, extend
  • Should contain inferring types
• A coplete getting started for frontend / app developers
  • Should contain apiClient, types, react-query, validator plugins
• Tutorial on writing a custom type
  • What the build in plugins do with custom types
• Tutorial on writing a custom generator
• Better documentation in api.md for all options and what those do

@lbu/insight

• How to use the logger

@lbu/server

• Dependencies used
• What it can it does and specific usecases
• How to test an server

@lbu/stdlib

• A list with alternative names for the provided functions / make them easier to find

@lbu/store

• How to test with Postgres
• How to test with Minio
• How to use migration system
• How to save files and use the file cache
• How to persist session info ( link with @lbu/server or should be in @lbu/server )
• How the queue works and specific use cases like interval based jobs and priorities

References #41

• fix rawMock in TypeBuilder.prototype.mock
• Also use rawMock in mockExec and skip the rest of the function

The postgres create insert template does not evaluates all property types correct. The default now is JSONB and thats why for example T.string() results in JSONB. Issue in;

compileSqlExec @lbu/code-gen/src/generators/sql/generator.js

Type creator optional function on reference field does not reflect in correct dumped queries.

 T.object("test", {
    id: T.uuid().primary(),
    entityId:  T.reference("namespace", "table")
      .field("id", "table_id")
      .searchable()
      .optional()
});

results;

CREATE TABLE test (
    id              UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
    entity_id       UUID NOT NULL REFERENCES table(id) ON DELETE CASCADE
)

expected NOT NULL to be left out

Currently when using yarn lbu help it will also list README as a possible script. Needs to be filtered out

• code-gen:
  • router: add array of middleware support in JSDoc
  • validator: add option to disable generator based on property on a 'type' (e.g. sql input doesn't need validator)
  • mock: add option to disable generator based on property
  • sql: check if = ANY() may be better to use than IN
  • sql: empty array for idIn results in syntax error in x IS NULL OR ...
  • react-query: move react-query to 2.0
  • api client: use axios interceptors for xRequestId
  • generate Upsert queries for searchable() fields , mentioned; https://github.com/lightbase-platforms/components/pull/129#discussion_r444834824
• stdlib:
  • add uuid.isValid function

• Should use the file enabled body parser in generated router
• Should use special types in the apiClient so it is still frontend and backend agnostic

All exposed api's should be easily explorable by the end-user. The current situation is not picked up by VSCode. We can possible aid those users more by automatically creating a .d.ts. file for all packages based on the JSDoc of the exported items.

The react query generated template (and others) do various imports. But not always are all of these imports used.
Github extracts these warnings (from the lint actions) and shows these inline. Maybe we should do an eslint disable next line/block for those import statements.

Depends on TanStack/query#626

Also make better use of dependent queries

Ping @dov11

type CustomError = Error & { response?: { data: { key: string } } };
export function useMyRoute(
  options: MutationOptions<T.MyRouteResponse, UseMyRouteProps, CustomError> = {},
): [
  MutateFunction<T.MyRouteResponse, UseMyRouteProps>,
  MutationResult<T.MyRouteResponse, CustomError>,
] {
  return useMutation(variables => myApi.route(variables.body), options);
}

Possible things to take in to consideration:

• AppError conversion done in packages/server/middleware/error.js
• AxiosError