Implementation of zoning district adjacent endpoints (Drizzle)

Description

Implement zoning district endpoints using Drizzle. The drizzle endpoints implementation may have outpaced the implementation in Mikro-orm. If the endpoint is not yet suppporeted in Mikro-orm, then the Feature flag route for Mikro Orm should throw an error.

Acceptance criteria

Description

From discussion #55, document the possible errors for zoning district class endpoints

Acceptance criteria

The following endpoints describe their listed error codes

Related tickets

Blocked by #67

Relevant documentation

The Reusing Responses documentation describes how to use response components to describe a response on an endpoint. Use the appropriate responses created in ticket #67 to describe the responses for these endpoints.

Implement Zod schemas based on OpenAPI document

Description

From discussion #55, we need to validate route parameters. We want to base this validation on the route params detailed in the openapi.yaml file. We are going to use Kubb to generate types and zod schemas based on the openapi spec file. This initial ticket is to create the schemas used for validation, and to implement the 400 validation response for one endpoint. It will be followed by other tickets to actually implement the validation for other endpoints.

Acceptance criteria

Install and configure necessary Kubb packages to generate typescript types and Zod schemas based on OpenAPI spec document.
Generate types and schema code with Kubb
Implement custom Nest validation pipe that uses Zod for validation based on Nest example
Implement route param validation for the /tax-lots/<bbl> endpoint to serve as reference code for other endpoints

Open API documentation for colors missing opacity

Description

The description for the colors types omit the opacity from the format and the example. This is reflected in the length of the regex and the structure of the example. These need to be updated.

Also, the index.html has not been rerendered to reflect the restructuring of the category colors.

Acceptance criteria

The regex for the category color format is increased from 6 to 8 characters
- in category colors
- in land use colors
The example color has an opacity of ff appended to it
- in category colors
- in land use colors
The index.html is regenerated with npm run redoc:build

Implement borough and land use error responses (mikro)

Description

From discussion #55, we need to implement error handling for borough and land use endpoints. Database access is the primary point of errors. This is accomplished differently in drizzle and mikro. Consequently, they will be implemented separately. This is the ticket to implement mikro.

Acceptance criteria

Error handling should be done within the drizzle featureFlag whenever it is an orm-specific handler.
Custom exceptions implemented in ticket #72 should be used whenever applicable
boroughs
- Throws a 500 when there is an error accessing the database
land-uses
- Throws a 500 when there is an error accessing the database

Related tickets

blocked by #68 to document the endpoints
blocked by #72 to implement the messages to throw

Implement zoning district error responses (mikro)

Description

From discussion #55, we need to implement error handling for zoning district endpoints. Schema enforcement and database access are primary points of errors. Each of these are accomplished differently in drizzle and mikro. Consequently, they will be implemented separately. This is the ticket to implement mikro.

Acceptance criteria

Related tickets

blocked by #70 to document the endpoints
blocked by #72 to implement the messages to throw
blocked by #74 to implement zod schemas

Serving OpenAPI documentation

Acceptance Criteria

Generate html with Redoc
Serve Static on Next endpoint with base path of root.

Endpoint implementation: Refactor endpoints that return top-level arrays to instead return objects (Drizzle and Mikro)

Description

Discussion #58 outlines the pitfalls of returning top-level arrays and suggests a refactor to instead return an object in which the array is held as a value in an object. This issue is to update the existing implementations to reflect this structure. (Drizzle and Mikro each have their own queries to implement these endpoints. Though, not all of the endpoints have been implemented in Mikro. If Mikro has implemented this endpoint, then it should be updated to reflect the structure. If Mikro has not implemented this endpoint, it does not need to be updated).

Related Ticket

This work is blocked by the Open API documentation in #59

Acceptance criteria

The implementations for the following endpoints are updated to return objects with the following structures:

/boroughs
- { boroughs: Array<borough> }
/land-uses
- { landUses: Array<landUse> }
tax-lots/<bbl>/zoning-districts
- { zoningDistricts: Array<zoningDistrict> }
tax-lots/<bbl>/zoning-districts/classes
- { zoningDistrictClasses: Array<zoningDistrictClass> }
zoning-districts/<uuid>/classes
- { zoningDistrictClasses: Array<zoningDistrictClass> }
zoning-district-classes
- { zoningDistrictClasses: Array<zoningDistrictClass> }
zoning-district-classes/category-colors
- { zoningDistrictCatagoryColors: Array<zoningDistrictCatagoryColor> }

Add boroughs endpoint OpenAPI Schema

Add the OpenAPI schema for the /boroughs endpoint to the Zoning API.

Implement tax lot error responses (drizzle)

Description

From discussion #55, we need to implement error handling for tax lot endpoints. Schema enforcement and database access are primary points of errors. Each of these are accomplished differently in drizzle and mikro. Consequently, they will be implemented separately. This is the ticket to implement drizzle. The code examples are meant to show general structure. Actual implementation may vary.

Acceptance criteria

Related tickets

blocked by #69 to document the endpoints
blocked by #72 to implement the messages to throw
blocked by #73 to implement zod schemas

MikroORM Multipolygons

Description

In support of tax lot and zoning district tables, Mikro Orm will need multipolygon geometry and geography types

Acceptance criteria

A multipolygon geometry type is usable for table schemas
- Support multiple coordinate reference systems
A multipolygon geography type exists
- Support multiple coordinate reference systems
- ~~returns json compatible coordinates~~

Related issues

Blocks #2 and a yet to be created zoning district ticket

Open API documentation for tax lot adjacent endpoints

Description

Open API documentation as outlined in #21, second ticket of three

Acceptance Criteria

Is blocked by

#25

OpenAPI documentation: Mark schema properties in Open API as required

Description

Mark all properties guaranteed to be returned by the API as "required" in the OpenAPI documentation

Acceptance Criteria

All properties guaranteed to be returned by successful API calls are marked as required.

Discussed in #87

^{Originally posted by TangoYankee December 6, 2023}
@TylerMatteo Should we be more diligent in marking properties/parameters as required in the Open API documentation? I noticed that when we neglect to mark them, Kubb marks them as possible undefined. I think this will lead to more defensive programming than necessary- checking for undefined values where they will never reasonably occur (because an error would have occurred first).

Relevant to any @NYCPlanning/open-source-engineering writing documentation

Refactor: Boroughs and Land Uses - Move database access from service to repository layer (drizzle)

Description

From discussion #63, our placement of database calls in the service files violates the separation of concerns between services and repositories. Services should handle the business logic and repositories should handle the database access.

We want to move the database access from the service to a repository. This is a drizzle-specific implementation. As such, we will need a drizzle specific repository. Mikro-orm already has the repository namespace: ie) borough.repository.ts with a class named BoroughRepository. Consequently, drizzle will use the abbreviated repo namespace to avoid a conflict ie) borough.repo.ts with a class named BoroughRepo.

Acceptance criteria

boroughs
- There is a file created within the borough folder called borough.repo.ts
- The borough.repo.ts file exports a class called BoroughRepo
- The BoroughRepo class has db property with access to the drizzle DB provider (tax lot example)
- Code segments within the borough service that directly access the database are moved to methods within the BoroughRepo class
  - The try-catches wrapping the database access should also be moved to the method inside the BoroughRepo
- The borough service calls the newly created BoroughRepo methods
- Add BoroughRepo to list of providers in the borough module
land-uses
- There is a file created within the land-use folder called land-use.repo.ts
- The land-use.repo.ts file exports a class called LandUseRepo
- The LandUseRepo class has db property with access to the drizzle DB provider (tax lot example)
- Code segments within the land use service that directly access the database are moved to methods within the LandUseRepo class
  - The try-catches wrapping the database access should also be moved to the method inside the LandUseRepo
- The land use service calls the newly created LandUseRepo methods
- Add LandUseRepo to list of providers in the land use module

Implement tax lot endpoints (drizzle)

Description

In continuation of #46 , we will implement tax-lot related end points using Drizzle. This will require spatial types, schemas, and the endpoints themselves. An example of the spatial types can be found in the drizzle-pgis folder of purple-hawk

Acceptance critera

Spatial types are created within a drizzle-pgis folder, inspired by purple-hawk
Drizzle schema are implemented for
- land use
- tax lot
Migrations are created to apply these schema to the database
The following endpoints for tax lots are implemented in Drizzle
- land-uses
- tax-lots/<bbl>
  - non-spatial tax-lot fields
    - land use value should be a nested land use object
    - borough value should be a nested borough object
- tax-lots/<bbl>/geojson
  - non-spatial fields as properties
    - land use value should be a nested land use object
    - borough value should be a nested borough object
  - wgs84 as geometry

Description

From discussion #55, document the possible errors for borough and land use endpoints

Acceptance criteria

Related tickets

Blocked by #67

Relevant documentation

The Reusing Responses documentation describes how to use response components to describe a response on an endpoint. Use the appropriate responses created in ticket #67 to describe the responses for these endpoints.

Implement the boroughs endpoint

Description

implement the boroughs endpoint in compliance with the openapi documentation
Utilize joi to validate requests to the endpoint

Description

From discussion #55, we need to implement error handling for zoning district class endpoints. Schema enforcement and database access are primary points of errors. Each of these are accomplished differently in drizzle and mikro. Consequently, they will be implemented separately. This is the ticket to implement drizzle.

Acceptance criteria

Related tickets

blocked by #70 to document the endpoints
blocked by #72 to implement the messages to throw
blocked by #73 to implement zod schemas

Add Borough db table schema to Zoning API

Description

Create a schema that mirrors the borough table of Data Engineering #302

Implement zod schemas (mikro)

Description

From discussion #55, we need to validate route parameters. To facilitate this validation, we will first create zod schemas for the route parameters. (These schemas will apply specifically to mikro-orm because drizzle will have its own schemas generated from its own package.)

Acceptance criteria

Implement borough and land use error responses (drizzle)

Description

From discussion #55, we need to implement error handling for borough and land use endpoints. Database access is the primary point of errors. This is accomplished differently in drizzle and mikro. Consequently, they will be implemented separately. This is the ticket to implement drizzle.

Acceptance criteria

Error handling should be done within the drizzle featureFlag whenever it is an orm-specific handler.
Custom exceptions implemented in ticket #72 should be used whenever applicable
boroughs
- Throws a 500 when there is an error accessing the database
land-uses
- Throws a 500 when there is an error accessing the database

Related tickets

blocked by #68 to document the endpoints
blocked by #72 to implement the messages to throw

Implement endpoints to redirect to tilesets

Description

We want frontends to be able to reference a single service for their data needs. However, the vector tilesets are hosted under a digital ocean space with a separate url. To consolidate these services from the perspective of frontends, we want the api to redirect to the spaces

Acceptance criteria

There are endpoints in the api that redirect to the tilesets
- /api/zoning-districts/<z>/<x>/<y>.pbf Redirects to the zoning district tileset where z, x, and y are zoom parameters
- /api/tax-lots/<z>/<x>/<y>.pbf Redirects to the tax lot tileset, where z, x, and y are zoom parameters
The redirect code is 302

Add Land Use db table schema to Zoning API

Description

Implement a schema to reflect the land use table in data-engineering#302

Open API documentation: tilesets

Description

Document existence of tilesets endpoints in the api

Implementation of zoning district endpoint

Description

Implement the zoning district endpoint

Acceptance criteria

An endpoint exists that fulfills the OpenApi description outlined in #25
- Omit tests to expedite development

Description

Add the schemas related to the zoning district tables outlined in Data Engineering 302 ERD

Acceptance Criteria

There is an entity to implement the zoning_district table
- The spatial types (geometry and geography) are blocked until #16 is merged
- On branch 2/tax-lot-schema, the tax-lot entity demonstrates how to use the spatial types
There is an entity to implement the zoning_district_class table
There is a Many to Many relationship between zoning_district and zoning_district_class
There is a single migration that creates these two entities and their relationship

Load application-related data into database

Add Zoning District Class and many to many relation db table schemas to Zoning API

Implementation of tax lot adjacent endpoints (Mikro-orm)

Description

Implement the tax lot adjacent endpoints

Acceptance criteria

Endpoints exist that fulfills the OpenApi description outlined in #26
- Omit tests to expedite development

Description

From discussion #55, we need to implement error handling for tax lot endpoints. Schema enforcement and database access are primary points of errors. Each of these are accomplished differently in drizzle and mikro. Consequently, they will be implemented separately. This is the ticket to implement mikro.

Acceptance criteria

Related tickets

blocked by #69 to document the endpoints
blocked by #72 to implement the messages to throw
blocked by #74 to implement zod schemas

Open API documentation: Create the error components

Description

From discussion #55 , create the reusable components of the errors. This includes the error schema and the three responses that use it.

Acceptance criteria

There is an error schema following the general structure of the code sample
There are error responses following the general structure of the code sample
- Bad Request
- Not Found
- Internal Server Error

Sample code

If copying code directly, be careful to adjust indentation to valid depths. Also, be mindful that the schema key may already exist in the documentation. The error schema should be placed within the existing schema key, rather than creating a new one.

components:
  responses:
    Bad Request:
      description: Invalid client request.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
    Not Found:
      description: Requested resource does not exist or is unavailable.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
    Internal Server Error:
      description: Server side error.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'

  schemas:
    Error:
      type: object
      properties:
        statusCode:
          type: number
        message:
          type: string
       error:
          type: string
      required:
        - statusCode
        - message

Implementation of zoning adjacent endpoints (mikro orm)

Description

Implement the zoning adjacent endpoints

Acceptance criteria

Endpoints exist that fulfill the OpenApi description outlined in #27
- Omit tests to expedite development

Description

Per ERD, the Zoning District Class should also include the URL property

DE Issue#302

Acceptance Criteria

Open API endpoint with 200 responses are written for the following path:
- zoning-district-classes
  - returns all rows of class schemas including the url property

Endpoint implementation: Refactor Zoning District Classes Category Colors endpoint to return an object (Drizzle and Mikro)

Description

The zoning-district-classes/category-colors endpoint was overlooked in #88. The documentation for the endpoint has been previously updated.

Related Ticket

Work to refactor affected endpoints: #88

Acceptance criteria

The implementation for the following endpoint is updated to return an object with the following structure:

zoning-district-classes/category-colors
- { zoningDistrictClassCategoryColors: Array<zoningDistrictClassCategoryColor > }

Add Zoning District and Zoning District Class schemas (Drizzle)

Description

Create the drizzle schemas for zoning district, zoning district classes, and their relationships. These schemas are outlined in the ERD written on Data Engineering ticket 302

Acceptance criteria

There is a schema for zoning districts
There is a schema for zoning district classes
There is a schema to hold the relationship between zoning districts and zoning district classes.
There is a migration to apply these changes to the database
The load.sql file is updated to load the relevant data into the the database

Open API documentation: Document the tax lot error responses

Description

From discussion #55, document the possible errors for tax lot endpoints

Acceptance criteria

The following endpoints describe their listed error codes

tax-lots/<bbl>
- 400: Bad request
- 404: Not Found
- 500: Internal Server Error
tax-lots/<bbl>/geojson
- 400: Bad request
- 404: Not Found
- 500: Internal Server Error
tax-lots/<bbl>/zoning-districts
- 400: Bad request
- 404: Not Found
- 500: Internal Server Error
tax-lots/<bbl>/zoning-districts/classes
- 400: Bad request
- 404: Not Found
- 500: Internal Server Error

Related tickets

Blocked by #67

Relevant documentation

The Reusing Responses documentation describes how to use response components to describe a response on an endpoint. Use the appropriate responses created in ticket #67 to describe the responses for these endpoints.

Refactor: Tax Lots - Move database access from service to repository layer (drizzle)

Description

From discussion #63, our placement of database calls in the service files violates the separation of concerns between services and repositories. Services should handle the business logic and repositories should handle the database access.

We want to move the database access from the service to a repository. This is a drizzle-specific implementation. As such, we will need a drizzle specific repository. Mikro-orm already has the repository namespace: ie) tax-lot.repository.ts with a class named TaxLotRepository. Consequently, drizzle will use the abbreviated repo namespace to avoid a conflict ie) tax-lot.repo.ts with a class named TaxLotRepo.

Acceptance criteria

tax-lots
- There is a file created within the tax-lot folder called tax-lot.repo.ts
- The tax-lot.repo.ts file exports a class called TaxLotRepo
- The TaxLotRepo class has db property with access to the drizzle DB provider (tax lot example)
- The #checkTaxLotByBbl prepared query statement is moved to the TaxLotRepo.
  - The execution of the prepared statement should accessed via method on the TaxLotRepo. (In line with the next listed acceptance criteria)
- Code segments within the tax lot service that directly access the database are moved to methods within the TaxLotRepo class
  - The try-catches wrapping the database access should also be moved to the method inside the TaxLotRepo
- The tax lot service calls the newly created TaxLotRepo methods
- Add TaxLotRepo to list of providers in the tax lot module

Integrate drizzle as a feature-flagged option

Description

As a developer, I would like an opportunity to compare Mikro-orm and Drizzle orm. Mikro orm is already integrated into the application but Drizzle is not. I would like to install drizzle orm. Because applications should only be served by a single orm at a time, the use of drizzle should be kept behind a feature flag. Future tickets will implement the full application functionality. However, this ticket should also have an MVP endpoint, to verify a successful installation. An example of integrating drizzle with nest is available at @tangoyankee/purple-hawk

Acceptance criteria

Open API documentation for zoning district adjacent endpoints

Description

Open API documentation as outlined in #21, third ticket of three

Acceptance Criteria

Open API schema is written for zoning district classes with fields that mirror the zoning district class entity
- (Note: zoning district schema is already implemented and can be reused throughout this ticket)
Open API endpoints with 200 responses are written for the following paths:
- tax-lots/<bbl>/zoning-districts/classes
  - returns class schemas for all zoning districts that spatially intersect the tax lot
- zoning-districts/<uuid>/classes
  - returns all class schemas for the specified zoning district
- zoning-district-classes
  - returns all rows of class schemas
- zoning-district-classes/category-colors
  - list of color and the class category it represents (may or may not be its own Open API schema)
- zoning-district-classes/<id>
  - returns class schemas for the specified class

Is blocked by

#25

Open API documentation for zoning district endpoint

Description

Open API documentation as outlined in #21, first ticket of three

Acceptance Criteria

An Open API schema is written for zoning districts with fields that mirror the zoning district entity
- exclude the spatial fields (wgs84 and li_ft)
An Open API endpoint with a 200 response is written for the following path:
- zoning-districts/<uuid>
  - returns non-spatial fields of specified zoning district

Blocks

#26
#27

Implement zoning district error responses (drizzle)

Description

From discussion #55, we need to implement error handling for zoning district endpoints. Schema enforcement and database access are primary points of errors. Each of these are accomplished differently in drizzle and mikro. Consequently, they will be implemented separately. This is the ticket to implement drizzle.

Acceptance criteria

Related tickets

blocked by #70 to document the endpoints
blocked by #72 to implement the messages to throw
blocked by #73 to implement zod schemas

Documentation implementation: Refactor documentation for endpoints that return objects instead top-level arrays

Description

Update documentation for endpoints to returning an object in which the array is held as a value in an object.

Related Ticket

Work to refactor affected endpoints: #88

Acceptance criteria

The documentation for the paths => /<path> => response => description following endpoints are updated to indicate endpoint will return return objects:

/boroughs
description: An object containing a list of boroughs
/land-uses
description: An object containing a list of land uses
tax-lots/<bbl>/zoning-districts
description: An object containing zoning districts
tax-lots/<bbl>/zoning-districts/classes
description: An object containing zoning district class schemas
zoning-districts/<uuid>/classes
description: An object of class schemas for the zoning district
zoning-district-classes
description: An object containing list of all zoning district class schemas
zoning-district-classes/category-colors
description: An object containing list of all zoning district category colors

Add Tax Lot db table schema to Zoning API

Description

A schema exists that reflects the tax lot table in Data Engineering #302

Description

From discussion #55, we need to implement error handling for zoning district class endpoints. Schema enforcement and database access are primary points of errors. Each of these are accomplished differently in drizzle and mikro. Consequently, they will be implemented separately. This is the ticket to implement mikro.

Acceptance criteria

Related tickets

blocked by #70 to document the endpoints
blocked by #72 to implement the messages to throw
blocked by #74 to implement zod schemas

Make PostGIS install part of repository setup

OpenAPI documentation: Refactor endpoint that return top level arrays to instead return objects.

Description

Discussion #58 outlines the pitfalls of returning top-level arrays and suggests a refactor to instead return an object in which the array is held as a value in an object. This issue is to update the OpenAPI documentation to reflect this structure.

Acceptance criteria

Open API documentation for the following endpoints are updated to reflect returning objects with the following structures

/boroughs
- { boroughs: Array<borough> }
/land-uses
- { landUses: Array<landUse> }
tax-lots/<bbl>/zoning-districts
- { zoningDistricts: Array<zoningDistrict> }
tax-lots/<bbl>/zoning-districts/classes
- { zoningDistrictClasses: Array<zoningDistrictClass> }
zoning-districts/<uuid>/classes
- { zoningDistrictClasses: Array<zoningDistrictClass> }
zoning-district-classes
- { zoningDistrictClasses: Array<zoningDistrictClass> }
zoning-district-classes/category-colors
- { zoningDistrictClassCategoryColors: Array<zoningDistrictClassCategoryColor> }

Example

The actual schema change will be to take the existing schema and nest it inside a property. For example, borough will go from:
main

          content:
            application/json:
              schema: 
                type: array
                items: 
                  $ref: '#/components/schemas/Borough'

to:
example branch

          content:
            application/json:
              schema: 
                type: object
                properties: 
                  boroughs:
                    type: array
                    items: 
                      $ref: '#/components/schemas/Borough'

Open API documentation: Document the zoning district error responses

Description

From discussion #55, document the possible errors for zoning district endpoints

Acceptance criteria

The following endpoints describe their listed error codes

Related tickets

Blocked by #67

Relevant documentation

The Reusing Responses documentation describes how to use response components to describe a response on an endpoint. Use the appropriate responses created in ticket #67 to describe the responses for these endpoints.

Implement reusable exceptions with custom messages

Description

From discussion #55, custom handled errors (in contrast to errors handled automatically by nest) should use exceptions with standardized messages. The standard messages are:

400 from bad url parameter type or format
- "Invalid data type or format for request parameter"
404 from specified resource id not existing
- Use the standard "Not Found" Message
500 from database failure
- "Error while retrieving data"

Context

As these are the first error messages, we will need to create place to hold them (and any future exceptions). This structure is outlined in the first half of the acceptance criteria. An example of this structure is on the experimental branch exp/repo. The custom errors are defined within an error folder. An example of using the error is shown on tax-lot service, line 25

nycplanning / ae-zoning-api Goto Github PK

ae-zoning-api's People

Contributors

Stargazers

Watchers

ae-zoning-api's Issues

Description

Acceptance criteria

Related

Description

Acceptance criteria

Related tickets

Relevant documentation

Description

Acceptance criteria

Description

Acceptance criteria

Description

Acceptance criteria

Related tickets

Description

Acceptance criteria

Related tickets

Acceptance Criteria

Description

Related Ticket

Acceptance criteria

Description

Acceptance criteria

Related tickets

Description

Acceptance criteria

Related issues

Description

Acceptance Criteria

Is blocked by

Description

Acceptance Criteria

Discussed in #87

Description

Acceptance criteria

Description

Acceptance critera

Related

Description

Acceptance criteria

Related tickets

Relevant documentation

Description

Related

Description

Acceptance criteria

Related tickets

Description

Description

Acceptance criteria

Description

Acceptance criteria

Related tickets

Description

Acceptance criteria

Description

Description

Description

Acceptance criteria

Related

Description

Acceptance Criteria

Description

Acceptance criteria

Related

Description

Acceptance criteria

Related tickets

Description

Acceptance criteria

Sample code

Description

Acceptance criteria

Related