dbyington / csv-crm-upload Goto Github PK

The CSV-CRM uploader needs a Postgresql database to save customer data that is to be sent to the CRM Service.

Acceptance Criteria

• Able to bring up a docker container to start a Postgres DB
• The docker container will be initialized with the basic database needed to run the uploader service

The CSV streamer needs a method to signal the CRM integrator when it has successfully inserted new customer data into the database.

AC:

• RPC Server that provides a new data signal method
• RPC Client to call the new data signal method

To send customer data to the CRM Service the data needs to be read from CSV files.

AC:

• read CSV records from file
• read header row from CSV file
• read records in such a way that the whole file does not need to be loaded into memory

The service needs to be able to insert, query, and update data in the database.

AC:

• Provide an INSERT method for inserting new rows
• Provide a SELECT method for retrieving rows now sent to the CRM Service
• Provide an UPDATE method to update rows successfully sent to the CRM Service

Once inserted into the database, customer data will need to be sent to the CRM Service, once.

AC:

• Accept signal to get newly inserted customer data
• POST customer data to the CRM Service
• Update customer data when successfully sent to CRM Service
• Querying for customer data should not be more frequent that every (configurable) milliseconds
• Querying for customer data maximum frequency should be 1 second and increase from last empty result set

To complete the data pipeline a mock CRM Service REST API will need to accept and respond to POST requests for new data.

AC:

• Accept POST with a JSON body

Optional:

• Respond on a random interval
• Randomly return a 500, after a minimum response time threshold
• Respond with a 400 if the data has already been sent