Code Monkey home page Code Monkey logo

always-right-temp-inc's Introduction

Always-Right-Temp-Inc

Anomaly Detector

Company: Always Right Temp Inc

Dev team:

Engineering Manager: Rafał Jaszczyński

Team lead: Rafał Jaszczyński

Senior dev: Rafał Jaszczyński

Senior dev: Rafał Jaszczyński

Senior dev: Rafał Jaszczyński

Senior QA dev: Rafał Jaszczyński

Product Owner: Rafał Jaszczyński

Functional Requirements

  • detect anomalies in temperature measurements
    • 2 algorithms
  • api to answer questions
  • scalable
  • high throughput

NonFunctional Requirements

  • 20k measurements per sec
  • kafka messages
  • REST api
  • algorithms choose option in properties file
  • postgres as persistent database for detected anomalies
  • redis to store recent measurements, because of easy and straightforward TTL mechanism (60 sec)
  • Command/Query separation
  • dead letter topic

Assumptions/Questions/Observations

No details on how sensors works, but I assume 1 sensor generate 1 measurement per second, this has implications for the application. We got roomId, so sensor could switch rooms.

Note for the Client: To avoid accidental anomaly detections sensor should shoot down when switching rooms.

Dev plan

Use CQRS separation:

  • microservice for receiving measurements, detecting anomalies and storing them in database
  • microservice for reading

There will probably be need scale heavily command microservice

Interacting with database and running the algorithm should take longer then just receiving and storing the message so there is another option, to add additional microservice:

  • microservice for receiving measurements and storing them without processing in database
  • microservice to process recent measurements and find anomalies with spring scheduling eg. each 1 second
  • microservice for reading

This also changes the behaviour: first version with 2 microservices is checking anomaly for each message and the second one is doing it on scheduled time based.

Because of simplicity and lack of knowledge about real world sensors I would choose option 1.

Frameworks/libraries to be used:

  • spring kafka/jpa/redis
  • spring webflux for REST api
  • testing: jupiter/testcontainers/spring-test
  • docker&compose to easily create containers of kafka, redis and postgres
  • lombok, mapstruct, liquibase

Todo

  • More test cases both unit and integration ones
  • Fix redis query (redis template)
  • Fix query with group by issue
  • fix anomaly analyzer lack of uuid support in entity
  • add support for list of alghoritms eg. anomaly.detector.strategy: average, limit (to check a range of temperatures eg. 0 to 40, so temp 45 C will )

  • Add both services to docker-compose
  • separate producer app for testing
  • and/or separate e2e test project

always-right-temp-inc's People

Contributors

rafaljdev avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.