Comments (6)
That's a good question. One thing you can do is model your application such that an event which updates a resource is only created after the resource is created.
You have the following 2 actions:
- Create resource A
- Update resource A
Say you create events for 1) and 2) simultaneously and send them for delivery, then there is a problem as you noticed because 2) can arrive before 1).
But you can always model your application so that an event for 2) is created only after 1) is created.
From an implementation point of view, you can create event 2) by having a "AFTER INSERT" trigger on resource A. Hope that makes sense.
from 3factor-example.
Another thing you can do is leverage the idempotency and retry-ability of your functions. This way even if something arrives in a unexpected order, it will fail gracefully and be retried till success.
from 3factor-example.
If you're relying on your application to make sure 1 always happens before 2, would you be better served by a synchronous process?
How do you handle conflict resolution for multiple updates arriving out of order, but are updating the same record? You could rely on timestamps, but you can't guarantee that clients have the same timestamps.
Obviously there are ways around all of this, but does it make it easier for the developer using this pattern? I'm not sure.
One option to fix this would be not allowing out of order events as much as possible. Using something like a FIFO queue would solve a fair bit of these problems.
from 3factor-example.
@ameech Hm...makes sense.
I think about this problem like this:
- Try to make sure that events are as independent as possible so that you can call serverless functions as events come without worrying about order
- If I can't ensure 1, I usually write the serverless function in a way that refetches latest state, checks if the state is as expected and then continues (perhaps in a DB transaction, if the state is fetched from a database that allows this). If state is unexpected, abort and return error with a retry-after backoff
- If 2 is also not possible, then put dependent events in a FIFO queue
Does that make sense? I usually manage to get away with 2 and this works because I think of each serverless function being "idempotent" and infinitely retry-able. It also keeps things simple from the point of view of writing the serverless function. I try not to worry about any external guarantees about how I expect things to be.
If this whole things seems synchronous, there could be 2 possibilities:
- Is this synchronous only for mutating database state? If yes, I can run all my business logic in a database transaction and I would move my code to the app-facing GraphQL API (or maybe even inside the database ;)).
- Does the logic seem synchronous, but is actually synchronous across external APIs or independent datastores where I can't do things transactionally? In this case, I would try to do things as mentioned in the 3 points above.
I'm not sure if I was able to convey my thoughts clearly. :)
LMK what you think!
from 3factor-example.
I think we're getting closer to how this should work, however I think 1 and 2 fall apart when you have multiples updates for the same record. How do you reconcile which one comes first? Seems to only be solvable by using something like a FIFO queue or some sort of log.
Also, a lot of these are based on the assumption of events being idempotent, but I find in real life that it's not possible to have that across the board. For example sending a message, you can't keep retrying that. You can get around it by using idempotency keys, but a lot of these issues just go away when you remove the "Out-of-order" best-practice.
from 3factor-example.
@ameech Yes, few cases can only be solved using a FIFO queue or some well ordered mechanism but do you think this is the common case?
Also, it is serverless best practice to not design application which are dependent on order. This is because when you hit a serverless endpoint, the delivery is not strictly once or ordered. 3factor requires you to be cognizant of this and hence take some extra effort, only when required, to make your serverless handlers as independent as possible.
For e.g. in the first case you mentioned (update before create), you should emit an update event only after create is successful. And in the second case (multiple updates), you should perhaps have something like a timestamp column which will void an update if it is later than the event timestamp.
from 3factor-example.
Related Issues (8)
- How to import Schema.sql with Hasura docker-compose HOT 11
- Fix analytics app to use one top level query per subscription HOT 4
- https://3factor-analytics-app.netlify.com/ is broken HOT 2
- Data fetching is broken HOT 3
- Preventing cloud event recursion HOT 4
- Update bootstrap and dependencies HOT 1
- Add routing in order-app HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from 3factor-example.