Comments (4)
Thanks a lot, yes you are right the problem was with the unique id, I was testing on the same id which will never be like that!
Now everything works as expected.
from bus.
It does seem something's not quite right. Could you provide a copy of what your workflow looks like? I'm curious to know how you're seeing stale update errors as there's only a couple of ways this might occur.
Also is your postgres persistence shared amongst all pods?
from bus.
Yes both pods are identical they connect to the same postgress conneciton pool.
import { Workflow, StartedBy, Handles } from '@node-ts/bus-workflow'
import { injectable, inject } from 'inversify'
import { GitopsInstallWorkflowData } from './Data'
import { LOGGER_SYMBOLS, Logger } from '@node-ts/logger-core'
import { BUS_SYMBOLS, Bus } from '@node-ts/bus-core'
import {
GitopsInstallWorkflowStart,
GitopsInstallWorkflowHandleValidation,
GitopsInstallWorkflowValidationHandled,
GitopsInstallWorkflowHandlePreparation,
GitopsInstallWorkflowPreparationHandled,
GitopsInstallWorkflowMCLDUpdateHandled,
GitopsInstallWorkflowHandleMCLDUpdate,
GitopsInstallWorkflowHandleMCLDUpdateWatch,
GitopsInstallWorkflowMCLDUpdateWatchHandled,
GitopsInstallWorkflowStatus,
GitopsWorkflowStages,
} from '@commons/events-commands'
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms))
}
@injectable()
export class GitopsWorkflow extends Workflow<GitopsInstallWorkflowData> {
stagesCount: number = Object.keys(GitopsWorkflowStages).length / 2
workflowName = 'Atlantis Gitops Workflow'
constructor(
@inject(BUS_SYMBOLS.Bus) private readonly bus: Bus,
@inject(LOGGER_SYMBOLS.Logger) private readonly logger: Logger,
) {
super()
}
/**
* Starts new Gitops Installation to AWS Account workflow
*/
@StartedBy<GitopsInstallWorkflowStart, GitopsInstallWorkflowData, 'startGitopsWorkflow'>(GitopsInstallWorkflowStart)
async startGitopsWorkflow(event: GitopsInstallWorkflowStart): Promise<Partial<GitopsInstallWorkflowData>> {
const { username, awsAccount, awsAccountEmail } = event
const description = `Atlantis Gitops Workflow started by ${username} and will be isntalled to AWS account ${awsAccount}`
this.logger.info(description)
await this.bus.send(new GitopsInstallWorkflowHandleValidation(awsAccount))
await this.bus.publish(
new GitopsInstallWorkflowStatus(
this.workflowName,
awsAccount,
username,
GitopsWorkflowStages.started,
this.stagesCount - 1,
'Initial Start',
description,
),
)
return {
username,
awsAccount,
awsAccountEmail,
workflowValid: false,
accountVariables: {},
mcldGitopsPRUpdateUrl: '',
mcldGitopsPRMerged: false,
}
}
@Handles<
GitopsInstallWorkflowValidationHandled,
GitopsInstallWorkflowData,
'handleGitopsInstallWorkflowValidationResponse'
>(GitopsInstallWorkflowValidationHandled, (event) => event.awsAccount, 'awsAccount')
async handleGitopsInstallWorkflowValidationResponse(
eventData: GitopsInstallWorkflowValidationHandled,
{ awsAccount, username }: GitopsInstallWorkflowData,
): Promise<Partial<GitopsInstallWorkflowData>> {
const description = `Atlantis Gitops Workflow received validation response: ${awsAccount} validity is: ${eventData.workflowValid}`
this.logger.info(description)
if (!eventData.workflowValid) {
// TODO: send command to complete workflow if eventData.workflowValid = false
this.logger.info(`Atlantis Gitops Workflow handling false start`)
// this.bus.send(new SomeEventToStopWorkflow())
return this.discard()
}
await sleep(10000)
// If validation passed, send message to preparation handler and update workflow state data with validity
await this.bus.send(new GitopsInstallWorkflowHandlePreparation(awsAccount))
await this.bus.publish(
new GitopsInstallWorkflowStatus(
this.workflowName,
awsAccount,
username,
GitopsWorkflowStages.validated,
this.stagesCount - 2,
'Account Validation',
description,
),
)
return { workflowValid: eventData.workflowValid }
}
@Handles<
GitopsInstallWorkflowPreparationHandled,
GitopsInstallWorkflowData,
'handleGitopsInstallWorkflowPreparationResponse'
>(GitopsInstallWorkflowPreparationHandled, (event) => event.awsAccount, 'awsAccount')
async handleGitopsInstallWorkflowPreparationResponse(
eventData: GitopsInstallWorkflowPreparationHandled,
{ awsAccount, username }: GitopsInstallWorkflowData,
): Promise<Partial<GitopsInstallWorkflowData>> {
const description = `Atlantis Gitops Workflow received preparation response for ${awsAccount}`
this.logger.info(description)
if (!eventData.accountVariables) {
// TODO: send command to complete workflow if eventData.accountVariables = null
this.logger.info(`Atlantis Gitops Workflow handling false start`)
// this.bus.send(new SomeEventToStopWorkflow())
return this.discard()
}
await sleep(10000)
// If preparation passed, send message to update MCLD handler and update workflow state data with validity
await this.bus.send(new GitopsInstallWorkflowHandleMCLDUpdate(awsAccount, eventData.accountVariables))
await this.bus.publish(
new GitopsInstallWorkflowStatus(
this.workflowName,
awsAccount,
username,
GitopsWorkflowStages.prepared,
this.stagesCount - 3,
'Account Preconfiguration',
description,
),
)
return {
accountVariables: eventData.accountVariables,
}
}
@Handles<
GitopsInstallWorkflowMCLDUpdateHandled,
GitopsInstallWorkflowData,
'handleGitopsInstallWorkflowMCLDUpdateResponse'
>(GitopsInstallWorkflowMCLDUpdateHandled, (event) => event.awsAccount, 'awsAccount')
async handleGitopsInstallWorkflowMCLDUpdateResponse(
eventData: GitopsInstallWorkflowMCLDUpdateHandled,
{ awsAccount, username }: GitopsInstallWorkflowData,
): Promise<Partial<GitopsInstallWorkflowData>> {
const description = `Atlantis Gitops Workflow received update MCLD response for ${awsAccount}`
this.logger.info(description)
if (!eventData.mcldGitopsPRUpdateUrl) {
// TODO: send command to complete workflow if eventData.accountVariables = null
this.logger.info(`Atlantis Gitops Workflow handling false start`)
// this.bus.send(new SomeEventToStopWorkflow())
return this.discard()
}
await sleep(10000)
// If mcld update passed, send message to update MCLD watch handler and update workflow state data with validity
await this.bus.send(new GitopsInstallWorkflowHandleMCLDUpdateWatch(awsAccount, eventData.mcldGitopsPRUpdateUrl))
await this.bus.publish(
new GitopsInstallWorkflowStatus(
this.workflowName,
awsAccount,
username,
GitopsWorkflowStages.iacUpdated,
this.stagesCount - 4,
'MCLD Infrastructure As Code',
description,
),
)
return {
mcldGitopsPRUpdateUrl: eventData.mcldGitopsPRUpdateUrl,
}
}
@Handles<
GitopsInstallWorkflowMCLDUpdateWatchHandled,
GitopsInstallWorkflowData,
'handleGitopsInstallWorkflowMCLDWatchResponse'
>(GitopsInstallWorkflowMCLDUpdateWatchHandled, (event) => event.awsAccount, 'awsAccount')
async handleGitopsInstallWorkflowMCLDWatchResponse(
eventData: GitopsInstallWorkflowMCLDUpdateWatchHandled,
{ awsAccount, username }: GitopsInstallWorkflowData,
): Promise<Partial<GitopsInstallWorkflowData>> {
const description = `Atlantis Gitops Workflow received MCLD watch update response for ${eventData.awsAccount}`
this.logger.info(description)
if (!eventData.mcldGitopsPRMerged) {
// TODO: send command to complete workflow if eventData.accountVariables = null
this.logger.info(`Atlantis Gitops Workflow handling false start`)
// this.bus.send(new SomeEventToStopWorkflow())
return this.discard()
}
this.logger.info(`Atlantis Gitops Workflow was completed. Initiated by: ${username}, Aws Account: ${awsAccount} `)
await sleep(10000)
await this.bus.publish(
new GitopsInstallWorkflowStatus(
this.workflowName,
awsAccount,
username,
GitopsWorkflowStages.completed,
this.stagesCount - 5,
'Infrastructure Changes landed',
description,
),
)
return this.complete({ mcldGitopsPRMerged: eventData.mcldGitopsPRMerged })
}
}
from bus.
Cool, I had a look through it and have a couple of thoughts:
Would the sleep()
calls inside the handlers be the cause of any of the stale data errors? Whilst a handler is sleeping, any other message that is handled by the workflow in the meantime could cause the underlying data to change and throw this error when the handler finally resumes and returns.
Another thought is how an event is mapped to a workflow:
(GitopsInstallWorkflowPreparationHandled, (event) => event.awsAccount, 'awsAccount')
For this to work without collisions, the assumption is that only one workflow instance is ever running for a single awsAccount
. If this is not the case (ie: you can have multiple concurrent gitops workflows per aws account), i'd suggest mapping based on a workflow-specific correlation id in the message attributes.
Interested to know your thoughts
from bus.
Related Issues (20)
- Construct workflows using container during mapping
- Port middleware over to master HOT 1
- RabbitMQ queue and exchange setup HOT 8
- The memory-queue is inappropriately deleting messages HOT 1
- Broken links in docs HOT 1
- Delay a retry by X seconds HOT 1
- Vulnerability in url-parse - dependency of amqplib HOT 1
- Rabbitmq transport polling HOT 2
- IoC with loopback HOT 4
- Named export <nameExport> not found HOT 2
- [bus-class-serializer]: package.json missing attributes HOT 1
- [Bug] Rabbitmq-transport.ts does not have reconnection if channel gets disconnected by some reasons
- How to setup DI container, when workflow needs the bus instance HOT 2
- Open to update `bus-sqs` to aws-sdk v3? HOT 1
- Support Mognodb Persistence for Workflows
- RabbitMQ transport doesn't allow custom message headers. HOT 4
- Problem with lodash and persisting workflow state
- Stuck workflows with error: "No existing workflow state found for message. Ignoring"
- [Feat] Support passing message and message context to Container Adapter HOT 1
- Persistent messages are not supported. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bus.