microsoft / abstrakt Goto Github PK
View Code? Open in Web Editor NEWBusiness oriented, config driven DevOps
License: MIT License
Business oriented, config driven DevOps
License: MIT License
Currently Abstrakt is only configured to pull charts from a directory. We need to learn how to download charts over https and implement it.
Describe the bug
If one of the unit tests fail, the pipeline does not fail and reports back everything is ok. Now you can view the unit test results and you will see that a test has failed. However, it appears by looking at the pipeline, everything succeeded.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The pipeline should fail so that PRs do not get merged into master
.
Screenshots
If you look at the build pipeline here you'll see everything is ok. However, if you look at the test results tab you will see a test has failed.
Name: WormholeSender_1
Id: 6209fed9-901f-4ad8-93eb-4e6ef11ad0a4
Type: WormholeSender
This make for hard to read relationships etc.
Name: Generated Link
Id: 73c51a4d-ff85-4e5a-aa68-3cd5380db499
Description: This relationship was generated by a tool
From: b7a9c21c-eb93-4a93-8433-b7198b300be4
To: 402cc1c8-b715-427c-af69-c1dd5a8651b0
This is messy and hard to undertand - the following would be better
Name: Generated Link
Id: 73c51a4d-ff85-4e5a-aa68-3cd5380db499
Description: This relationship was generated by a tool
From: WormholeSender_1
To: WormholeSender_2
Description
Once the build pipeline has publish a new build artifact we would like to trigger a release pipeline which takes the build artifact and creates a new release in GitHub.
Work has started on this on branch 160-git-releases and comments on this can be found here: AB#160
Acceptance Criteria
Find out how many minutes available to us.
Currently helm charts are generated without a namespace. It would be a useful feature to be able to deploy services to separate namespaces
Describe the bug
If you have two services with either the same Id
or Name
, abstrakt does not pick up on this and does not throw an error. This is especially troublesome when it comes to relationships and mapping between different services.
To Reproduce
Steps to reproduce the behavior:
constellation.yaml
and make sure to have two services with the same name. Use sample/constellation/http_constellation.yaml
to get you started.go run main.go visualise -f sample/constellation/http_constellation.yaml | dot -Tpng > result.png
and you see all your services mapped out.Expected behavior
Abstrakt should be smart and check for unique names and IDs before doing anything.
Screenshots
N/A.
Additional context
N/A.
Sample charts that can be used to demonstrate real composition of a constellation
A/C
Description
Go 1.14 is out and it's time to upgrade!
Acceptance Criteria
.devcontainer
is updated to use Go 1.14.0.go.mod
is updated to use Go 1.14.0.Looks like semantic-release
has a dependency on a package with a known security vulnerability. We need to update semantic-release
.
Describe the bug
Abstrakt exit 0s on error
To Reproduce
Steps to reproduce the behavior:
➜ /workspace git:(feature/testdata_golden_dataset) ✗ ./abstrakt compose -f sample/constellation/sample_constellation.yaml -m sample/constellation/sample_constellation_maps.yaml -o /tmp
event_hub_sample_event_generatorevent_hub_sample_event_hubevent_hub_sample_event_loggerevent_hub_sample_event_logger1Chart was saved to: /tmpThere was an error saving the chart: directory /tmp/deps/event_hub_sample_event_generator not found#
➜ /workspace git:(feature/testdata_golden_dataset) ✗ echo $?
0
➜ /workspace git:(feature/testdata_golden_dataset) ✗
Expected behavior
expect an non 0
exit code
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.
Code coverage should be reported in the default readme file along side the build status badge.
The visualize test doesn't execute the command. It only tests the internal methods. Unit tests should test the execution of the command.
Describe the bug
This makes linting locally slower
To Reproduce
Steps to reproduce the behavior:
run make lint-all more than once and observe the delay
Expected behavior
The tool will only be downloaded if it doesn't already exist locally
While doing the PR #42 for issue #26, I ran into a flaky test. This seems to be because of the way we're instantiating commands. We essentially create a command and assign it to a variable, then use that variable in the tests. This seems to make tests prone to side-effects. The following test passes when run on it's own, but fails when it's run as part of the package tests:
func TestComposeCmdVerifyRequiredFlags(t *testing.T) {
expected := "required flag(s) \"constellationFilePath\", \"mapsFilePath\", \"outputPath\" not set"
output, err := executeCommand(composeCmd, "")
if err != nil {
checkStringContains(t, err.Error(), expected)
} else {
t.Errorf("Expecting error: \n %v\nGot:\n %v\n", expected, output)
}
}
We could attempt to fix this by changing the way we create our commands to use a function that returns a new command, as shown here: https://github.com/gohugoio/hugo/blob/0efb00c2a86ec3f52000a643f26f54bb2a9dfbd6/commands/version.go#L28
Describe the bug
These tests will not work as they do not modify the t object, they just return.
https://github.com/microsoft/abstrakt/blob/master/internal/chartservice/chartservice_test.go#L29
The devcontainer is slow if you have to rebuild it as it re-downloads all the go and other objects each time. This fix will cache these assets where possible to mounted volumes.
As per this comment on databricks operator Azure/azure-databricks-operator#100 (review)
There's many places where fmt
is used to print to console. We should use logger.go
instead.
Create a basic first Helm Chart output. Use the maps and the DAG to produce the output chart.
A/C
Describe the bug
Currently equivalent helm tgz files can have different contents due to reordered entries, removal of yaml comments etc. The current solution is to check the size which will be changed in the manipulation of these files.
Description
Currently abstrakt's version is hardcoded in version.go
. If we're going to implement auto versioning referenced in #54 with semantic release then the version number returned by version.go
should reflect the current version in master.
Acceptance Criteria
abstrakt version
should return the current version generated by semantic release.version.go
and be easily updatable.Describe the bug
Golang linting fails for compose command.
To Reproduce
run make lint-all
Expected behavior
run make lint-all should complete successfully.
Create a real set of sample charts that can be used by the sample constellation to deploy to a Kind based Kube cluster in a .devcontainer.
A/C
The tests only executes the command, it doesn't test that the version was actually printed.
It doesn't make much sense to have both command
and cmd
folders since they both mean the same thing.
Either move cmd
to command
or command
to cmd
.
Describe the bug
If a service referenced in multiple relationships' To
and/or From
, compose
will not pick up on them and instead take the first relationship it encounters.
If you look here, Compose
loops over the services in the constellation but when it comes to the relationships, it only looks if the service is referenced and returns the first relationship instance it appears in.
To Reproduce
Steps to reproduce the behavior:
Edit sample/constellation/http_constellation.yaml
to look like the following:
Name: "HTTP Microservice Sample"
Id: "d6e4a5e9-696a-4626-ba7a-534d6ff450a5"
Services:
- Name: "Sender 1"
Id: "9e1bcb3d-ff58-41d4-8779-f71e7b8800f8"
Type: "WormholeSender"
Properties:
CHAIN_ARG: "1"
- Name: "Sender 2"
Id: "5d43a954-7214-4872-9d8f-758bb4238aee"
Type: "WormholeSender"
Properties:
CHAIN_ARG: "2"
- Name: "Receiver"
Id: "aa382e25-8662-4d11-bca9-d9a230c22487"
Type: "WormholeReceiver"
Properties: {}
Relationships:
- Name: "Sender to Sender Link"
Id: "211a55bd-5d92-446c-8be8-190f8f0e623e"
Description: "Link between the first and second senders"
From: "9e1bcb3d-ff58-41d4-8779-f71e7b8800f8"
To: "5d43a954-7214-4872-9d8f-758bb4238aee"
Properties: {}
- Name: "Sender to Receiver Link"
Id: "ff5444b5-35b7-4688-98ef-56c2587b314d"
Description: "Link between the second sender and receiver"
From: "5d43a954-7214-4872-9d8f-758bb4238aee"
To: "aa382e25-8662-4d11-bca9-d9a230c22487"
Properties: {}
- Name: "Sender to Receiver Link"
Id: "ff5444b5-35b7-4688-98ef-56c2587b314e"
Description: "Link between the second sender and receiver"
From: "9e1bcb3d-ff58-41d4-8779-f71e7b8800f8"
To: "aa382e25-8662-4d11-bca9-d9a230c22487"
Properties: {}
Put a breakpoint on this line.
Put this in your .vscode/launch.json
:
{
"name": "Launch",
"type": "go",
"request": "launch",
"mode": "auto",
"program":"${workspaceFolder}/main.go",
"env": {},
"dlvLoadConfig": {
"followPointers": true,
"maxVariableRecurse": 1,
"maxStringLen": 5000,
"maxArrayValues": 64,
"maxStructFields": -1
},
"args": ["compose", "-f", "./sample/constellation/http_constellation.yaml", "-m", "./sample/constellation/http_constellation_maps.yaml", "-o", "./output/http_sample", "-v"]
}
Run abstrakt in debug mode and notice how relationship ff5444b5-35b7-4688-98ef-56c2587b314e
is ignored.
Expected behavior
If a service if referenced in multiple relationships, then compose should be able to handle it.
Screenshots
N/A.
Additional context
N/A.
Describe the bug
For some reason the build pipeline is failing when running make lint-all
because for some reason the pipeline is trying to access kind and it's returning an error.
Script contents:
make lint-all
========================== Starting Command Output ===========================
/bin/bash --noprofile --norc /home/vsts/work/_temp/00f7308d-6ee8-4265-99a2-732e18f5788d.sh
No kind clusters found.
ERROR: unknown flag: --name
Installing golangci-lint
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b /home/vsts/work/1/s/../gopath/bin v1.23.1 > /dev/null 2>&1
Linting
golangci-lint run ./...
Vetting
go vet ./...
##[error]Bash wrote one or more lines to the standard error stream.
##[error]No kind clusters found.
##[error]ERROR: unknown flag: --name
To Reproduce
Steps to reproduce the behavior:
Expected behavior
make lint-all
shouldn't access kind at all.
Screenshots
N/A
Additional context
N/A
Describe the bug
Run abstrakt and examine the Chart.yaml
. the Name is Output
.
Many contributors have committed code to the project and it's about time we cleaned things up.
docs
folder in the root of project.
usage.md
sitting under cmd
.internal
.
buildmapservice
, chartservice
, composeservice
, and dagconfigservice
can be renamed to something a little nicer.LoadMapFromFile
can be renamed to LoadMapFile
.rbac.yaml
in the root of the project should be moved to samples
.Helpful video on how we can refactor the project: https://www.youtube.com/watch?v=cmkKxNN7cs4
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.