stamp-project / camp Goto Github PK
View Code? Open in Web Editor NEWAmplify your Configuration tests!
Home Page: https://stamp-project.github.io/camp
License: MIT License
Amplify your Configuration tests!
Home Page: https://stamp-project.github.io/camp
License: MIT License
Problem executing camp execute.
follow the tutorial at this link:
https://stamp-project.github.io/camp/pages/execute.html
docker run -it -v $(pwd):/camp/workspace fchauvel/camp:v0.2.3 camp execute -c simple/config.ini
the error is:
CAMP v0.2.3 (MIT)
Copyright (C) 2017, 2018 SINTEF Digital
Traceback (most recent call last):
File "/usr/local/bin/camp", line 9, in
load_entry_point('camp==0.2.3', 'console_scripts', 'camp')()
File "/usr/local/lib/python2.7/dist-packages/camp/run.py", line 59, in main
runner.start(argv[1:])
File "/usr/local/lib/python2.7/dist-packages/camp/run.py", line 34, in start
command.send_to(self._camp)
File "/usr/local/lib/python2.7/dist-packages/camp/commands.py", line 183, in send_to
camp.execute(self)
TypeError: execute() takes exactly 2 arguments (1 given)
Preview to follow the link or open file .github/ISSUE_DOC.md
Can't configure CAMP in oder to run Citygo. I generated the neccesary files following Github readme.
services:
db:
imgfeature: [ubuntu]
web:
imgfeature: [python]
mongo:
imgfeature: [mongo]
orion:
imgfeature: [orion]
cygnus-common:
imgfeature [cygnus-common]
cygnus-ngsi:
imgfeature [cygnus-ngsi]
constraints:
- services['web'].alive()
- varconst(services['db'], 'ThreadPerChild', iv1, varconst(services['db'], 'ThreadLimit', iv2, iv1.value < iv2.value))
outputs:
big-thread-perchild:
- varconst(services['db'], 'ThreadPerChild', iv1, iv1.value >= 100)
small-thread-limit:
- varconst(services['db'], 'ThreadLimit', iv1, iv1.value <= 64)
cat features.yml
ubuntu:
python:
orion:
mongo:
db:
cygnus-common:
cygnus-ngsi:
web:
downloadimages:
ubuntu:
features: [ubuntu]
name: ubuntu
tag: latest
python:
features: [python]
name: python
tag: 2.7
orion:
features: [orion]
name: fiware/orion
tag: latest
mongo:
features: [mongo]
name: mongo
tag: 3.2
cygnus-common:
features: [cygnus-common]
name: fiware/cygnus-common
tag: latest
cygnus-ngsi:
features: [cygnus-ngis]
name: fiware/cygnus-common
tag: latest
buildingrules:
db:
requires: [ubuntu]
adds: [ubuntu]
svar: [ThreadLimit, ThreadPerChild]
mongo:
adds: [mongo]
orion:
adds: [orion]
depends: [mongo]
web:
requires: [python]
adds: [python]
depends: [db,orion]
cygnus-commom:
adds: [cygnus-common]
cygnus-ngsi:
adds: [cygnus-ngsi]
maxfreebuild: 10
constraints:
- BuildImage.forall(bi1, Not(And(bi1.features.contains(features['db']), bi1.features.contains(features['web']))))
ThreadLimit:
ThreadLimit64:
type: Int
value: 64
operations:
- substituion1:
engine: regexp
filename: "docker-compose.yml"
placement: "ThreadLimit=64"
replacement: "ThreadLimit=64"
ThreadLimit32:
type: Int
value: 128
operations:
- substituion1:
engine: regexp
filename: "docker-compose.yml"
placement: "ThreadLimit=64"
replacement: "ThreadLimit=128"
ThreadPerChild:
ThreadPerChildFree:
type: Int
operations:
- substituion1:
engine: regexp
filename: "docker-compose.yml"
placement: "ThreadsPerChild=25"
replacement: "ThreadsPerChild=${value}"
ThreadLimit:
ThreadLimit64:
type: Int
value: 64
operations:
- substituion1:
engine: regexp
filename: "docker-compose.yml"
placement: "ThreadLimit=64"
replacement: "ThreadLimit=64"
ThreadLimit32:
type: Int
value: 32
operations:
- substituion1:
engine: regexp
filename: "docker-compose.yml"
placement: "ThreadLimit=64"
replacement: "ThreadLimit=32"
ThreadPerChild:
ThreadPerChildFree:
type: Int
operations:
- substituion1:
engine: regexp
filename: "docker-compose.yml"
placement: "ThreadsPerChild=25"
replacement: "ThreadsPerChild=${value}"
root@osboxes:/home/osboxes/Proyectos/STAMP/WP2_ATOS_USE_CASE/CAMP/camp-samples/citygo# docker run -it -v $(pwd):/root/workingdir songhui/camp /bin/bash allinone.sh
Working directory is ./workingdir
Start searching for images
Traceback (most recent call last):
File "ozepy/stamp/dockerall-var.py", line 474, in <module>
main(sys.argv[1:])
File "ozepy/stamp/dockerall-var.py", line 470, in main
generate(workingdir)
File "ozepy/stamp/dockerall-var.py", line 310, in generate
img = DefineObject(name, DownloadImage)
File "/root/ozepy/src/model.py", line 736, in DefineObject
_consolas_assert(not (name in _all_objects), 'Object name "%s" is already used' % name)
File "/root/ozepy/src/model.py", line 18, in _consolas_assert
raise ConsolasException(msg)
src.model.ConsolasException: Object name "orion" is already used
cp: cannot stat './workingdir/out/ampimages.yml': No such file or directory
cp: cannot stat './workingdir/out/genimages.yml': No such file or directory
cp: cannot stat './workingdir/out/ampcompose.yml': No such file or directory
Input file is ./workingdir/genimages.yml
Working directory is ./workingdir
Traceback (most recent call last):
File "camp/src/dockerfilegen.py", line 87, in <module>
main(sys.argv[1:])
File "camp/src/dockerfilegen.py", line 82, in main
generate(inputfile, workingdir)
File "camp/src/dockerfilegen.py", line 31, in generate
with open(file, 'r') as stream:
IOError: [Errno 2] No such file or directory: './workingdir/genimages.yml'
Input file is ./workingdir/ampcompose.yml
Working directory is ./workingdir
Traceback (most recent call last):
File "camp/src/composegen.py", line 216, in <module>
main(sys.argv[1:])
File "camp/src/composegen.py", line 211, in main
generate(seedfile, workingdir, inputfile)
File "camp/src/composegen.py", line 121, in generate
with open(amp_result_file, 'r') as stream:
IOError: [Errno 2] No such file or directory: './workingdir/ampcompose.yml'
===========Searching finished.=========
Build all the generated images by: cd <your_dir>/build && bash ./build.sh
Launch one of the configurations by: docker-compose <your_dir>/docker-compose/docker-comopose-<number>.yml up
Preview to follow the link or open file .github/ISSUE_DOC.md
During Stockholm meeting, need for supporting performance test emerged (ATOS UC). Apache JMeter has been identified has the target performance test system, so now we need to add the support for JMeter to CAMP execute feature.
Problem following the Lutece use-case guide.
camp generate -d . --all
give me errors on camp.yml (tabs and wrong type:version)
Preview to follow the link or open file .github/ISSUE_DOC.md
It would be nice to be able to run the solver and the config generation part of CAMP as a library that can be integrated in existing code.
For example for XWiki I'm implementing #12 and we could imagine generating various configurations using the TestContainers approach (see https://www.testcontainers.org/).
Current CAMP tool requires a number of related input models for describing the configuration amplification application domain, including features.yml, images.yml, genimages.yml, composite.yml, variables.yml, etc.
Managing all these models, and providing new instances for different application domains could be complex due to the fragmentation of the CAMP input model across all these models.
We recommend to group all these modeling aspects (features, images, variables, etc) within different subsection of a common CAMP input model (i.e. camp.yml)
N/A
Preview to follow the link or open file .github/ISSUE_DOC.md
In my opinion, it could be better for CAMP users and developers to have the next points
- Camp Samples
- Set up your example : Brief explanation about files involved
- How to run your new configuration
Best,
Fernando
We need to integrate code bases of https://github.com/STAMP-project/tecor and https://github.com/STAMP-project/camp-realize
On XWiki we have the following use cases related to configuration testing:
This means being able to run various configurations on developer's machines and to be able to debug from developer's IDEs, and start/stop the whole environment from the JUnit tests.
FTR this is what I'm currently developing using TestContainers (https://www.testcontainers.org/).
I'm currently documenting the experiments I've done and will be posting a link in this issue soon.
In camp/samples/stamp/atos/out/config_1/images/build_images.sh
, I had to update the script so that it looks like this (basically remove the -f
option and instead directly use the folder containing the Dockerfile as argument for the docker build
):
docker build -t camp-postgres_0 ./postgres_0
docker build -t camp-showcase_0 ./showcase_0
echo 'All images ready'
See ATOS use case
After successfully installing all CAMP prerequisites, I launched CAMP installation with pip, but pip doesn't install anything, showing only the message Requirement already satisfied: pip from https://github.com/stamp-project/camp.git@master#egg=pip in /usr/local/lib/python2.7/dist-packages (19.0.3)
Execute as root
root@vmi2:~#
pip install https://github.com/stamp-project/camp.git@master#egg=pip
as mentioned in CAMP documentation
In the purescript sample, I am trying to apply CAMP to make some changes in package.yaml
. This file looks like this:
name: purescript
...
dependencies:
- aeson >=1.0 && <1.5
- aeson-better-errors >=0.8
...
...
stability: experimental
If the generated configuration, this file looks like this:
name: purescript
...
dependencies:
- aeson == 1.0.0.0
- aeson-better-errors >=0.8
...
...
stability: experimental
tal
al
al
al
l
al
l
I might do something wrong in my camp file.... or something fishy is happening in CAMP when making changes in this file....
Of course, the problem is that this package.yaml
is thus invalid. Also, different package.yaml
in different configurations seem to have different problems (though quite similar).
I can surely find a workaround with sed
to remove everything after stability: experimental
, so this issue is not completely urgent.
We have this support for docker images, need docker compose
Please refer to this bug #1
Issue Type: improvement
Reproducibility: always
Severity: minor
Tool/Service/Component: CAMP v0.2.2
Execution Environment: Debian 9.5
Reporter: Franck Chauvel (on behalf of Assad Montasser)
CAMP throws an exception when the CAMP model can't be found. A user-friendly message would be more satisfactory.
I pulled the docker image and I had this error:
$ sudo docker run -it -v $(pwd):/camp/workspace fchauvel/camp:latest camp generate -d workspace
[sudo] password for assadm:
Unable to find image 'fchauvel/camp:latest' locally
latest: Pulling from fchauvel/camp
a92a4af0fb9c: Pull complete
e6259e154a86: Pull complete
d5986d2ded11: Pull complete
7af9db4c0f03: Pull complete
0397f5f63bfa: Pull complete
548a851ed8e2: Pull complete
3e38740b726d: Pull complete
Digest:
sha256:a0f15ca9c1c8b5bdbc2b06f43125055be104528e8f76abcfcfd0ba6e31aa73c6
Status: Downloaded newer image for fchauvel/camp:latest
CAMP v0.2.2 (MIT)
Copyright (C) 2017, 2018 SINTEF Digital
Traceback (most recent call last):
File "/usr/local/bin/camp", line 9, in <module>
load_entry_point('camp==0.2.2', 'console_scripts', 'camp')()
File "/usr/local/lib/python2.7/dist-packages/camp/run.py", line 59,
in main
runner.start(argv[1:])
File "/usr/local/lib/python2.7/dist-packages/camp/run.py", line 36,
in start
command.send_to(self._camp)
File "/usr/local/lib/python2.7/dist-packages/camp/commands.py", line
126, in send_to
camp.generate(self)
File "/usr/local/lib/python2.7/dist-packages/camp/core.py", line 33,
in generate
model = self._load_model()
File "/usr/local/lib/python2.7/dist-packages/camp/core.py", line 47,
in _load_model
file_name, model, warnings = self._input.model
File "/usr/local/lib/python2.7/dist-packages/camp/directories.py",
line 47, in model
file_name = self._find_model()
File "/usr/local/lib/python2.7/dist-packages/camp/directories.py",
line 59, in _find_model
raise ValueError("Unable to find the CAMP model")
ValueError: Unable to find the CAMP model
Preview to follow the link or open file .github/ISSUE_DOC.md
Using hyphens in component names of camp.yml (e.g., mysql-db) is not supported. The error message is not clear and it might be painful for developers (including me) to figure out that a hyphen is the root cause.
Thanks.
Preview to follow the link or open file .github/ISSUE_DOC.md
'variable' should be called 'variationpoint', 'value' is actually 'variation' or 'option'. This makes more sense.
In order to ease the integration of CAMP within CI/CD pipelines a starting point is to have it executing as a Maven goal.
The contents of the input files are not validated. If some mandatory field is missing, you typically get an error message about an internal datastructure that is invalid. Guessing what is actually wrong is tricky.
Any assumption made by the tool about the input files, should be validated and an appropriate error message should be given if those assumption are broken.
As an example, run the tool with no mandatoryfeature in images.yml
I tried to update CAMP installation within STAMP CI/CD server (vmi2.stamp-project.eu), using pip to update it, but I got following error:
Complete output from command python setup.py egg_info:
error in camp setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-zGl9AM/
The worst thing is that now camp is no longer working (and the Lutece Jenkins pipeline doesn't work anymore). If I type:
camp -h
I get following error:
Traceback (most recent call last):
File "/usr/local/bin/camp", line 6, in <module>
from camp.run import main
ImportError: No module named camp.run
As root user:
cd camp
git pull
pip install -e .
I tried also, after removing camp folder, to update it launching following command:
pip install -U git+https://github.com/STAMP-project/camp.git@master#egg=camp
The output is quite similar:
Collecting camp from git+https://github.com/STAMP-project/camp.git@master#egg=camp
Cloning https://github.com/STAMP-project/camp.git (to revision master) to /tmp/pip-install-kG_3I6/camp
Complete output from command python setup.py egg_info:
error in camp setup command: 'install_requires' must be a string or list of strings containing valid project/version requirement specifiers
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-kG_3I6/camp/
Preview to follow the link or open file .github/ISSUE_DOC.md
CAMP users need to know how to upgrade or uninstall CAMP. This can be accomplished writing a new section in CAMP installation page which describes all needed steps.
I get this error:
$ docker build -t fchauvel/camp .
Sending build context to Docker daemon 442.2MB
Step 1/10 : FROM debian:9-slim
9-slim: Pulling from library/debian
27833a3ba0a5: Already exists
Digest: sha256:bade11bf1835c9f09b011b5b1cf9f7428328416410b238d2f937966ea820be74
Status: Downloaded newer image for debian:9-slim
---> c08899734c03
Step 2/10 : ARG DEBUG
---> Running in 93405d8c3ac2
---> 42548c9587e2
Step 3/10 : ARG PYTHON_VERSION=3.5
---> Running in 10004cf95709
---> 57c12c6c6d4d
Step 4/10 : ARG WITH_TESTS
---> Running in cd29dc05a890
---> 38235291a18d
Step 5/10 : ARG Z3_VERSION=4.8.4.d6df51951f4c
---> Running in bae80552bffd
---> 29f18f1109b3
Step 6/10 : ARG Z3_PLATFORM=x64-debian-8.11
---> Running in 1a8ba0edbd48
---> 48d7b4f18825
Step 7/10 : LABEL maintainer "[email protected]"
---> Running in ecb02bbade0c
---> 8c045bc2c846
Step 8/10 : WORKDIR /camp
---> b9f7155683f5
Step 9/10 : COPY . /camp
---> c2ad9e61eb13
Step 10/10 : RUN bash install.sh --install-python --python-version ${PYTHON_VERSION} --install-z3 --z3-version ${Z3_VERSION} --z3-platform ${Z3_PLATFORM} --install-docker --camp-from-sources ${WITH_TESTS} ${DEBUG}
---> Running in b3395602c1ed
install.sh: line 2: $'\r': command not found
: invalid option nameet: errexit
: invalid option nameet: errtrace
: invalid option nameet: pipefail
: invalid option nameet: nounset
install.sh: line 7: $'\r': command not found
install.sh: line 10: $'\r': command not found
install.sh: line 12: $'\r': command not found
install.sh: line 23: $'\r': command not found
install.sh: line 24: $'\r': command not found
install.sh: line 52: $'\r': command not found
install.sh: line 53: $'\r': command not found
install.sh: line 54: $'\r': command not found
install.sh: line 55: $'\r': command not found
install.sh: line 56: syntax error near unexpected token `$'{\r''
'nstall.sh: line 56: `abort () {
The command '/bin/sh -c bash install.sh --install-python --python-version ${PYTHON_VERSION} --install-z3 --z3-version ${Z3_VERSION} --z3-platform ${Z3_PLATFORM} --install-docker --camp-from-sources ${WITH_TESTS} ${DEBUG}' returned a non-zero code: 2
Error: A shell command failed (code: 1)
docker run --name camp \ -it \ -v /var/run/docker.sock:/var/run/docker.sock \ -t fchauvel/camp:latest bash
cd samples/java
camp generate -d .
camp realize -d .
camp execute -d . -c tests -t maven
And this is the content of camp_execute.log:
camp@bash:./out/config_3/images$ bash build_images.sh
All images ready.
camp@bash:./out/config_3$ docker-compose up -d
config_3_greetings_1 is up-to-date
Starting config_3_tests_1 ... done
camp@bash:./out/config_3$ docker-compose run tests mvn test -B
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.083 s
[INFO] Finished at: 2019-04-01T12:38:51+00:00
[INFO] Final Memory: 8M/275M
[INFO] ------------------------------------------------------------------------
[ERROR] The goal you specified requires a project to execute but there is no POM in this directory (/tests). Please verify you invoked Ma
ven from the correct directory. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MissingProjectException
Preview to follow the link or open file .github/ISSUE_DOC.md
Please refer to STAMP-project/camp-samples#1 for the initial discussions
wrote by @fermenreq
Hi Anatoly,
Thanks! It works !
The above sources and directories has been generated and my question is how to deploy CityGo with the new configurations ?
root@osboxes:/home/osboxes/Proyectos/STAMP/WP2_ATOS_USE_CASE/CAMP/camp-samples/atos# ls
ampcompose.yml build compose2 docker-compose genimages.yml out resolmodel.yml variables.yml~
ampimages.yml compose1 composite.yml features.yml images.yml repo variables.yml
It could be better if we have some steps in the Readme
Bests,
Fernando
CAMP does not warn about substitition for which no match is performed. CAMP silently generate consistently generate configuration without realizing the variable.
Create a CAMP modle where one component declares a variable, whose realization is broken. For instance, in the example below, the issue occurs when the DockerFile does not contain the selected pattern python:3.7-slim-stretch
.
variables:
python:
values: [ v2.7, v3.5, v3.7 ]
realization:
- targets: [ greetings/Dockerfile ]
pattern: "python:3.7-slim-stretch"
replacements:
- python:2.7-slim-stretch
- python:3.5-slim-stretch
- python:3.7-slim-stretch
It would be good to be able to estimate the number of configurations before actually generating them all... so that we know :-)
Preview to follow the link or open file .github/ISSUE_DOC.md
The following error ocurrs when running camp execute
CAMP v0.3.0 (MIT)
Copyright (C) 2017 -- 2019 SINTEF Digital
Loaded 'workspace/camp.yaml'.
Loading configurations from 'workspace/out' ...
Test execution aborted!
That's all folks!
docker run -it -v $(pwd):/camp/workspace fchauvel/camp:v0.3.0 camp generate -d workspace -a
docker run -it -v $(pwd):/camp/workspace fchauvel/camp:v0.3.0 camp realize -d workspace
docker run -it -v $(pwd):/camp/workspace fchauvel/camp:v0.3.0 camp execute -d workspace -c tests -t maven
Preview to follow the link or open file .github/ISSUE_DOC.md
The CAMP tool is actually a series of scripts that are run as one. If one of the first scripts fail for some reason, execution keep going, resulting in misleading and confusing error messages. The start.sh script should check the result of each step of the execution, and bail out as soon as a step fails to complete
Run tool with an invalid images.yml file, first step will fail but other steps will still be executed.
On the atom sample, I am doing substitution in a package.json file
This original file is not empty. However, when calling camp realize
(after camp generate
), camp complains that it cannot do substitution:
CAMP v0.3.9 (MIT)
Copyright (C) 2017 -- 2019 SINTEF Digital
Loaded './camp.yaml'.
Loading configurations from './out' ...
Error:
Vain substitution: No match found!'
- pattern: '"resolve": "^1.1.6"'
- target file: './out/config_1/images/atom_0/src/package.json'
Is the pattern approptiate? What about the file content?
That's all folks!
The thing is that the file at ./out/config_1/images/atom_0/src/package.json
appears to be empty, hence the vain substitution... Note that all the other files around the package.json file are property copied in the out folder (with content).
No such file or directory error.
The error occurs when following the CAMP installation guide. In the second step, when running the command cd samples/java
Seems that the 'java' directory does not exist.
Preview to follow the link or open file .github/ISSUE_DOC.md
Dear @vassik,
The name of some vars in my docker-compose.yml
file are changed, cause Im working on a different apache module, i.e mpm-event.conf
.
Attached a piece of my docker-compose.yml
file, with the new vars.
environment:
- StartServers=2
- MinSpareThreads=25
- MaxSpareThreads=75
- ThreadLimit=64
- ThreadsPerChild=25
- MaxRequestWorkers=150
- MaxConnectionsPerChild=0
As I see in composite.yml
,images.yml
and variables.yml
, I changed : ThreadPerChild
to ThreadsPerChild
but it gave me an error
Working directory is ./workingdir
Start searching for images
Traceback (most recent call last):
File "ozepy/stamp/dockerall-var.py", line 474, in <module>
main(sys.argv[1:])
File "ozepy/stamp/dockerall-var.py", line 470, in main
generate(workingdir)
File "ozepy/stamp/dockerall-var.py", line 321, in generate
img.force_value('svar', resolve_variables(value.get('svar', [])))
File "ozepy/stamp/dockerall-var.py", line 267, in resolve_variables
return [variables[n] for n in variablenames]
KeyError: 'ThreadsPerChild'
cp: cannot stat './workingdir/out/ampimages.yml': No such file or directory
cp: cannot stat './workingdir/out/genimages.yml': No such file or directory
cp: cannot stat './workingdir/out/ampcompose.yml': No such file or directory
Input file is ./workingdir/genimages.yml
Working directory is ./workingdir
Traceback (most recent call last):
File "camp/src/dockerfilegen.py", line 87, in <module>
main(sys.argv[1:])
File "camp/src/dockerfilegen.py", line 82, in main
generate(inputfile, workingdir)
File "camp/src/dockerfilegen.py", line 31, in generate
with open(file, 'r') as stream:
IOError: [Errno 2] No such file or directory: './workingdir/genimages.yml'
Input file is ./workingdir/ampcompose.yml
Working directory is ./workingdir
Traceback (most recent call last):
File "camp/src/composegen.py", line 216, in <module>
main(sys.argv[1:])
File "camp/src/composegen.py", line 211, in main
generate(seedfile, workingdir, inputfile)
File "camp/src/composegen.py", line 120, in generate
reference = yaml.load(stream)
File "/usr/local/lib/python2.7/dist-packages/yaml/__init__.py", line 71, in load
return loader.get_single_data()
File "/usr/local/lib/python2.7/dist-packages/yaml/constructor.py", line 37, in get_single_data
node = self.get_single_node()
File "/usr/local/lib/python2.7/dist-packages/yaml/composer.py", line 36, in get_single_node
document = self.compose_document()
File "/usr/local/lib/python2.7/dist-packages/yaml/composer.py", line 55, in compose_document
node = self.compose_node(None, None)
File "/usr/local/lib/python2.7/dist-packages/yaml/composer.py", line 84, in compose_node
node = self.compose_mapping_node(anchor)
File "/usr/local/lib/python2.7/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
item_value = self.compose_node(node, item_key)
File "/usr/local/lib/python2.7/dist-packages/yaml/composer.py", line 84, in compose_node
node = self.compose_mapping_node(anchor)
File "/usr/local/lib/python2.7/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
item_value = self.compose_node(node, item_key)
File "/usr/local/lib/python2.7/dist-packages/yaml/composer.py", line 84, in compose_node
node = self.compose_mapping_node(anchor)
File "/usr/local/lib/python2.7/dist-packages/yaml/composer.py", line 133, in compose_mapping_node
item_value = self.compose_node(node, item_key)
File "/usr/local/lib/python2.7/dist-packages/yaml/composer.py", line 82, in compose_node
node = self.compose_sequence_node(anchor)
File "/usr/local/lib/python2.7/dist-packages/yaml/composer.py", line 110, in compose_sequence_node
while not self.check_event(SequenceEndEvent):
File "/usr/local/lib/python2.7/dist-packages/yaml/parser.py", line 98, in check_event
self.current_event = self.state()
File "/usr/local/lib/python2.7/dist-packages/yaml/parser.py", line 384, in parse_block_sequence_entry
if not self.check_token(BlockEntryToken, BlockEndToken):
File "/usr/local/lib/python2.7/dist-packages/yaml/scanner.py", line 116, in check_token
self.fetch_more_tokens()
File "/usr/local/lib/python2.7/dist-packages/yaml/scanner.py", line 257, in fetch_more_tokens
% ch.encode('utf-8'), self.get_mark())
yaml.scanner.ScannerError: while scanning for the next token
found character '\t' that cannot start any token
in "./workingdir/docker-compose/docker-compose.yml", line 27, column 8
===========Searching finished.=========
Build all the generated images by: cd <your_dir>/build && bash ./build.sh
Launch one of the configurations by: docker-compose <your_dir>/docker-compose/docker-comopose-<number>.yml up
Could you please give me details about how to change it ?
Thanks,
Fernando
CAMP execute triggers commands that fails when it is run from a container, which uses its host's docker deamon to spawn sibling containers.
$ docker run -v /var/run/docker.sock:/var/run/docker.sock camp_docker bash
root@fbf49c1dc546:/camp/samples/java# camp execute -d . -c tests -t maven
camp execute -d . -c tests -t maven
CAMP v0.3.0 (MIT)
Copyright (C) 2017 -- 2019 SINTEF Digital
Loaded './camp.yaml'.
Loading configurations from './out' ...
- Executing ./out/config_1
1. Building images ...
$ bash build_images.sh (from './out/config_1/images')
2. Starting Services ...
$ docker-compose up -d (from './out/config_1')
3. Running tests ...
$ docker-compose run --user=0 -v /camp/samples/java/out/config_1/images/tests_0/:/tests tests mvn test (from './out/config_1')
Test execution aborted!
- Error: A shell command failed (code: 1)
$ docker-compose run --user=0 -v /camp/samples/java/out/config_1/images/tests_0/:/tests tests mvn test
Check out logs in 'camp_execute.log'.
That's all folks!
Looking at the log file generated, we see that the mounting fails, at least maven does not found any pom.xml. My guess is that the mounting command is interpreted by the host (as we reuse the host docker deamon to spawn sibling container) and the given mounting path make no sense on the host.
camp@bash:./out/config_1/images$ bash build_images.sh
All images ready.
camp@bash:./out/config_1$ docker-compose up -d
Creating network "config_1_default" with the default driver
Creating config_1_greetings_1 ...
Creating config_1_tests_1 ...
^[[2B
camp@bash:./out/config_1$ docker-compose run --user=0 -v /camp/samples/java/out/config_1/images/tests_0/:/tests tests mvn test
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.373 s
[INFO] Finished at: 2019-03-26T14:33:27+00:00
[INFO] Final Memory: 7M/121M
[INFO] ------------------------------------------------------------------------
[ERROR] The goal you specified requires a project to execute but there is no POM in this directory (/tests). Please verify you invoked Maven from the correct directory. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.gg
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MissingProjectException
May relate to Issue #33.
CAMP realize fails to perform substitutions when they target a file within a nested structure directory in the template. such as sphinx/src/setup.py
.
For instance, in the following snippet I declare a variable whose realization targets the file sphinx/src/setup.py
.
variables:
jinja2:
values: [ v2.3, v2.3.1 ]
realization:
- targets: [ sphinx/src/setup.py ]
pattern: 'Jinja2>=2.3'
replacements:
- Jinja2==2.3
- Jinja2==2.3.1
When I run CAMP generate, I got the following error:
$ camp realize -d .
CAMP v0.3.5 (MIT)
Copyright (C) 2017 -- 2019 SINTEF Digital
Loaded './camp.yml'.
Loading configurations from './out' ...
Unexpected error:
- [Errno 2] No such file or directory: './out/config_1/images/sphinx_0/setup.py'
In file: /home/fchauvel/Documents/camp/camp/realize.py
169: with open(path, "r") as resource_file:
Please report this at 'https://github.com/STAMP-project/camp/issues'.
That's all folks!
Which indicates that CAMP search a file a the sphinx
directory of the template, instead of searching it in the src
directory.
Here is the structure of my CAMP project
tree .
.
├── camp.yml
├── out
│ ├── config_1
│ │ ├── configuration.dot
│ │ ├── configuration.yml
│ │ └── images
│ │ └── sphinx_0
│ │ ├── Dockerfile
│ │ └── src
│ │ └── setup.py
│ └── config_2
│ ├── configuration.dot
│ └── configuration.yml
├── template
│ └── sphinx
│ ├── Dockerfile
│ └── src
│ └── setup.py
└── z3_errors.log
9 directories, 10 files
None
we define rules and images to download in two different files, this can be moved to a single file
Current CAMP documentation does not explicitly describes the required the CAMP input files for configuring the configuration amplification domain model (features.yml, images.yml, genimages.yml, composite.yml, variables.yml) including instructions to generate other domain input models. Examples are given for XWiki and Atos use cases, but not instructions to provide other model instances for other scenarios. Therefore, the application of CAMP in order scenarios requires the support of CAMP technical team.
Therefore, the metamodel of every CAMP input model should be specify in CAMP documentation (or reference to the specifications of these metamodels should be provided) together with examples (code snippets) for relevant modeling elements (e.g. modeling a feature variations, image building rules, etc).
N/A
See #14 for a requirement to specify CAMP input model locations.
Preview to follow the link or open file .github/ISSUE_DOC.md
I am running camp execute on the sphinx use case. I get this error:
Unexpected error:
- 'ascii' codec can't decode byte 0xc3 in position 8123: ordinal not in range(128)
In file: /usr/lib/python3.5/encodings/ascii.py
26: return codecs.ascii_decode(input, self.errors)[0]
Please report this at 'https://github.com/STAMP-project/camp/issues'.
That's all folks!
I am using camp from docker-hub: fchauvel/camp. I pulled the latest image, but I am not sure it is up to date. I will try re-building the image locally and see if it helps.
The path camp/samples/xwiki used in xwiki.html page is wrong.
The correct one is camp/samples/stamp/xwiki.
Visit: https://stamp-project.github.io/camp/pages/xwiki.html
and follow the tutorial.
docs/pages/xwiki.md
Preview to follow the link or open file .github/ISSUE_DOC.md
Seems impossible to run several goals:
goals:
running:
- A
- B
I've followed the CAMP documentation for CityGo use case. Everything goes well and CAMP generates 10 configurations.
Questions:
1- I always get 10 configurations generated, regardless the CAMP command I issue:
camp generate -d . --all
or
camp generate -d . --coverage
Is there a configuration to ask CAMP to generate a number of configurations (or at least up to that number)?
2- Where can we described in the camp.yml file the location of the input docker-compose.yml file?
Is this one by default: ./template/docker-compose.yml?
Bugs:
1- To generate the images, I issue the build_images.sh script generated by CAMP in the out/config_X folder, but I get the exception:
yosu@avalon:~/Projects/STAMP/Git/camp/samples/stamp/atos/out/config_1/images> sh ./build_images.sh
"docker build" requires exactly 1 argument.
See 'docker build --help'.
Usage: docker build [OPTIONS] PATH | URL | -
Build an image from a Dockerfile
"docker build" requires exactly 1 argument.
See 'docker build --help'.
Usage: docker build [OPTIONS] PATH | URL | -
Build an image from a Dockerfile
All images ready.
I think the script is missing the path . at the end of each docker build line, so it should be:
docker build -t camp-postgres_0 -f ./postgres_0/Dockerfile .
instead of
docker build -t camp-postgres_0 -f ./postgres_0/Dockerfile
2- Once I fix this, and run the docker build, I get an building error:
Step 13/18 : ADD init-db.sql $DIR_DB
ADD failed: stat /home/docker/tmp/docker-builder408682663/init-db.sql: no such file or directory
I will investigate this from my side.
Preview to follow the link or open file .github/ISSUE_DOC.md
CAMP does not detect when there is a mismatch between the number of value provided for a given variable, and the number of value provided in the realization. For instance, in the following snippet, I declare two values but I give 3 substitutions.
flask:
values: [ v0.12.4, v1.0.1 ]
realization:
- targets: [ greetings/setup.py ]
pattern: "flask == 1.0.2"
replacements:
- flask == 0.12.4
- flask == 1.0.1
- flask == 1.0.2
Simply run the camp generate
command. No warning or error is reported.
Relate to Issue #21 about model model validation.
Companion website has some links to CAMP repositories, mainly for example purposes. For instance, CAMP execute page has some references to the Java sample webapp, and some files within it. Following the links, it results in "page not found error", because the referenced url is http://github.com/stamp-project/camp/samples/java, while should be https://github.com/STAMP-project/camp/tree/master/samples/java.
The version of libgomp required makes apt-get install fail. Work around is to change version to 4.9.2-10+deb8u2
. You can check the error below.
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
libgomp1 : Depends: gcc-4.9-base (= 4.9.2-10+deb8u1) but 4.9.2-10+deb8u2 is to be installed
E: Unable to correct problems, you have held broken packages.
just run checkout last version and run:
docker build --no-cache .
Preview to follow the link or open file .github/ISSUE_DOC.md
The new module of CAMP tool generates only 1 instance configuration generated by CAMP. The principal idea of CAMP tool and a good approach is:
for n configurations generated by CAMP:
- Build and deploy <my_app> instance
- Run tests suit selected
- Recollect Test results
- Show recollected metrics
CAMP tool
Preview to follow the link or open file .github/ISSUE_DOC.md
When generating two configs, it seems that the generated docker-compose file for one of the two generated configs is erroneous.
In the build:
parameter of generated compose, the reference to the right dockerfile file is not always correct. It does correctly the changes for one configuration but not for the other.
I had to define the changes manually in camp.yml to make this working as below:
- targets: [ docker-compose.yml ]
pattern: "build: ./springpetclinicMS"
replacements: [ "build: ./images/springpetclinicpg_0" ]
However, I didn't need to define the replacements for the other config as it was correctly generated, defining the right build reference.
-generate 2 configs.
-check if the the build parameter in the generated dokcer-compose files references to the right dockerfile location.
Preview to follow the link or open file .github/ISSUE_DOC.md
Command on the documentation page is outdated using v1.0.0 here./camp/pages/setup.html
Just try the command on the documentation here.
$ docker run -it -v $(pwd):/camp/workspace fchauvel/camp:v1.0.0 camp generate -d workspace
Preview to follow the link or open file .github/ISSUE_DOC.md
CAMP execute stops and report that a shell command has failed, whereas only a test failed, and the test command (e.g., mvn test
) returned a zero code.
None
None
we can specify location of feature.yml
we can specify location of composite.yml
we can specify location of images.yml
we can specify location of output directory with generated configurations
root@maas-1:~/docker-flame-graphs/camp-0.3.1/samples/java# docker-compose -version
docker-compose version 1.23.2, build 1110ad01
root@maas-1:~/docker-flame-graphs/camp-0.3.1/samples/java# docker -v
Docker version 18.09.5, build e8ff056
I hit the following error message when running camp over greetings example:
root@maas-1:~/docker-flame-graphs/camp/samples/java# docker run -t -v $PWD:/campworkingdir camp camp execute -d /campworkingdir
CAMP v0.3.4 (MIT)
Copyright (C) 2017 -- 2019 SINTEF Digital
Loaded '/campworkingdir/camp.yaml'.
Loading configurations from '/campworkingdir/out' ...
- Executing /campworkingdir/out/config_1
1. Building images ...
$ bash build_images.sh (from '/campworkingdir/out/config_1/images')
2. Starting Services ...
$ docker-compose up -d (from '/campworkingdir/out/config_1')
5. Stopping Services ...
$ docker-compose down --volumes --rmi all (from '/campworkingdir/out/config_1')
Test execution aborted!
- Error: A shell command failed (code: 1)
$ docker-compose down --volumes --rmi all
Check out logs in 'camp_execute.log'.
That's all folks!
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/camp/execute/engine.py", line 220, in execute
self._start_services(each_path)
File "/usr/local/lib/python3.5/dist-packages/camp/execute/engine.py", line 244, in _start_services
self._shell.execute(self._START_SERVICES, path)
File "/usr/local/lib/python3.5/dist-packages/camp/execute/engine.py", line 42, in execute
result = self._run_shell(command)
File "/usr/local/lib/python3.5/dist-packages/camp/execute/engine.py", line 66, in _run_shell
stderr)
camp.execute.engine.ShellCommandFailed: docker-compose up -d (with code 1
Output:
b''
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/camp/core.py", line 139, in execute
reports = engine.execute(configurations)
File "/usr/local/lib/python3.5/dist-packages/camp/execute/engine.py", line 231, in execute
self._stop_services(each_path)
File "/usr/local/lib/python3.5/dist-packages/camp/execute/engine.py", line 326, in _stop_services
self._shell.execute(self._STOP_SERVICES, path)
File "/usr/local/lib/python3.5/dist-packages/camp/execute/engine.py", line 42, in execute
result = self._run_shell(command)
File "/usr/local/lib/python3.5/dist-packages/camp/execute/engine.py", line 66, in _run_shell
stderr)
camp.execute.engine.ShellCommandFailed: docker-compose down --volumes --rmi all (with code 1
Output:
b''
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/camp", line 10, in <module>
sys.exit(main())
File "/usr/local/lib/python3.5/dist-packages/camp/run.py", line 26, in main
command.send_to(camp)
File "/usr/local/lib/python3.5/dist-packages/camp/commands.py", line 185, in send_to
camp.execute(self)
File "/usr/local/lib/python3.5/dist-packages/camp/core.py", line 156, in execute
self._ui.shell_command_failed(error)
File "/usr/local/lib/python3.5/dist-packages/camp/ui.py", line 145, in shell_command_failed
self._print(error.output)
File "/usr/local/lib/python3.5/dist-packages/camp/ui.py", line 221, in _print
self._output.write(pattern+ "\n")
TypeError: can't concat bytes to str
-git clone https://github.com/STAMP-project/camp
-docker rmi camp
-docker build . -t camp
-cd samples/java
-docker run -t -v $PWD:/campworkingdir camp camp generate -d /campworkingdir (works fine)
-docker run -t -v $PWD:/campworkingdir camp camp realize -d /campworkingdir (works fine)
-docker run -t -v $PWD:/campworkingdir camp camp execute -d /campworkingdir (error)
Preview to follow the link or open file .github/ISSUE_DOC.md
Hi @jesus-gorronogoitia, @danzone
To sum up our recent conversation in context of WP4 about feature for CAMP, I have created an issue where we can discuss features for an upcoming evaluation of CAMP. I have created one issue where I have listed some improvements #14. Please comment here things I forgot to mention or you can just file bugs.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.