At the moment, if run under a service the credentials provided as CLI parameters are seen in the process list.

16664 ?        S      0:00 /usr/sbin/nagios -d /etc/nagios/nagios.cfg
16665 ?        S      0:00  \_ /usr/bin/python /usr/bin/ss-execute --nagios -w 15 -u abc -p xxx --parameters testclient:cloudservice=ec2-eu-west,apache:cloudservice=ec2-eu-west --endpoint https://54.217.108.111 examples/tutorials/service-testing/system

Native image on AWS and Exoscale already has a user with uid=1000, the uid chosen for the tuser account. Remove the explicit assignment of the uid and let the OS chose it.

I've been trying to get ss-run-get to work with Slipstream 2.0.3 and have had no success.

The server is accessible in firefox. It can also be accessed by curl, but ONLY when specifying '-3' (use SSL v3 only).

From digging further, it appears that httplib2 has a problem with SSLv3, so without either getting it patched or making the Slipstream client HttpClient use urllib2 (with appropriate options) or requests, it won't work:
http://stackoverflow.com/questions/8007348/error-opening-ssl-connection-using-httplib2

On StratusLab the new image ID is searched in PDisk. If it's not found, Orch. should fail the build.

At the moment, in the case of such failure, the global state is Terminal and the Node pictograms are green.

Upgrade to the latest Ubuntu 12.04 and CentOS 6.4 images in the examples to minimise the deployment and configuration time.

Add something like the following options to ss-execute to support smoke testing:

• --wait-for-completion: should trigger the run and wait for it to terminate, either successfully or not
• --nagios: return 0 if ok and 2 if failed (http://nagios.sourceforge.net/docs/3_0/pluginapi.html)

-{run,terminate,describe}-instances commands should read user credentials from configuration file. At the moment user Cloud credentials are visible on the host.

57304 pts/1    S      0:00      \_ python /usr/bin/vcloud-run-instances --username ssadmin@helixnebula --password xxx

A common way to do this has to be established via Java connector and update to *-instances CLIs. E.g., a naive approach

• CLIs should accept -c <conf_file> parameter
• Java connector creates temp file with the credentials and provides to the CLI
• Java connector removes the temp file after the command returns

Currently the client rpm only pulls in the tarball containing the client commands. The rpm should also unpack those commands so that they are available without further configuration.

The "native image" identifiers are missing from the modules when the they are uploaded from a serialised XML file.

Right now SlipStreamClient relies on httplib2 which is old and doesn't provide modularity and extensibility.
Requests library is a modern way of dealing with HTTP and easier to use/modify.

The slipstream.setenv file is currently buried in the /opt/slipstream/client/sbin directory. Having it either in the home directory or in /tmp would simplify Clara's life

The tgz files generated by maven are not readable by the tarfile module of python 2.7.

For example when the orchestrator start, the bootstrap script downloads slipstreamclient.tgz but fail to extract it. So the deployment never start.

The SlipStreamClient should abort if the script take too much time to run.
The value of the timeout should be retrieved in user parameters.

The only way to terminate a "Run Image" from SlipStream is to click on the Terminate button.
But in this case the status of the Run will go to Cancelled.
Maybe it's better to go in status "Done" for the case of a "Run Image".

SS examples refer to the hardcoded CC-IN2P3, LLR and stratuslab cloud connector instances which may not be configured on the SS instance against which the ss-module-upload gets invoked.

For the screenshots see slipstream/SlipStreamServer#96

The configuration of the torque example always seems to leave one worker node in the "down" state. Determine why this is the case and fix it.

With the schema upgrade from 2.2, the examples don't work anymore, as the ss-module-upload returns a forbidden
Also: the error log seems different when running locally and on a system wide installation, with the local version providing more useful error

Orchestrator should be able to abort the Run on any error. In this case Orch failed on loading connector module but didn't abort the Run. The state of the Run was Inactive.

================================================
=== slipstream-orchestrator bootstrap script ===
================================================
Retrieving the latest version of the SlipStream from: https://212.159.217.24/downloads/slipstreamclient.tgz
Expanding tarball: /opt/slipstream/client/slipstreamclient.tgz
Retrieving stratuslab library from https://212.159.217.24/downloads/slipstreamclient-pdisk.tgz
Expanding tarball: /opt/stratuslab/slipstreamclient-pdisk.tgz
Creating reports directory: /tmp/slipstream/reports
PYTHONPATH environment variable set to: /opt/stratuslab:/opt/slipstream/client/lib:
Done bootstrapping!

Traceback (most recent call last):
  File "/opt/slipstream/client/sbin/slipstream-orchestrator", line 49, in <module>
    MainProgram()
  File "/opt/slipstream/client/sbin/slipstream-orchestrator", line 32, in __init__
    super(MainProgram, self).__init__(argv)
  File "/opt/slipstream/client/lib/slipstream/CommandBase.py", line 88, in __init__
    self._callAndHandleErrorsForCommands(self.doWork.__name__)
  File "/opt/slipstream/client/lib/slipstream/CommandBase.py", line 113, in _callAndHandleErrorsForCommands
    res = self.__class__.__dict__[methodName](self, *args, **kw)
  File "/opt/slipstream/client/sbin/slipstream-orchestrator", line 45, in doWork
    orchestrator.execute()
  File "/opt/slipstream/client/lib/slipstream/executors/Machine.py", line 29, in execute
    executor = self.executorFactory.createExecutor(self.configHolder)
  File "/opt/slipstream/client/lib/slipstream/executors/orchestrator/OrchestratorExecutorFactory.py", line 34, in createExecutor
    cloudWrapper.initCloudConnector(configHolder)
  File "/opt/slipstream/client/lib/slipstream/wrappers/CloudWrapper.py", line 37, in initCloudConnector
    createConnector(configHolder or self.configHolder)
  File "/opt/slipstream/client/lib/slipstream/cloudconnectors/CloudConnectorFactory.py", line 35, in createConnector
    return loadModule(get_connector_module_name(configHolder)). \
  File "/opt/slipstream/client/lib/slipstream/util.py", line 379, in loadModule
    return __import__(name, fromlist=namespace)
  File "/opt/slipstream/client/lib/slipstream/cloudconnectors/stratuslab/StratuslabClientCloud.py", line 26, in <module>
    from stratuslab.ConfigHolder import ConfigHolder as StratuslabConfigHolder
ImportError: No module named stratuslab.ConfigHolder

When selecting 'pdisk' for Build image the SL cloud client times out not being able to find tag:SlipStream-bb657136-7e18-4e8b-abff-acee16b7472a. The problem is that the new machine disk has the following properties

Owner   pdisk
Visibility  PRIVATE

Are we running too old PDisk / CLI (TMSaveCache.py setting wrong Visibility)? Should we upgrade?

[root@pd-atos ~]# rpm -qa | grep pdisk
stratuslab-pdisk-host-13.05.0-0.20130527020758.el6.noarch
stratuslab-pdisk-server-13.05.0-0.20130527020758.el6.noarch
[root@pd-atos ~]# 

[root@fe-atos ~]# rpm -qa | grep stratuslab-api
stratuslab-api-externals-13.05.0.RC2-0.20130614020513.el6.noarch
stratuslab-api-13.05.0.RC2-0.20130614020513.el6.noarch
[root@fe-atos ~]# 

pdisk user can get the image.

~ konstan$ stratus-describe-volumes --pdisk-endpoint=212.159.217.21:8445 --pdisk-username=konstan --pdisk-password=xxx -f tag:SlipStream-bb657136-7e18-4e8b-abff-acee16b7472a
No disk to show
~ konstan$ stratus-describe-volumes --pdisk-endpoint=212.159.217.21:8445 --pdisk-username=pdisk --pdisk-password=xxx -f tag:SlipStream-bb657136-7e18-4e8b-abff-acee16b7472a
:: DISK 49f98fc1-5add-4d36-aad0-782de95fd91b
    count: 0
    tag: SlipStream-bb657136-7e18-4e8b-abff-acee16b7472a
    owner: pdisk
    identifier: FDTm7WKQJbt4g1P4CkLylF0oSl7
    size: 6
~ konstan$ 

Bootstrap fails on StratusLab's Ubuntu 14.04 https://marketplace.stratuslab.eu/metadata/KhGzWhB9ZZv5ZkLSZqm6pkWx7ZF

root@onevm-203:~# cat /tmp/slipstream/reports/orchestrator.slipstream.log 
--2014-05-27 08:18:33--  https://134.158.75.137/downloads/slipstream.bootstrap
Connecting to 134.158.75.137:443... connected.
WARNING: cannot verify 134.158.75.137's certificate, issued by '/C=EU/CN=134.158.75.137':
  Self-signed certificate encountered.
HTTP request sent, awaiting response... 200 OK
Length: 13603 (13K) [application/octet-stream]
Saving to: '/tmp/slipstream.bootstrap'

     0K .......... ...                                        100%  275M=0s

2014-05-27 08:18:33 (275 MB/s) - '/tmp/slipstream.bootstrap' saved [13603/13603]

dpkg-preconfigure: unable to re-open stdin: No such file or directory
warning: no files found matching 'pip/cacert.pem'
warning: no files found matching '*.html' under directory 'docs'
warning: no previously-included files matching '*.rst' found under directory 'docs/_build'
no previously-included directories found matching 'docs/_build/_sources'
dpkg-preconfigure: unable to re-open stdin: No such file or directory
root@onevm-203:~# 

Should be published to clojars for distribution

Here's the error: "Unknown key web.1:multiplicity"

This is needed in the case of partially completed and failed builds when user wants/needs to Terminate the Run.

Need to find a way how to do this because the connector actually doesn't start the VM, but instead uses directly stratuslab.Creator.

With this enhancement all Clouds which have the vApp concept are correctly handled.
This enhancement also adds the ability, in certain cases, to terminate only the orchestrator.

All connectors should be updated:

• StratusLab
• OpenStack
• CloudStack
• PhysicalHost
• Amazon EC2
• CloudSigma
• vCloud
• Abiquo
• IBM SmartCloud Entry

The ss-module-upload command should take file arguments so that the command doesn't have to be run from the directory containing the files to upload.

Connectors which uses TerminateRunServerSide feature will finish a "Build Image" run in the state Cancelled even if everything went well.

These commit are related:
18114ec
eb82fb8
slipstream/SlipStreamServer@a2a2367

C'est un problème d'accès par SSH. Les workers ne sont pas configurés pour se connecter en SSH au master.
Il y a plusieurs façons de faire. L'une d'elle consiste à créer une clé SSH sans mot de passe et l'ajouter à tous les workers, par exemple de la façon suivante :

# créé une clé RSA sans mot de passe.
mkdir RSASansPassword
ssh-keygen -t rsa -N "" -f ~/RSASansPassword/id_rsa
# Copie la clé RSA sur chaque worker (à répéter autant que nécessaire)
scp ~/RSASansPassword/* root@<nomduworker>:/home/tuser/.ssh/
# fait en sorte que master accepte la clé sans mot de passe pour l'utilisateur tuser
cat ~/RSASansPassword/id_rsa.pub | ssh tuser@<nomdumaster> "cat >> /home/tuser/.ssh/authorized_keys"

Cela pemettra à tuser@ de se connecter en tant que

tuser@<nomdumaster>

Pour récupérer les données au travers d'un script, il faudra rajouter "-o" comme option pour qu'à la première connexion il n'y ait pas la question "êtes-vous sûr d'accepter cette clé SSH ?".

Une façon plus simple serait de préconfigurer les master et worker nodes pour partage une clé commune. C'est tout à fait faisable mais il faudrait personnaliser le script d'initialisation des machines. Autre possibilité : héberger les données sur un HTTP externe, ou encore créer un disque virtuel persistant.

Autre chose, je suis allé présenter à Cal les problèmes que l'on a rencontré ce matin ainsi que nos suggestions.
Une bonne partie devrait être corrigée dans la prochaine version de Slipstream et il va chercher pourquoi le worker nodes plantent parfois.

Bon courage,
Julien

At the moment, if VMs fail to start on a cloud provider the Run stays in Initialising state.

The recursive upload of the examples fails because the modules are added in the wrong order. Fix the ordering or add ability to upload modules when parents are not defined.

Orchestrator failing to instantiate VMs should have a capability to act according to "General.On Error Run Forever" switch.

The client currently requires the python-httplib2 library. This is only available from EPEL and requires that it be added to the yum configuration. Moreover, the SSL handling within this library (should we ever use it) is poor. The python-requests library is a part of the core CentOS distribution (not needing the EPEL repository) and handles SSL much more sensibly.

I would propose to switch from the httplib2 library to the requests library to minimise the repositories and improve the SSL handling.

https://github.com/slipstream/SlipStreamClient/blob/master/THIRD-PARTY.txt

Instead of using unlimited thread pool of immediately started threads - use a task queue with a limited number of working threads.

The StratusLab client has been refactored to allow backward compatibility with previous StratusLab deployments while supporting the newer CIMI interface for new deployments.

The SlipStream client relies directly on the Runner class for OpenNebula. This will fail when StratusLab infrastructures deploy a newer StratusLab release. The client should be refactored to use the interface and add a parameter to switch between which is active for a given infrastructure.

Move the examples to the client package so that the ss-module-upload/download commands and examples are in the same package.

The file slipstream.client.conf is required when running the client locally. This should not be the case.

The help for the ss-user-put argument describes this as the username whereas it is actually the XML file which contains the modified user record.

Resolve errors indicated by pylint.

When running the server on a standard CentOS 6 machine, the ss-upload-module command (and probably others) prints the following to the standard output. This happens within the httplib2 request. This output isn't useful for the user and should be turned off.

Compiler warnings:
  WARNING:  'org.apache.xerces.jaxp.SAXParserImpl: Property 'http://javax.xml.XMLConstants/property/accessExternalDTD' is not recognized.'
Warning:  org.apache.xerces.parsers.SAXParser: Property 'http://javax.xml.XMLConstants/property/accessExternalDTD' is not recognized.
Warning:  org.apache.xerces.parsers.SAXParser: Property 'http://www.oracle.com/xml/jaxp/properties/entityExpansionLimit' is not recognized.

The ss-module-upload/download commands import modules from com.sixsq.slipstream.* where this should be just slipstream.*.

Currently the StratusLab connector uses an external messaging service to recover the created image's checksum/image ID. Use an alternate method to recover this information rather than the external service.

Currently if the value given to ss-set is larger than the limit on the server, then the user receives a 500 server error with a not very helpful error message. Fixing this is related to the issue slipstream/SlipStreamServer#201.

Set the initial value of url.ssh to ssh://user@machine for the nodes. This value can be reset by the user but the default value should be correct.

In this case 26 Nodes were requested as part of the deployment. Only 17 were provisioned. And after the below failure Orch continued running w/o uploading logs to the server.

==== Starting instances
ERROR: Error starting instances with error: [VirtualMachineAllocate] User [2] not authorized to perform action on virtual machine.
ERROR: Error executing node, with detail: [VirtualMachineAllocate] User [2] not authorized to perform action on virtual machine.
Traceback (most recent call last):
  File "/opt/slipstream/client/lib/slipstream/executors/MachineExecutor.py", line 51, in _execute
    getattr(self, 'on' + state)()
  File "/opt/slipstream/client/lib/slipstream/executors/orchestrator/OrchestratorDeploymentExecutor.py", line 34, in onInitializing
    self.wrapper.startImages()
  File "/opt/slipstream/client/lib/slipstream/wrappers/CloudWrapper.py", line 70, in startImages
    userInfo, nodesForThisOrchestrator)
  File "/opt/slipstream/client/lib/slipstream/cloudconnectors/BaseCloudConnector.py", line 244, in startNodesAndClients
    self._startNodeInstantiationTasksWaitFinished(user_info, nodes_info)
  File "/opt/slipstream/client/lib/slipstream/cloudconnectors/BaseCloudConnector.py", line 254, in _startNodeInstantiationTasksWaitFinished
    self._waitNodesStartupTasksFinshed()
  File "/opt/slipstream/client/lib/slipstream/cloudconnectors/BaseCloudConnector.py", line 262, in _waitNodesStartupTasksFinshed
    self.tasksRunnner.wait_tasks_finished()
  File "/opt/slipstream/client/lib/slipstream/utils/tasksrunner.py", line 41, in wait_tasks_finished
    self._process_exc_queue()
  File "/opt/slipstream/client/lib/slipstream/utils/tasksrunner.py", line 75, in run
    super(ThreadWrapper, self).run()
  File "/usr/lib/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/opt/slipstream/client/lib/slipstream/cloudconnectors/BaseCloudConnector.py", line 277, in _startNodeInstanceAndClient
    cloudSpecificData)
  File "/opt/slipstream/client/lib/slipstream/cloudconnectors/stratuslab/StratuslabClientCloud.py", line 190, in _startImage
    runner = self._runInstance(imageId, configHolder)
  File "/opt/slipstream/client/lib/slipstream/cloudconnectors/stratuslab/StratuslabClientCloud.py", line 247, in _runInstance
    runner = self._doRunInstance(imageId, configHolder)
  File "/opt/slipstream/client/lib/slipstream/cloudconnectors/stratuslab/StratuslabClientCloud.py", line 260, in _doRunInstance
    runner.runInstance()
  File "/opt/stratuslab/stratuslab/vm_manager/Runner.py", line 539, in runInstance
    vmId = self.cloud.vmStart(vmTpl)
  File "/opt/stratuslab/stratuslab/cloud/one.py", line 96, in vmStart
    self._raiseIfError(isSuccess, detail)
  File "/opt/stratuslab/stratuslab/cloud/one.py", line 135, in _raiseIfError
    raise OneException(reason)
OneException: [VirtualMachineAllocate] User [2] not authorized to perform action on virtual machine.
ERROR: Failing... [VirtualMachineAllocate] User [2] not authorized to perform action on virtual machine.