vespene-io / _old_vespene Goto Github PK

Sorry, I had to. 😆

Disclaimer: I'm not really familiar with Django, so there might be something version related.
This is on a clean CentOS 7.5 box.

An incorrect supervisord.conf was being generated - it turns out there is an issue with the default value of the omitted source parameter for generate_supervisor.py

This source = options.get('source', '/opt/vespene/') is not working - source becomes None

This does work:

parser.add_argument('--source', type=str, help='source', default='/opt/vespene/')
..
source = options.get('source')

Edit: (I see you just fixed this by adding the source= parameter while I was typing this message).

On each page, just as build details, it would be nice to be able to stop the build without going back to the build list.

To do this, we should add a generic facility, through the view classes, that can return custom HTML to appear below the Django Crispy Form.

Stopping builds should be a first goal, though the linked suggestion below talks about hyperlinks to related resources, which could also be useful.

The widget should be drawn so that it is not confusingly part of the forum, perhaps surrounded in some (default-collapsed) accordion.

https://talk.msphere.io/t/crosslink-variables-from-variable-sets-to-corresponding-variables-definitions/101/4

The page http://docs.vespene.io/development_setup.html says to clone https://github.com/vespene/vespene, which is clearly not this repo.

Bug description
during bash 5_tutorial.sh , there will be an error message "AssertionError: database connection isn't set to UTC"

Steps to reproduce

I just follow all steps in http://docs.vespene.io/setup.html

Expected behavior

no error

Actual behavior

2018-11-13 02:25:41 [administrator@ubuntu setup]$ bash 5_tutorial.sh 
detected Ubuntu/Debian
Traceback (most recent call last):
  File "manage.py", line 18, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.6/dist-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.6/dist-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.6/dist-packages/django/core/management/base.py", line 316, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.6/dist-packages/django/core/management/base.py", line 353, in execute
    output = self.handle(*args, **options)
  File "/opt/vespene/vespene/management/commands/tutorial_setup.py", line 112, in handle
    user1 = admins.first()
  File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py", line 605, in first
    for obj in (self if self.ordered else self.order_by('pk'))[:1]:
  File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py", line 268, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py", line 1186, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/usr/local/lib/python3.6/dist-packages/django/db/models/query.py", line 63, in __iter__
    for row in compiler.results_iter(results):
  File "/usr/local/lib/python3.6/dist-packages/django/db/models/sql/compiler.py", line 1466, in cursor_iter
    for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
  File "/usr/local/lib/python3.6/dist-packages/django/db/models/sql/compiler.py", line 1466, in <lambda>
    for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
  File "/usr/local/lib/python3.6/dist-packages/django/db/utils.py", line 96, in inner
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/django/db/backends/postgresql/utils.py", line 6, in utc_tzinfo_factory
    raise AssertionError("database connection isn't set to UTC")
AssertionError: database connection isn't set to UTC

Operating system and version

 cat /etc/os-release 
NAME="Ubuntu"
VERSION="18.04 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

Any additional info

I only change the database password in 0_common.sh

Certain types of objects - projects in particular, would be great if they had a copy button.

This button can copy all fields but should present a dialog to enter the name before doing the copy.

Bug description: Created a test build with the script:

#!/bin/bash
echo "replace this section"

Getting this in the message:

    0.00m | mkdir -p 3
    0.00m | chmod 770 3
    0.00m | ----------
Pre hooks...
    0.00m | running build asunder user: nobody
    0.00m | ----------
Building...
    0.00m | chmod -R 777 3
    0.00m | chmod: cannot access '3': No such file or directory

    0.00m | build failed with exit code 1
    0.00m | ----------
Post hooks...

Running cat /var/log/vespene/worker_tutorial-pool.log:

2018-11-03 01:54:54,031 - vespene - INFO - serving queue: tutorial-pool
2018-11-03 01:55:21,757 - vespene - INFO - serving queue: tutorial-pool
2018-11-03 01:56:51,018 - vespene - DEBUG - building: 3, project: Testing Vespene
2018-11-03 01:56:51,053 - vespene - DEBUG - executing: mkdir -p 3
2018-11-03 01:56:51,073 - vespene - DEBUG - executing: chmod 770 3
2018-11-03 01:56:51,102 - vespene - DEBUG - executing: chmod -R 777 3
2018-11-03 01:56:51,120 - vespene - ERROR - an error occurred
Traceback (most recent call last):
  File "/opt/vespene/vespene/workers/builder.py", line 158, in go
    self.main()
  File "/opt/vespene/vespene/workers/builder.py", line 69, in main
    self.run_build_script()
  File "/opt/vespene/vespene/workers/builder.py", line 91, in run_build_script
    self.isolation_manager.execute()
  File "/opt/vespene/vespene/workers/isolation.py", line 55, in execute
    return self.provider.execute()
  File "/opt/vespene/vespene/plugins/isolation/sudo.py", line 84, in execute
    commands.execute_command(self.build, "chmod -R %s %s" % (self.chmod, self.build.working_dir), output_log=False, message_log=True)
  File "/opt/vespene/vespene/workers/commands.py", line 132, in execute_command
    raise Exception("Failed")
Exception: Failed
2018-11-03 01:56:51,124 - vespene - DEBUG - flagging build as failure
2018-11-03 01:56:51,130 - vespene - DEBUG - flagging build as done

Steps to reproduce: Create a new project with some test code, re-use the tutorial worker. (this is on a fresh install, I haven't actually run any of the tutorial projects (deleted them))

Operating system and version: Ubuntu 18.04 LTS

I'm guessing this is a permissions issue. I've installed vespene using a vespene user.

Anytime the bash 1_prepare.sh runs and sources the 0_common.sh it should also update the database.py file in the /etc/vespene/settings.d/database.py file.

In a few places in the code we use the Django messages feature to feature alerts at the top of the screen.

We need to make sure we html escape any project names or user data that can be shown there to make sure they do not include HTML.

This is already done in the the list view pages and is handled by the forms, but needs to happen in the messages usage as well.

To make it easy for people to set up the application, I've added by suggestion the "--bind 0.0.0.0:8000" option to supervisor's configuration for now.

This setting is good for people who are trying it out for the first time, but not ideal for those who want to proxy the service with NGINX or Apache, which should be the additional production run - they would need to know to edit the supervisor configuration.

To fix this, we can either mention this in the docs in the security guide or add a new option for bind options in 0_common.sh, which would also make it super easy to reassign the port.

I think I like that last option better.

Would it be useful to have a docker compose file that installs on an ubuntu container with maybe a separate postgresql container?

A lot of folks have expressed interest in getting Vespene to work on more than the "official" list of Linux distros or occasionally have installed from very minimal environments.

For instance, an environment might be missing "ssh-add" or it might be out of path. They also might have forgotten to create the build root.

In these scenarios, the worker really needs a precheck class (workers/pre_check.py) that runs when the daemon starts. This can use shutil.which to find if certain CLI tools are in path and that the build environment is as expected.

It could also choose to warn if the permissions of the build root were not locked down enough.

reference #66 #67
and
https://talk.vespene.io/t/simplest-project-throws-nonetype-object-has-no-attribute-append-message/46/8

Bug description

Running the 0_common.sh script with sudo tells me to use sudo.

Steps to reproduce

Using a minimal debian run the script. Running "whoami" with sudo will return "root" so this is not a valid test.

Expected behavior

No error when run with sudo.

Actual behavior

Error saying to use sudo.

Operating system and version

Debian Stretch

Any additional info

Bare with me. Testing on Debain Stretch. Will try to do my first GitHub pull request. This is not funny.

I plan to write an autoscaling plugin system.

It will be invoked like so:

python manage.py autoscaler

And it will then look at the plugin configuration and work somewhat like my comments mentioned in this thread:

https://talk.vespene.io/t/slurm-isolation/

It should be usable to execute anything, but a trivial shell executor that can easily pass arguments to something like terraform or a custom shell script would be the easiest thing to write.

I'm not positive exactly what configuration will look like, but building this will also involve a new docs chapter.

The common use case this is intended to cover is the classic "AWS autoscaling" worker case.

To do this, we can also add a new flag to the worker, that allows it to run one and only one job and then exit, which could make some of this easier.

(We could also add a flag to the worker so it doesn't try to process the organization).

Configuration of autoscaling workers would be somewhat of an advanced topic, requiring construction of a good worker image preconfigured to speak to the DB, but in all this seems like it would be a very easy system to maintain. By suggesting tools like terraform and making it pluggable, we can avoid maintaining any cloud specific plugins.

Pipenv is now considered a recommended way to handle dependencies in Python.

I personally find it much more pleasant to work with Pipenv instead of using virtualenv manually.

Edit: typo

add a boolean flag on project called "recursive_checkout"

If this flag is set on the project, the git plugin should pass "--recursive" on checkout to allow for submodules.

There shouldn't be any need to do this in the import code for .vespene files (aka github.py), just git.py.

right now a .vespene file (see http://docs.vespene.io/importing.html) can attach a project to a pipeline as well as a stage.

But it can't create a new pipeline. Folks want to be able to do this.

This would require adding a new option to organizations like "allow_pipeline_defintion" (boolean) and if set could look at a .vespene field like

pipeline_definition: [ 'dev', 'stage', 'prod' ]

And if set, would define the named pipeline (pipeline: foo) and all the stages.

This is pretty easy to do, but is also potentially dangerous if a user wants to remove or rename stages of a pipeline, or if different projects want to define the pipeline differently.

So what we might do is make it so that if the pipeline already exists, it won't create it. This might take a little thinking.

We also need to edit the existing pipeline docs to make it a bit more clear pipelines can be managed from source control, as it's pretty obvious people want to be able to do that, and we kind of mostly can already with .vespene files.

Bug description
If you want to run Vespene on a port different from 8000 (still behing Nginx) and change it in the 0_common.sh it doesn't work.

Steps to reproduce
Change GUNICORN_OPTS="--bind 127.0.0.1:8000" to GUNICORN_OPTS="--bind 127.0.0.1:1000"

Expected behavior
The app should work on the defined port.

Actual behavior
The app dies with not much (if any) useful info in the logs.

Any additional info

Bug description
Any type of script I can come up with errors with the same output. More details here: https://talk.vespene.io/t/simplest-project-throws-nonetype-object-has-no-attribute-append-message/46

Steps to reproduce
Install on Debian 9, create a new project, in the script echo anything.

Expected behavior
The script should successfully execute and show the intended echo stuff in output.

Actual behavior
The script dies with the message 'NoneType' object has no attribute 'append_message'

Any additional info
Logging output from the worker log:

2018-11-01 23:27:35,204 - vespene - DEBUG - building: 9, project: stanovanja-build
2018-11-01 23:27:35,270 - vespene - DEBUG - executing: mkdir -p 9
2018-11-01 23:27:35,294 - vespene - DEBUG - executing: chmod 770 9
2018-11-01 23:27:35,342 - vespene - DEBUG - adding SSH key with passphrase!
2018-11-01 23:27:35,344 - vespene - DEBUG - /tmp/tmpegsf1c2o
2018-11-01 23:27:35,345 - vespene - DEBUG - /tmp/tmpb4q8bbs2
2018-11-01 23:27:35,345 - vespene - DEBUG - executing: /usr/bin/expect -f /tmp/tmpegsf1c2o
2018-11-01 23:27:35,353 - vespene - ERROR - an error occurred
Traceback (most recent call last):
  File "/opt/vespene/vespene/workers/builder.py", line 158, in go
    self.main()
  File "/opt/vespene/vespene/workers/builder.py", line 63, in main
    self.ssh_manager.add_all_keys()
  File "/opt/vespene/vespene/workers/ssh_agent.py", line 30, in add_all_keys
    self.add_key(access)
  File "/opt/vespene/vespene/workers/ssh_agent.py", line 48, in add_key
    self.ssh_add_with_passphrase(keyfile, access.get_unlock_password())
  File "/opt/vespene/vespene/workers/ssh_agent.py", line 86, in ssh_add_with_passphrase
    commands.execute_command(None, "/usr/bin/expect -f %s" % fname)
  File "/opt/vespene/vespene/workers/commands.py", line 128, in execute_command
    build.append_message("build failed with exit code %s" % process.returncode)
AttributeError: 'NoneType' object has no attribute 'append_message'
2018-11-01 23:27:35,355 - vespene - DEBUG - flagging build as failure
2018-11-01 23:27:35,387 - vespene - DEBUG - flagging build as done

When a build fails due to a problem with the build script (or sudo, whatever), we get an exception like this:

2019-01-08 15:03:56,751 - vespene - ERROR - an error occurred
Traceback (most recent call last):
File "/Users/michaeldehaan/Devel/vespene/vespene/workers/builder.py", line 158, in go
self.main()
File "/Users/michaeldehaan/Devel/vespene/vespene/workers/builder.py", line 69, in main
self.run_build_script()
File "/Users/michaeldehaan/Devel/vespene/vespene/workers/builder.py", line 91, in run_build_script
self.isolation_manager.execute()
File "/Users/michaeldehaan/Devel/vespene/vespene/workers/isolation.py", line 55, in execute
return self.provider.execute()
File "/Users/michaeldehaan/Devel/vespene/vespene/plugins/isolation/sudo.py", line 85, in execute
commands.execute_command(self.build, sudo_command, log_command=False, output_log=True, message_log=False)
File "/Users/michaeldehaan/Devel/vespene/vespene/workers/commands.py", line 132, in execute_command
raise Exception("Failed")
Exception: Failed
2019-01-08 15:03:56,751 - vespene - DEBUG - flagging build as failure
2019-01-08 15:03:56,754 - vespene - DEBUG - flagging build as done

To avoid this appearing to be a bug (it does show a traceback), we should use custom typed exceptions (such is in commands.py, line 132) and instead share a message that the user should see the build log in the console or API for more information.

hello,

Have you considered/though of using ansible instead of the shell scripts?

They are all free for Open Source projects like this one.

https://github.com/marketplace/category/continuous-integration

For easier sudo configuration with the sudo isolation plugin, we really want the vespene user to be in sudoers.

This is not something that the setup automation currently sets up.

We should at least add a note in the docs for this

Bug description
When importing a project, via the organisations tab in the GUI, using a .vespene I got an error when the project running and looked at the variables and saw they were stored in single quotes instead of double quotes. After changing it to double quotes the project ran fine.

Steps to reproduce

My .vespene file is the following:

name: "Tommys Second Project"
script: "scripts/example.sh"
timeout: 60
repo_branch: master
variables:
   testvar: "hehehoho"
   luggage_code: "5150"
worker_pool: 'tutorial-pool'

This builds me a project that I should then be able to run.

Actual behavior

After importing the variables in the project are as follows:

When I run the project I get this error pop up:

Traceback (most recent call last):
  File "/opt/vespene/vespene/plugins/variables/common.py", line 39, in _get_variables
    obj_variables = json.loads(obj.variables)
  File "/usr/lib64/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/vespene/vespene/views/project.py", line 141, in project_start
    start_project_from_ui(request, obj)
  File "/opt/vespene/vespene/manager/jobkick.py", line 81, in start_project_from_ui
    start_project(project, launch_answers=launch_answers, request=request)
  File "/opt/vespene/vespene/manager/jobkick.py", line 94, in start_project
    variables = compute_variable_chain(project)
  File "/opt/vespene/vespene/manager/jobkick.py", line 163, in compute_variable_chain
    return VariableManager(project).compute()
  File "/opt/vespene/vespene/common/variables.py", line 22, in compute
    computed = plugin.compute(self.project, results)
  File "/opt/vespene/vespene/plugins/variables/common.py", line 60, in compute
    variables.update(self._get_variables(project))
  File "/opt/vespene/vespene/plugins/variables/common.py", line 41, in _get_variables
    raise Exception("failed to parse JSON from object: %s" % obj.variables)
Exception: failed to parse JSON from object: {'testvar': 'hehehoho', 'luggage_code': '5150'}

I changed the single quotes to double quotes manually as so:

And the project without problems.

Operating system and version
Amazon Linux 2

Mentioned text areas are not selectable making it hard to copy-paste the build logs. Discussed here.

Workaround: use Developer tools (right-click the text area -> Inspect element) and double-click the text area's value.

It's easy to type in a repo URL for a project but leave the repo type set to None.

If we try to build a project with a repo URL filled in but the repo type is None, we should raise a clear error message, rather than just leaving an empty build directory.

This can be confusing as users will have all the info entered and wonder why their checkouts didn't happen.

Super easy fix, just needs to happen.

I'm in the process of making sure HTML escaping happens consistently and it occurs to me that it's time to convert the various routes back to django reversed view names using reverse()

If the sudo user is left blank, confusing error messages can result, that would be more clear if we didn't allow it to be blank.

I am proposing the sudo_user field on the worker pool default to "vespene_build" and be set to be non-nullable.

This value will not be used if using basic_container isolation, but will still be used for sudo isolation, making it more clear to new users that they need to change the username when trying things out for the first time.

Feature:

• Add a boolean option to project model, "webhooks_trigger_any_branch"
• In the webhook code, if the branch doesn't match, trigger that branch IF the project is so configured with "webhooks_trigger_any_branch=True"
• Also in the build view columns (UI), show the branch name of the project.
• Also in the importer, add an option, "webhooks_trigger_any_branch" so the .vespene files can control this parameter
• Document the webhooks_trigger_any_branch feature in both the webhooks docs and the import docs

This will enable CI/CD of arbitrary feature branches in Vespene without having to create a project definition for each branch.

Per forum discussion, a .vespene file should be able to define more than one project:

---
name: "Tommys Project"
script: "scripts/example.sh"
timeout: 60
repo_branch: master
variables:
   testvar: "hello!"
   luggage_code: "5150"
worker_pool: 'tutorial-pool'
---
name: "Tommys Second Project"
script: "scripts/example.sh"
timeout: 60
repo_branch: master
variables:
   testvar: "hehehoho"
   luggage_code: "5150"
worker_pool: 'tutorial-pool'

This would greatly simplify the install process. Especially just running docker-compose up -d and bootstraping a postgres container.

I have a fork that is currently being worked on, but am currently running into difficulties around the current install process. Namely, sudo and database. Nothing insurmountable, just the first roadblock of a naive attempt

Any preference as to which base image python3, debian, xenial, alpine?

In particular, there are some things to document here at least:

https://talk.msphere.io/t/setup-questions/21/34

By default the Django auth model only allows uniqueness for usernames not emails. I have a custom user model already if you want me to submit it that will allow for unique email and username and can allow to login with whichever one you want to setup as default.

let me know, thanks

In the security guide we should talk about putting the vespene_build user in the same group as the vespene user, and then we can expose "permissions_hex" on the worker pool DB object, allowing the build roots to be 770 vs wide open.

https://talk.msphere.io/t/i-cannot-get-a-build-to-work/107/7

In 1_prepare.sh you call pip install -r ./requirements.txt.

This tries to build cryptography which won't build without libssl-dev installed on Ubuntu. Installing libssl-dev and rerunning 1_prepare.sh fixes the issue.

Bug description
[vespene@centos setup]$ ./3_application.sh
detected RHEL/CentOS
/usr/bin/python3.6
sudo chown -R vespene /etc/vespene
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions, vespene
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
Applying admin.0002_logentry_remove_auto_add... OK
Applying admin.0003_logentry_add_action_flag_choices... OK
Applying contenttypes.0002_remove_content_type_name... OK
Applying auth.0002_alter_permission_name_max_length... OK
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying auth.0007_alter_validators_add_error_messages... OK
Applying auth.0008_alter_user_username_max_length... OK
Applying auth.0009_alter_user_last_name_max_length... OK
Applying sessions.0001_initial... OK
Applying vespene.0001_initial... OK
Applying vespene.0002_auto_20181027_1530... OK
Applying vespene.0003_auto_20181101_2224...Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
psycopg2.ProgrammingError: syntax error at or near "WITH ORDINALITY"
LINE 6: FROM unnest(c.conkey) WITH ORDINALITY co...
^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "manage.py", line 18, in
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.6/site-packages/django/core/management/init.py", line 381, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.6/site-packages/django/core/management/init.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 316, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 353, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 83, in wrapped
res = handle_func(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 203, in handle
fake_initial=fake_initial,
File "/usr/local/lib/python3.6/site-packages/django/db/migrations/executor.py", line 117, in migrate
state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
File "/usr/local/lib/python3.6/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
File "/usr/local/lib/python3.6/site-packages/django/db/migrations/executor.py", line 244, in apply_migration
state = migration.apply(state, schema_editor)
File "/usr/local/lib/python3.6/site-packages/django/db/migrations/migration.py", line 124, in apply
operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
File "/usr/local/lib/python3.6/site-packages/django/db/migrations/operations/fields.py", line 216, in database_forwards
schema_editor.alter_field(from_model, from_field, to_field)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/schema.py", line 523, in alter_field
old_db_params, new_db_params, strict)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/postgresql/schema.py", line 122, in _alter_field
new_db_params, strict,
File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/schema.py", line 531, in _alter_field
fk_names = self._constraint_names(model, [old_field.column], foreign_key=True)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/schema.py", line 1029, in _constraint_names
constraints = self.connection.introspection.get_constraints(cursor, model._meta.db_table)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/postgresql/introspection.py", line 158, in get_constraints
""", ["public", table_name])
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 100, in execute
return super().execute(sql, params)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 89, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: syntax error at or near "WITH ORDINALITY"
LINE 6: FROM unnest(c.conkey) WITH ORDINALITY co...
^

Steps to reproduce

1. update 0_common.sh and change 127.0.0.1 to 0.0.0.0 for GUNICORN_OPTS
2. continue executing the scripts until 3_application.sh
   Expected behavior

Actual behavior

Operating system and version
CentOS Linux release 7.5.1804 (Core)
Any additional info

http://vespene.io/ is unaccessabled and seem there's no any DNS records in this domain.

This'd be useful

I think the setup process needs to be a bit more detailed and reformatted for readability.

• Poor readability at the moment. The docs just seem like a wall of text. I can submit a PR for this if required. +1 for inline code as well.

• A link to the FAQ and more directions for troubleshooting would be great too. Using an external Postgres database, creating firewall rules to whitelist gunicorn, and so on. (DigitalOcean and Vultr have firewalls enabled in images it seems)

• Point out what's stored in these:

    /opt/vespene
    /etc/vespene
    /var/log/vespene
    /var/spool/vespene
    /tmp/vespene/
  

  I mean sure, I know Vespene's using these but what for? Where are the builds stored? Where can I keep project files? Is a .vespene file created when I create a project? Where is it?

• GUNICORN_OPTS="--bind 127.0.0.1:8000" was changed to GUNICORN_OPTS="--bind 0.0.0.0:8000" and back to GUNICORN_OPTS="--bind 127.0.0.1:8000". I was following the setup guide on docs.vespene.io so I didn't catch this and figured it was still using 0.0.0.0.

  This should go into the docs since we're asking people to run the scripts and not giving specific instructions on installing Vespene.

• Since hostnames are used for referring to build workers, one way would be to refer to the master host using it's hostname as well to avoid the 127.0.0.1 and 0.0.0.0 issue.

Edit: TIL there's also http://docs.vespene.io/development_setup.html ;_;

When I try to login with my Github account on the forum the follow happens. I tried removing authorization for the forum in my Github account settings.

6_services.sh calls

$PYTHON manage.py generate_supervisor --controller --path /etc/vespene/supervisord.conf --workers "$WORKER_CONFIG" --executable=$PYTHON

which yields
manage.py generate_supervisor: error: unrecognized arguments: --controller

and indeed https://github.com/vespene-io/vespene/blob/master/vespene/management/commands/generate_supervisor.py does not seem to support an argument named --controller ?

Consider change the way to check the operating system debian during installation

Debian is independent of Ubuntu even though the same package manager is used. (APT)

Issues during install on debian 9.5 (dist-upgrade):

1_prepare.sh
sudo apt-add-repository universe
Error: 'universe' invalid

2_database.sh
elif [[ "$DISTRO" == "ubuntu" ]]; then
sudo apt install -y postgresql postgresql-contrib
CONFIG="/etc/postgresql/10/main/pg_hba.conf"

The base repository in debian installs postgres9.6 then not exist /etc/postgresl/10/... and CONFIG fail.

tee: /etc/postgresql/10/main/pg_hba.conf: ----> No such file or directory

Is it necessary to install postgresql10 version ? if is yes should use "PostgreSQL Apt Repository"
https://www.postgresql.org/download/linux/debian/

https://talk.msphere.io/t/unsure-how-to-use-output-variables/114/2

Currently, {{ version }} isn't expanded at the login page and you get Vespene {{ version }}. (C) 2018 Michael DeHaan LLC + Contributors.

Once you login, Vespene 1.0-dev. (C) 2018 Michael DeHaan LLC + Contributors. works.

My Setup:
Distributor ID: Ubuntu
Description: Ubuntu 18.04.1 LTS
Release: 18.04
Codename: bionic

When running sudo bash 1_prepare.sh I am seeing the below after fixing the wrong repository piece in the file.

I'm debugging now but thought I'd put this here for tracking.

Installing python packages...
/usr/bin/pip3 install -r ../../requirements.txt --trusted-host pypi.org --trusted-host files.pypi.org --trusted-host files.pythonhosted.org
Traceback (most recent call last):
  File "/usr/bin/pip3", line 9, in <module>
    from pip import main
ImportError: cannot import name 'main'