owncloud-ansible / owncloud Goto Github PK

LDAP would be great to have configurable and therefore it should imho be part of ansible.

Currently when the tarball could not be downloaded successfully, the Ansible scripts leave the system behind in maintenance mode. It would be more fail-safe to check the download before putting the system in maintenance mode. Or to gracefully return the system from maintenance mode if the installation couldn't be done.

You can define a custom skeleton directory, there should be an extra variable like owncloud_create_empty_skeleton.

Additionally it would be cool if you could upload a folder from a local source or download a tar.gz from a webserver with another additional variable.

We should provide a way to setup the new (Phoenix frontend)[https://github.com/owncloud/phoenix].

The way I see it this should be a different role, as this can also be run on a different machine.

There are at least three ways to deploy Phoenix:

Install as an app

• enable Phoenix as an app
• some config tweaks needed, see below (to be confirmed)

Install on the same host

• have the web server serve the Phoenix files (JS bundle + config.json)
• might require config tweaks, see below

Install Phoenix on a separate host

• setup ownCloud with oauth2 app enabled
• adjust config.php settings, especially CORS (see script below)
• Phoenix needs a web server to serve the main JS bundle and config.json, there are no server side logic.
  • should we use Apache there ?
  • or use ocis-phoenix directly as go binary?

Here is my local script to automatically setup OC for Phoenix development:

#!/bin/bash

OCDIR="/srv/www/htdocs/owncloud"
DATADIR="$OCDIR/data"
OCC="sudo -uwwwrun ./occ"

HOSTNAME=phoenixhost.local
PHOENIX_HOSTNAME=$HOSTNAME:8300

PHOENIX_CLIENTID=***REMOVED***
PHOENIX_CLIENTSECRET=***REMOVED***

cd "$OCDIR"

echo Setting up ownCloud for host name "$HOSTNAME" to connect with Phoenix on "${PHOENIX_HOSTNAME}"

$OCC config:system:set overwrite.cli.url --value="['http://$HOSTNAME/owncloud']"

$OCC config:system:set trusted_domains --type=json --value="[\"${HOSTNAME}\"]"
$OCC config:system:set cors.allowed-domains --type=json --value="[\"http://${PHOENIX_HOSTNAME}\", \"http://localhost:9876\"]"
$OCC config:system:set dav.enable.tech_preview --type=boolean --value=true
$OCC config:system:set phoenix.baseUrl --type=string --value="http://${PHOENIX_HOSTNAME}"

$OCC app:enable oauth2

$OCC oauth2:add-client "Phoenix" "${PHOENIX_CLIENTID}" "${PHOENIX_CLIENTSECRET}" "http://${PHOENIX_HOSTNAME}/oidc-callback.html" 

echo Please copy the OAuth2 client id into Phoenix\'s config.json: "$PHOENIX_CLIENTID"

To get an oauth2 secret, I had to first do the oauth2 pairing manually, then I looked up the client id and secret in the database to reuse in this script.

On the Phoenix side, config.json:

{
  "server" : "http://phoenixhost.local/owncloud/",
  "theme": "owncloud",
  "version": "0.1.0",
  "auth": {
    "clientId": "***REMOVED**",
    "url": "http://phoenixhost.local/owncloud/index.php/apps/oauth2/api/v1/token",
    "authUrl": "http://phoenixhost.local/owncloud/index.php/apps/oauth2/authorize"
  },
  "apps" : ["files"]
}

Connecting to OCIS

• ability to connect Phoenix to an ocis-backend instead of regular ownCloud

Other

• Optionally/Later: Phoenix should also be deployable with OpenIDConnect (I have no experience there)

3rd party apps need to be disabled before upgrade. This affects also deprecated apps if the marketplace is not available.

The json object that is created as parameter for the json_decode call in user.config.php is invalid and so the function return null.
Reproducible on RHEL8 and Ubuntu 20.04

In a test environment it does not become visible unless you test if the values are really overwriting config.php because the issue does not break the script.

Reproduction can be done by adding a line with print $CONFIG or vardump($CONFIG) which both return NULL when executed via php user.config.php

Research so far:

PHP Docs for json_decode()

Note:
PHP implements a superset of JSON as specified in the original » RFC 7159.

so i tested an object from my user.config.php file in an online validator

There are more RFC's for Json and the validator has to match to 7159 otherwise the validators show it as valid but json_decode() still returns NULL.

For RFC 8259

For RFC 7159

https://jsonformatter.curiousconcept.com/

Finally, the json object that was created as parameter from this role

<?php
/* Ansible managed */
 
$CONFIG = json_decode('{
    "accounts.enable_medial_search": true,
    "allow_user_to_change_display_name": true,
    "apps_paths": [
        {
            "path": "/var/www/owncloud/apps",
            "url": "/apps",
            "writable": false
        }
    ],
    "cron_log": true,
    "csrf.disabled": false,
    "default_language": "de_DE",
    "defaultapp": "files",
    "enable_avatars": true,
    "integrity.ignore.missing.app.signature": [
        "theme-example"
    ],
    "knowledgebaseenabled": true,
    "log.syslog.format": "[%reqId%][%remoteAddr%][%user%][%app%][%method%][%url%] %message%",
    "log_rotate_size": 0,
    "log_type": "owncloud",
    "logdateformat": "Y-m-d H:i:s.u",
    "logfile": "/var/log/owncloud.yml",
    "loglevel": 2,
    "logtimezone": "Etc/UTC",
    "memcache.distributed": "\\OC\\Memcache\\Redis",
    "memcache.local": "\\OC\\Memcache\\APCu",
    "memcache.locking": "\\OC\\Memcache\\Redis",
    "overwrite.cli.url": "loalhost",
    "remember_login_cookie_lifetime": 1296000,
    "session_keepalive": true,
    "session_lifetime": 86400,
    "show_server_hostname": false,
    "skeletondirectory": "/var/www/owncloud/core/skeleton",
    "syslog_tag": "ownCloud",
    "token_auth_enforced": false,
    "trusted_domains": [
        "localhost",
    ],
    "user.search_min_length": 3,
    "version.hide": true
}', true);

Finally I could limit the issue to the following single failure

Invalid character found at position 3. [Code 180, Structure 0]

Stackoverflow is giving some inputs on that.

As a result, when the playbook executes occ install ... the values from user.config.php are not written into config.php

Should be possible to add a custom skeleton path using ansible:
https://github.com/owncloud/core/blob/master/config/config.sample.php#L260-L264

eg.

'operation.mode' => 'clustered-instance', (only after completed install on primary node) .

thanks @enbrnz

Use Case:
I have a fuctioning LDAP configuration and want to migrate my data and users.

For that i would like to be able to deploy e.g. a ldap.config.php along the config.php and user.config.php

When overwriting owncloud_download_url with an absolute path, e.g. a RC, the update is not performed as the criteria is based on owncloud_version value.

The need to change the version is unintuitive and automatic detection of the version based on the sources would be more convenient.

Already two times it happened to me that when running setup.yml with an empty machine, it would install everything and at the end ownCloud would say that the "APCu is not present".
After manually restarting Apache the module was found.

This might be some kind of race condition as it doesn't happen every time.

To fix it, we should probably add an Apache restart at the end, or at least after the PHP steps.

I noticed that this is missing

We should configure ownCloud 10.3 to use Redis session locking by default

It seems that the current way of setting occ config is not efficient as it calls occ config:* for every single key. If technically possible within Ansible, it would be possible to do a bigger diff of settings.

Settings can be exported using occ config:list as json for the facts. Then it could be diffed against a JSON representing the keys in question, but limit the diff to the "managed keys".
Then import the json using occ config:import.

This is based on my very limited knowledge of Ansible and am not sure if this fits best practices. :-)

Additionally to occ system:cron, there are other jobs that are recommended to be added as separate system cron entries:

See https://doc.owncloud.com/server/10.1/admin_manual/configuration/server/background_jobs_configuration.html#available-background-jobs

These occ commands require extra cron jobs in the system crontab, where the interval could be different. Maybe we could add extra variables for each of these jobs and a configurable interval ?

Without deeper knowlegde ansible has some pitfalls, so a beginners guide would be great

The documentation should include an example that shows how the expected array should be structured.

As I'm not familiar yet with this syntax, it is difficult for me to infer from the code:

If the S3 configuration is given in the owncloud_config_extra section like this:

owncloud_config_extra:
  - objectstore:
      class: 'OCA\\Files_Primary_S3\\S3Storage'
      arguments:
        bucket: "owncloud"
        part_size: "5242880"
        concurrency: "5"
        options:
          version: "2006-03-01"
          region: "us-east-1"
          credentials:
            key: "username"
            secret: "password"
          use_path_style_endpoint: true
          endpoint: "http:\/\/hostname:8000"

Ansible aborts the execution due to a class not found exception in occ check. A separate execution of occ check looks like this:

occ check
An unhandled exception has been thrown:
Error: Class 'OCA\Files_Primary_S3\S3Storage' not found in /var/www/owncloud/lib/private/legacy/util.php:115
Stack trace:

The reason is that the apps are installed

When trying to install owncloud on multiple nodes within one play only the first node is installed correctly.
i.e., using a play such as this where owncloud is a group containing multiple hosts

- name: Install owncloud and required services
  hosts: owncloud
  become: true
  roles:
    - owncloud-mariadb
    - owncloud-redis
    - owncloud-apache
    - owncloud-php
    - owncloud

The reason for this seems to be the run_once set for the following task using the occ tool to finish installation:

- name: Automatically finish setup via the occ tool
  register: __owncloud_register_occ_install
  when: owncloud_do_autosetup | bool
  command: |
    {{ owncloud_occ_executable | quote }} maintenance:install
    "--data-dir={{ owncloud_data_path }}"
    "--database={{ owncloud_db_type }}"
    "--database-host={{ owncloud_db_host }}"
    {% if owncloud_db_type in ['mysql', 'pgsql'] %}
    "--database-name={{ owncloud_db_name }}"
    "--database-user={{ owncloud_db_user }}"
    "--database-pass={{ owncloud_db_password }}"
    "--database-table-prefix={{ owncloud_db_tableprefix }}"
    {% endif %}
    {% if owncloud_admin_username %}
    "--admin-user={{ owncloud_admin_username }}"
    "--admin-pass={{ owncloud_admin_password }}"
    {% endif %}
  run_once: True
  become: True
  become_user: "{{ owncloud_app_user }}"

From what I can see in the issues of this repo I believe you added this flag in case of cluster setups where this step only has to run on the main node (#9 ). Now sadly this decision completely breaks the usecase of installing unrelated instances using the same play as shown in the above example play.

My suggestion to support both is to replace the static True with a variable e.g., run_once: {{ owncloud_cluster_install }}
This way the feature can be turned on/off.

Always good to start with an updates system when installing something new, so I would add

apt update optionally apt upgrade to the top of the commands list on
https://owncloud-ansible.github.io/getting_started/setup/#general

I ran in to the same issue that @jnweiger ran in to, the wheel issue. Had to search Chat to find out the solution.
Imho needs to be documented that you need:

pip3 install wheel ansible instead of just pip3 install ansible

It seems like the occ script checking the $USER variable is not compatible with cron:
/usr/local/bin/occ: line 10: USER: unbound variable.

Reasons:

• caused by set -o nounset
• $USER not set in cron environments

When installing ownCloud on several servers for one instance,

we can save time by not running the full owncloud playbook against the replica nodes.

One full blown installation should suffice, copying to the new instances should be quicker and less confusing.

thanks @enbrnz

The ownCloud role sets a bunch of sharing options which are not set in a manual installation.
We should show case how to change them in the playground or docs instead.

App installation of password_policy, totp and encryption is not really the right place in the owncloud role defaults, this should also move to playground/docs.

Currently the URL of a provisioned instance always shows "index.php" in the URL.
There's an option to make it cleaner.

To make this work, I had to manually do the following on the provisioned instance:

• edit config.php and add a key 'htaccess.RewriteBase' => '/'
• run sudo -u www-data ./occ maintenance:update:htaccess

TODO: create concept for a clustered installation.

Certain tasks need to run only once (as they run an occ command), others need to apply to all config.php on all instances of the cluster, these need to be identified.

[root@host ~]# echo $PATH
/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin

But wrapper script is installed in /usr/local/bin/occ