Code Monkey home page Code Monkey logo

appliance's People

Contributors

edamrose avatar jnweiger avatar ngulden avatar stevehobs1 avatar voroyam avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

appliance's Issues

Update of apps causes trouble

Right now if you update user_ldap you can not connect with sync client.

Error:
image

Error in the logs:

{
  "reqId": "049fa3fe-eae0-4495-ac3e-7d81fa9c1a73",
  "level": 4,
  "time": "2019-11-27T20:45:57+00:00",
  "remoteAddr": "172.17.42.1",
  "user": "--",
  "app": "webdav",
  "method": "PROPFIND",
  "url": "/owncloud/remote.php/webdav/",
  "message": "Exception: HTTP/1.1 503 Error: 
Using $this when not in object 
context: {\"Exception\":\"Sabre\\\\DAV\\\\Exception\\\\ServiceUnavailable\",\"Message\":\
"Error: Using $this when not in object context\",\"Code\":0,\"Trace\":\"
#0 [internal function]: {closure}(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\
n#1 \\/var\\/www\\/owncloud\\/lib\\/composer\\/sabre\\/event\\/lib\\/EventEmitterTrait.php(105): call_user_func_array(Object(Closure), Array)\\
n#2 \\/var\\/www\\/owncloud\\/lib\\/composer\\/sabre\\/dav\\/lib\\/DAV\\/Server.php(466): Sabre\\\\Event\\\\EventEmitter->emit('beforeMethod', Array)\\
n#3 \\/var\\/www\\/owncloud\\/lib\\/composer\\/sabre\\/dav\\/lib\\/DAV\\/Server.php(254): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\
n#4 \\/var\\/www\\/owncloud\\/remote.php(70): Sabre\\\\DAV\\\\Server->exec()\\
n#5 \\/var\\/www\\/owncloud\\/remote.php(169): handleException(Object(Error))\\
n#6 {main}\",\"File\":\"\\/var\\/www\\/owncloud\\/remote.php\",\"Line\":68}"
}
{
  "reqId": "21b7bf88-c404-46c7-9867-8c3a7ccb2244",
  "level": 3,
  "time": "2019-11-27T20:46:06+00:00",
  "remoteAddr": "172.17.42.1",
  "user": "--",
  "app": "PHP",
  "method": "PROPFIND",
  "url": "/owncloud/remote.php/webdav/",
  "message": "Non-static method OCA\\User_LDAP\\Helper::loginName2UserName() should 
   not be called statically at /var/www/owncloud/lib/private/legacy/hook.php#105"
}

full logfile:

https://cloud.owncloud.com/index.php/f/5033985


ownCloud Version 10.2.1

ownCloud App version 10.2.1-1

User_LDAP Version on installation 0.13.0
updated version 0.14.0


Probable cause of the issue: the app user_ldap is in /var/www/owncloud/apps

This directory is read only.

When you update the app via occ or the market app in the webUI the new user_ldap app will be loaded in /var/www/owncloud/custom but the app in /var/www/owncloud/apps will be still there. So you have 2 user_ldap in different directories with 2 different versions.

  'apps_paths' =>
  array (
    0 =>
    array (
      'path' => '/var/www/owncloud/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 =>
    array (
      'path' => '/var/www/owncloud/custom',
      'url' => '/custom',
      'writable' => true,
    ),
  ),

Tested our official docker images: Same error.

root@5e7e9b59e205: /var/www/owncloud # cat apps/user_ldap/appinfo/info.xml | grep version
<?xml version="1.0"?>
        <version>0.13.0</version>
                <owncloud min-version="10.0.10" max-version="10" />

root@5e7e9b59e205: /var/www/owncloud # cat custom/user_ldap/appinfo/info.xml | grep version
<?xml version="1.0"?>
        <version>0.14.0</version>
                <owncloud min-version="10.2" max-version="10" />

Same behaviour in 10.3.1

"index.php-less URLs" don't work with the app

Owncloud provides docs on how to get rid of the "index.php" in the URL:
https://doc.owncloud.org/server/10.0/admin_manual/configuration/server/index_php_less_urls.html

Unfortunately this does not work with the app.

root@owncl-66241678: /var/www/owncloud # grep AllowOverride
/etc/apache2/sites-enabled/000-default.conf
    AllowOverride All
      AllowOverride All

root@owncl-66241678: /var/www/owncloud # a2enmod php7.0
Considering conflict php5 for php7.0:
Module php7.0 already enabled

root@owncl-66241678: /var/www/owncloud # a2enmod env
Module env already enabled

root@owncl-66241678: /var/www/owncloud # a2enmod rewrite
Module rewrite already enabled

root@owncl-66241678: /var/www/owncloud # grep -E
'(overwrite.cli.url|htaccess.RewriteBase)' config/config.php

  'overwrite.cli.url' => 'http://localhost/',
  'htaccess.RewriteBase' => '/',

As you can see all requirements specified in the docs are fulfilled. I didn't need to change the Apache site and the Apache modules were already enabled.
The "update htaccess" command doesn't output any errors, even with -vvv.
Nothing is written to owncloud.log either.

root@owncl-66241678: /var/www/owncloud # ls -l .htaccess
-rw-r--r-- 1 www-data www-data 4019 Aug 20 22:05 .htaccess

root@owncl-66241678: /var/www/owncloud # occ -vvv
maintenance:update:htaccess
.htaccess has been updated

root@owncl-66241678: /var/www/owncloud # ls -l .htaccess
-rw-r--r-- 1 www-data www-data 4019 Aug 20 22:06 .htaccess

As you can see the .htaccess is modifed or rather "touched", no changes are actually being made.

Accessing /owncloud/login results in 404 from the Apache inside the container. It's a regular "Not found" error page, no owncloud branding whatsoever. /owncloud/index.php/login works.
The article states that, starting with Owncloud 10, this should actually work out of the box and index.php-less URLs should be default

online office owncloud apps missing in ownCloud Docker image

I just tested a new build of the ONLYOFFICE + ownCloud app appliance and found out that the onlyoffice app is not pre-installed. This led to the problem that the onlyoffice app could not be enabled by the ownCloud join script.

After starting and activating the appliance, I found the following lines in /var/log/univention/join.log:

[03.JOIN] check for installation of ONLYOFFICE
onlyoffice not found
Config value DocumentServerUrl for app onlyoffice set to https://master.ucs.demo/onlyoffice-documentserver

Running the following commands, I conclude that the onlyoffice app is not available in the first place:

$ univention-app shell owncloud occ app:enable onlyoffice
onlyoffice not found
$ univention-app shell owncloud occ market:install onlyoffice
onlyoffice: Installing new app ...
onlyoffice: App installed.
$ univention-app shell owncloud occ app:enable onlyoffice
onlyoffice enabled

The same is true for the collabora owncloud app.

Please make sure that both apps are part of the ownCloud Docker image so that the integrations with the online office solutions Collabora and ONLYOFFICE are still working. They assume that those ownCloud apps are pre-installed in the Docker image. One reason is the feature that it should be possible for prospects to start app appliances without Internet connection. The online office appliances with ownCloud are affected by this problem.

Environment:

$ univention-app info
UCS: 4.4-1 errata241
Installed: onlyoffice-ds=5.3.0.243 4.3/owncloud=10.2.1

smb.conf missing in ownCloud docker container

Since we've started using the sernet-samba-client package instead of smbclient inside the docker container on the appliance, smbclient execution fails because of missing smb.conf.
This is because the sernet-samba-client package is not a drop in replacement for the ubuntu smbclient package. The sernet-samba-client package doesn't do any postinstallation steps where the smbclient package creates a standard smb.conf in /etc/samba/.

Apache config is broken

Me and @spaceone were recently trying to get univention-owncloud-saml to work with the current owncloud app on UCS 4.3 due to the request of a UCS and Owncloud enterprise subscription customer.

Configuring SAML requires modifications to the Apache config inside the container. Unfortunately we were unable to successfully restart the Apache (using apache2ctl to not stop the container) because the config seems to be broken:

$ apache2ctl restart
[Thu Mar 28 17:00:59.932073 2019] [core:warn] [pid 933] AH00111: Config variable ${APACHE_LOCK_DIR} is not defined
[Thu Mar 28 17:00:59.932375 2019] [core:warn] [pid 933] AH00111: Config variable ${APACHE_RUN_DIR} is not defined
apache2: Syntax error on line 4 of /etc/apache2/apache2.conf: DefaultRuntimeDir must be a valid directory, absolute or relative to ServerRoot
Action 'restart' failed.
The Apache error log may have more information.

These messages can be obtained on a freshly installed UCS with Owncloud, so it seems that this is currently broken in all installations starting from a certain version. I believe there were a lot of adjustments being made to Apache in the container in one of the recent updates?

Docker Compose

I tried to make the docker compose setup in the appliance but failed.

This is for documenting the process of docker compose integration in the appliance.

Running command: docker-compose -p owncloud pull
The OWNCLOUD_DOMAIN variable is not set. Defaulting to a blank string.
The ADMIN_USERNAME variable is not set. Defaulting to a blank string.
The ADMIN_PASSWORD variable is not set. Defaulting to a blank string.
The OWNCLOUD_VERSION variable is not set. Defaulting to a blank string.
The HTTP_PORT variable is not set. Defaulting to a blank string.
The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for volumes: 'files'
Unsupported config option for services: 'owncloud'
Command docker-compose -p owncloud pull failed with: The OWNCLOUD_DOMAIN variable is not set. Defaulting to a blank string.

Cleanup: Unify app logging

raised by @edamrose

The app integration currently creates files and logs to several different places right now. This makes debugging the installation/updates hard(er). It also clutters the UCS system.

Currently the following additional files are used by the app integration. One cannot see that these belongs to owncloud.
/root/setup-ldap.sh -> Script should be relocated to /var/lib/univention-appcenter/apps/owncloud/
/var/log/appcenter-install.log (inside and outside the container) -> To simplify debugging, the output of app scripts should not be redirected. This way, it will appear in the respective logfiles (stdout, /var/log/univention/management-console-module-appcenter.log, ...) If the output should be redirected to an additional file, it can be redirected with e.g. the tee command.

I would like to see improvements by using the mechanisms available in UCS, and keep any additional logging that is required in the app specific directory /var/lib/univention-appcenter/apps/owncloud/.


@voroyam:

@edamrose where should we change the logging from appcenter-install.log?

or would you prefer we just remove the 2>&1
from this line

/var/log/appcenter-install.log 2>&1


@edamrose:

As suggested in the initial request, somewhere below /var/lib/univention-appcenter/apps/owncloud/... There is already an owncloud.log, so maybe log into
/var/lib/univention-appcenter/apps/owncloud/data/files/owncloud-appcenter.log

so, instead of logging like this: >>/var/log/appcenter-install.log 2>&1
Log with 2>&1 | tee --append /var/lib/univention-appcenter/apps/owncloud/data/files/owncloud-appcenter.log

Upgrade Testing

10.2.1-1 upgrade to 10.3.0

OSError: [Errno 2] No such file or directory: '/var/lib/docker/overlay2/b5bf81890a930cc2a8ab5341148e6d5121fa07cae46296a20972f0624866e855/merged/var/univention/tmp/tmpFqJ2Fh'

/var/lib/docker/overlay2/b5bf81890a930cc2a8ab5341148e6d5121fa07cae46296a20972f0624866e855# ls
diff  link  lower  work

Full Stacktrace:

Executing interface restore_data_before_setup for owncloud
No interface defined
Executing interface setup for owncloud
Copying App Center's setup to container's /usr/share/univention-docker-container-mode/setup_custom
[02.DOCKER_SETUP] Enable user_ldap app
.
[02.DOCKER_SETUP] user_ldap enabled successfully! after 1 tries
[02.DOCKER_SETUP] Waiting for LDAP app testing...
.
Executing interface update_available for owncloud
No interface defined
[Errno 2] No such file or directory: '/var/lib/docker/overlay2/b5bf81890a930cc2a8ab5341148e6d5121fa07cae46296a20972f0624866e855/merged/var/univention/tmp/tmpFqJ2Fh'
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/__init__.py", line 226, in call_with_namespace
    result = self.main(namespace)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/upgrade.py", line 81, in main
    return self.do_it(args)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/install_base.py", line 119, in do_it
    self._do_it(app, args)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/docker_upgrade.py", line 126, in _do_it
    self._upgrade_image(app, args)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/docker_upgrade.py", line 182, in _upgrade_image
    self._install_new_app(app, args)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/docker_upgrade.py", line 69, in _install_new_app
    return Install._do_it(self, app, args)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/docker_install.py", line 71, in _do_it
    ret = super(Install, self)._do_it(app, args)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/install.py", line 111, in _do_it
    if self._install_app(app, args):
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/docker_install.py", line 67, in _install_app
    self._setup_docker_image(app, args)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/docker_install.py", line 86, in _setup_docker_image
    process = self._execute_container_script(app, 'setup', args, cmd_kwargs={'_logger': logger})
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/docker_base.py", line 139, in _execute_container_script
    return process
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/docker.py", line 283, in tmp_file
    tmp_file.close()
  File "/usr/lib/python2.7/tempfile.py", line 431, in close
    self.unlink(self.name)
OSError: [Errno 2] No such file or directory: '/var/lib/docker/overlay2/b5bf81890a930cc2a8ab5341148e6d5121fa07cae46296a20972f0624866e855/merged/var/univention/tmp/tmpFqJ2Fh'
Traceback (most recent call last):
  File "/usr/bin/univention-app", line 91, in <module>
    main()
  File "/usr/bin/univention-app", line 78, in main
    ret = args.func(args)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/__init__.py", line 226, in call_with_namespace
    result = self.main(namespace)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/upgrade.py", line 81, in main
    return self.do_it(args)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/install_base.py", line 119, in do_it
    self._do_it(app, args)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/docker_upgrade.py", line 126, in _do_it
    self._upgrade_image(app, args)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/docker_upgrade.py", line 182, in _upgrade_image
    self._install_new_app(app, args)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/docker_upgrade.py", line 69, in _install_new_app
    return Install._do_it(self, app, args)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/docker_install.py", line 71, in _do_it
    ret = super(Install, self)._do_it(app, args)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/install.py", line 111, in _do_it
    if self._install_app(app, args):
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/docker_install.py", line 67, in _install_app
    self._setup_docker_image(app, args)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/docker_install.py", line 86, in _setup_docker_image
    process = self._execute_container_script(app, 'setup', args, cmd_kwargs={'_logger': logger})
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/actions/docker_base.py", line 139, in _execute_container_script
    return process
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/lib/python2.7/dist-packages/univention/appcenter/docker.py", line 283, in tmp_file
    tmp_file.close()
  File "/usr/lib/python2.7/tempfile.py", line 431, in close
    self.unlink(self.name)
OSError: [Errno 2] No such file or directory: '/var/lib/docker/overlay2/b5bf81890a930cc2a8ab5341148e6d5121fa07cae46296a20972f0624866e855/merged/var/univention/tmp/tmpFqJ2Fh'

@edamrose any idea what the error is comming from?

The settings script maybe?

Make branding customizable via App Settings

This adds to #37 but I'm creating this issue to maintain a better overview over the separate requests.

It would be nice if the Owncloud branding was customizable via App Settings.

Specifically defaultEntity, defaultTitle, defaultBaseUrl and defaultSlogan in /var/www/owncloud/lib/private/legacy/defaults.php

In a customer's environment we currently do the following to apply a branding:

docker cp background.jpg ID:/var/www/owncloud/core/img/background.jpg
univention-app shell owncloud sed -i "s/defaultEntity\ \=\ 'ownCloud'/defaultEntity\ \=\ 'CUSTOMER'/g" /var/www/owncloud/lib/private/legacy/defaults.php
univention-app shell owncloud sed -i "s/defaultTitle\ \=\ 'ownCloud'/defaultTitle\ \=\ 'CUSTOMER'/g" /var/www/owncloud/lib/private/legacy/defaults.php
univention-app shell owncloud sed -i "s/defaultBaseUrl\ \=\ 'https:\/\/owncloud.org'/defaultBaseUrl\ \=\ 'https:\/\/www.CUSTOMER.TLD'/g" /var/www/owncloud/lib/private/legacy/defaults.php
univention-app shell owncloud sed -i "s/'A safe home for all your data'/'CUSTOMER SLOGAN'/g" /var/www/owncloud/lib/private/legacy/defaults.php

10.1.1 Release Checklist

  • Database:

  • Works on new install

  • Works on update from earlier versions (10.0.4)

  • App Settings / Variable Setting

  • Is OWNCLOUD_DOMAIN set correctly?

  • Does the enterprise+Market API Key works via app settings?

Automatic restore

After install, remove, and install again, db and config should be restored.
Could be done in join script.

echo "[01.PRE_INST] database check"
mysql -uroot -p$(cat /etc/mysql.secret) -e "show databases";
# [ -f ${OWNCLOUD_BACKUP_DIR}/database.sql ] && \
#   echo "[01.PRE_INST] Backup found" && \
#   mv ${OWNCLOUD_BACKUP_DIR}/conf ${OWNCLOUD_PERM_DIR}/conf && \
#   mysql \
#     -uroot \
#     -p${cat /etc/mysql.secret} \
#     owncloud < ${OWNCLOUD_BACKUP_DIR}/database.sql

How to enable Rich Documents app via call

The ownCloud image comes with the Rich Documents App pre-installed in ownCloud marketplace. The app is deactivated by default.

What call needs to be executed to activate this app from outside the container, let's say the UCS host or another container? If it is not possible from outside the container, please let me know what has to be executed in the container.

Idee: When Collabora is installed on UCS, it should check for the ownCloud app in one of the app scripts and then execute the above mentioned call to activate the Rich Documents app.

This will prevent manual action needed by the admin user to enable office functionality within ownCloud.

It would be nice to have this information within the next few days so that I can continue on the Collabora App Appliance.

Users are only available for sharing after their first login

Given is a UCS system with ownCloud installed.

  1. Add user A and B within UCS.
  2. Login to ownCloud with user A. Upload a file.
  3. Share the file with user B.

Sharing with user B is not possible at this point, because user B has not yet logged in to ownCloud. To enable direct sharing, the user needs to login at least once first.

This is a usability draw back. It would be better, if the newly created user is available in ownCloud almost immediatly to enable interaction with other ownCloud users that have not yet logged in before.

App settings are not read, fix configure script chmod call

While porting the apps configure script to python3 the chmod call after writing the /etc/entrypoint.d/05-univention-env.sh was not adapted. This results in incorrect file permissions. This could on reason why the App Settings appear to not be applied.

Fix the chmod call in the configure script, the line has to be
os.chmod(local_container_env_file, 0o755)

Backup strategy

@patrickjahns wrote:

@voroyam

As discussed today:

I find it very appealing if we can provide user with appropriate messages informing of what is happening.

To summarize what we also discussed today:

before uninstalling - we notify the user that a backup will be created (would it be possible to have an opt-out button here as well?)
before updating - we notify the user that a backup will be created, but if possible we can provide a opt-out button so a user might skip this

We also would like to provide more documentation on:

where the backup would be found
how a backup can be removed (when users are concerned of their storage)

A very interesting article would also be to talk about "disaster recovery" - what to do when an upgrade has failed.

(these things could also be a series of blog posts ? )

cc @felixboehm


@ngulden wrote:

@patrickjahns , @voroyam

The "notification" before uninstallation is currently just a presentation of static text. There is currently no possibility for an opt-out like suggested above.

For the start I'd suggest that:

The app takes care of the backup and stores it at a "default" location".
The respective README files of the app inform the user about the backup and where he can find it.

We should then look at the feedback and see if such an opt-out is really needed.

FYI: We are currently changing the App Center behavior in a way that uninstallation will not copy the persistent data folders to a separate location on UCS. It will just stay there and will not be touched. It can therefore be used after a reinstallation. See UCS Bug #45670 for details..

non upgradeable marketplace apps cause univention app update failure

A customer tried to update the owncloud app via the App Center. The update failed and the new container did not start. After a few workarounds the owncloud-appliance container started but stopped immediately. The log showed something like (sorry, could not copy the exact log entries)

Cannot update with the following enterprise apps enabled, disable with occ:
theme-enterprise
files_ldap_home

But one cannot use the occ command without a running container. So the apps were manually disabled in the MySQL database, and the container started.

I think the App scripts should either prevent the update if there are owncloud apps that block an update, or deactivate such apps before the update.
The current user experience is very disappointing, especially because customers with enterprise apps are affected.

missing documentation about firewall prerequisites

The installation of the appliance does not finish when there is no internet connection. The same applies, when the outgoing connections are firewalled.

A lot of customers have rather strict security policies in place and do not want open their infrastructure more than necessary.

So, the request is that the documentation states in clear fashion which ports should be opened and which domains MUST be reachable for the installation to finish without errors.

Make overwrite.cli.url configurable

The owncloud app automatically resets the overwrite.cli.url config parameter in /var/www/owncloud/config/config.php whenever a restart or upgrade happens.

This happens, like #33 and #34, due to a script, that's being executed when the container starts:

/etc/owncloud.d/45-cli.sh:4:occ config:system:set overwrite.cli.url --value http://${OWNCLOUD_DOMAIN}${OWNCLOUD_SUB_URL}

As with #33 and #34 it would be nice to have an App setting for the ownCloud app so administrators can set this value via the Management Console.

Feedback: Enable Webdav

Univention received the following Feedback for the ownCloud Appliance:

The OwnCloud appliance also fails to start webdav correctly out of the box which isnt impressive for a pre-configured appliance

I have no more information about the tester and what he tried.

Add warning to ad join install option

It would reduce the amount of critical cases we get if we just ad a warning to this install option as it requires deep knowledge and the appliance is sold to customer without deep knowledge.

10.0.7 Checklist/Testplan

  • Upgrade to new version from old version
  • Log in ownCloud via IP and HTTP
  • * https & IP
  • * https & domain name
  • Upload a file
  • Overwrite a file by uploading a new version
  • Enable OnlyOffice in ownCloud
  • Install & Configure the OnlyOffice App
  • Create new document in ownCloud
  • Enable Collabora in ownCloud
  • Install & Configure the Collabora App
  • Rename a file
  • Create a folder
  • Move a file inside a folder
  • Move a folder inside another
  • Remove a file
  • Delete a folder
  • Create a new local user
  • Create new user and group in UCS
  • Sync LDAP users in ownCloud
  • Check the users tab for the LDAP users
  • Share a file by link
  • Share a file with a local group
  • Login as local user in that group and check
  • Share a file with a LDAP group
  • Login as LDAP user in that group and check
  • Activate the enterprise trial

added tests from @CaptionF feedback

  • WND: mount storage
  • Email-Configuration
  • Guest app: Invite guest to share

  • All tests completed

Make htaccess.RewriteBase configureable

The owncloud app automatically resets the htaccess.RewriteBase config parameter in /var/www/owncloud/config/config.php whenever a restart or upgrade happens.

This happens, like #33, due to a script, that's being executed when the container starts:

/etc/owncloud.d/90-services.sh:4:occ config:system:set htaccess.RewriteBase --value ${OWNCLOUD_SUB_URL}

As with #33 it would be nice to have an App setting for the ownCloud app so administrators can set this value via the Management Console.

How install onlyoffice app via occ command

Is there a possibility to install onlyoffice marketplace app via a command to the ownCloud container?

I can install the app manually over the ownCloud admin interface. I found out how to configure it with occ, for example:

univention-app shell owncloud occ config:app:set onlyoffice DocumentServerUrl --value="https://master.ucs.demo/onlyoffice-documentserver"

How can I install the onlyoffice app without going the manual way as described in https://doc.owncloud.com/server/10.0/admin_manual/installation/apps_management_installation.html#manually-installing-apps

Update notification in owncloud

With the current owncloud app 10.2.1-1 i am seeing a yellow popup in the owncloud webinterface with a nofitication about the latest owncloud version 10.3.0, it links to [1]. I thought update notifications were disabled inside the Univention owncloud app, because Administrators have to update through the App Center. The link points to the general owncloud documentation, which will not work with the App Center version.

How can the notification be disabled again, or was this a conscious change?

[1] https://doc.owncloud.org/server/10.2/admin_manual/maintenance/upgrade.html

Add strong warning about migrating user data across mountpoints when updating to owncloud app

A large customer wanted to migrate from the owncloud82 to the owncloud app. The migration script apparently simply moves the files from the data directory to the new docker app directory:

mv /var/lib/owncloud/* $OWNCLOUD_DATA/files

This is very risky! The customer of course had a separate mount for owncloud user data:

/dev/sdb1        32G    8,0G   22G   27% /var
/dev/sdd1        32T    954G   30T    4% /var/lib/owncloud

During the migration the harddisk sdb1 mounted on /var was filled and several services stopped operation. A cleanup session was necessary. Partly 'moved' files had to be moved back.

Please add a strong warning to the migration guide, what happens during the migration and what customer have to check before starting the migration.

https://doc.owncloud.com/server/latest/admin_manual/appliance/howto-update-owncloud.html#upgrading-from-versions-prior-to-10-0

10.0.8 Testing

Login and Upgrade

  • Upgrade to new version from old 9.1 Version
  • Log in ownCloud via IP and HTTP
  • * https & IP
  • * https & domain name

File Operations

  • Upload a file
  • Overwrite a file by uploading a new version
  • Rename a file
  • Create a folder
  • Move a file inside a folder
  • Move a folder inside another
  • Remove a file
  • Delete a folder

Office

  • Enable OnlyOffice in ownCloud
  • Install & Configure the OnlyOffice App
  • Create new document in ownCloud
  • Enable Collabora in ownCloud
  • Install & Configure the Collabora App

User Management

  • Create a new local user
  • Create new user and group in UCS
  • Sync LDAP users in ownCloud
  • Check the users tab for the LDAP users

Sharing

  • Share a file by link
  • Share a file with a local group
  • Login as local user in that group and check
  • Share a file with a LDAP group
  • Login as LDAP user in that group and check

Enterprise

  • Activate the enterprise trial
  • WND: mount storage
  • Email-Configuration
  • Guest app: Invite guest to share

  • All tests completed

Make ownCloud reachable at a custom setting

In addition to previous changes the apache config both inside and outside the container would have to be changed as well. In the customer's prototype we currently do the following to reach Owncloud at /

Inside the container

univention-app shell owncloud sed -i '8,12s/^/#/' /etc/apache2/sites-enabled/000-default.conf
univention-app shell owncloud sed -i '39,43s/^/#/' /etc/apache2/sites-enabled/000-default.conf

The lines that are being changed to comments look as follows, one block for the VirtualHost at :80 and one block for the VirtualHost at :443. If the customer intends to make Owncloud reachable somewhere at /cloud or something else than /, those would have to be modified and not commented out of course.

#  RewriteEngine On
# RewriteCond %{REQUEST_URI} !^/owncloud
# RewriteRule ^(.*)$ /owncloud$1 [R=301,L]

# Alias /owncloud /var/www/owncloud

Outside the container, on the UCS, the following ProxyPass config has to be changed aswell:

/etc/apache2/sites-available/000-default.conf
Template: /etc/univention/templates/files/etc/apache2/sites-available/000-default.d/10univention-appcenter

...
ProxyPass /owncloud http://127.0.0.1:40000/owncloud retry=0
ProxyPassReverse /owncloud http://127.0.0.1:40000/owncloud
...

/etc/apache2/sites-available/default-ssl.conf
Template: /etc/univention/templates/files/etc/apache2/sites-available/ssl.d/10univention-appcenter

...
ProxyPass /owncloud https://127.0.0.1:40001/owncloud retry=0
ProxyPassReverse /owncloud https://127.0.0.1:40001/owncloud
...

The templates get /owncloud from a variable %(web_interface)s. IIRC this is hard coded by the App Provider in Provider Portal. If it was modified by OWNCLOUD_SUB_URL aswell at least the apache config outside the container would be sufficiently customizable

Finally both the apache inside and outside the container need to be restarted to apply the config.

Improvements

A collection of things to do, to fix, to improve.

  • Check the other ownCloud apps, can they be installed / removed after the recent changes?
  • Move "docker script setup" from univention scripts to the docker scripts inside the appliance docker image
  • Move "ldap-setup.sh" from univention scripts inside the docker scripts inside the appliance docker image
  • Set the USC server address to a variable to fix this: #6

Collabora plugin: whopi_url not automatically set in join script

During my app appliance test of the Collabora + ownCloud appliance today with VirtualBox no document was created or opened with Collabora.

Reason was that the whopi_url was only configured to https://. This is not enough. After manually setting the url to the correct FQDN, it worked:

univention-app shell owncloud occ config:app:set richdocuments wopi_url --value https://master.collabora.demo/
Config value wopi_url for app richdocuments set to https://master.collabora.demo/

Appliance: https://appcenter-test.software-univention.de/univention-apps/current/collabora/Univention-App-collabora-virtualbox.ova

According to /var/log/univention/join.log Collabora was installed first and then ownCloud. Therefore, the ownCloud join script was responsible for activating the collabora richdocuments plugin. It did so partially:

RUNNING 50owncloud.inst
2019-10-16 13:23:53.411838580+02:00 (in joinscript_init)
[…]
[03.JOIN] check for installation of Collabora
richdocuments enabled
2019-10-16 13:24:43.036642275+02:00 (in joinscript_save_current_version)
[…]
EXITCODE=0

According to the join script in line 155 the whopi_url is only set, if the variable is empty.

I suspect that it was not empty when the join script ran. It contained just https://. Did the behavior of the richdocuments plugin change here and it provides this snipped as default value? If so, the check in the join script needs to be changed. The checks purpose is to not overwrite custom settings made by an administrator when the join script is run later with a newer version of the script.

Changes to database.config.php are not persistent

(Maybe this is an issue for the owncloud docker container repository, if so, please move)

When trying to use another database server, one has to modify the file database.config.php
After that, the external database is used. But this setting is not persistent. When restarting the container, the script /etc/owncloud.d/20-config.sh is executed. In the script, all config files are rebuild from defaults if they do not exist yet. However, the database.config.php is missing this check, so it gets rebuild and rewritten on every container start.

Issues with VM-Ware and ESX Image of the Appliance/UCS

Today a customer wanted to install the Appliance on his VM-Ware setup - it failed because it could not verify the certificates from univention.

Then we tried to install the UCS Server Appliance (4.3) it failed because the verification process could not finish.

Question: Are the ESX images of the Appliance tested regularly? If yes - what are the results?

@ngulden @edamrose

Add example quota in help text next to quota field

A customer recently asked in which format a quota is supposed to be set in the LDAP. The help message next to the quota field could be enhanced as follows. The example is copied from Owncloud's user management module:
quota example

diff 03-join.sh.orig 03-join.sh 
81c81
<   --set longDescription='How much disk space may the user use' \
---
>   --set longDescription='How much disk space may the user use (ex: "512 MB" or "12 GB")' \

New 10.0.9 Testing

Login and Upgrade

  • Upgrade to new version from old 9.1 Version
  • Upgrade to new version from old Appliance
  • Log in ownCloud via IP
  • Log in https & domain name
  • offline setup

File Operations

  • Upload a file
  • Overwrite a file by uploading a new version
  • Rename a file
  • Create a folder
  • Move a file inside a folder
  • Move a folder inside another
  • Remove a file
  • Delete a folder

Office

  • Enable OnlyOffice in ownCloud
  • Install & Configure the OnlyOffice App
  • Create new document in ownCloud with OnlyOffice
  • Enable Collabora in ownCloud
  • Install & Configure the Collabora App
  • Create new document in ownCloud with Collabora

User Management

  • Create a new local user
  • Create new user and group in UCS
  • Sync LDAP users in ownCloud
  • Check the users tab for the LDAP users

Sharing

  • Share a file by link
  • Share a file with a local group
  • Login as local user in that group and check
  • Share a file with a LDAP group
  • Login as LDAP user in that group and check

Enterprise

  • Activate the enterprise trial
  • WND: mount storage
  • Email-Configuration
  • Guest app: Invite guest to share

  • Testcase: Docker container does not start - can user still upgrade to new version?

Completed

  • All tests completed

FR. Leave the original name of the docker image in comments

If we release the app, the docker image get's copied to the univention servers and the link in the docker yml file changes to the univention server repositories.

My suggestion is that we leave the original name in the comments, so we can track what image is actually in the release.

Also for other images, like redis, I would like to know which version we had there.

Rich Documents App should be included in ownCloud docker container

Our automatic appliances tests for the case "No Internet connection" failed. Investigation showed that the container tries to download an install the ownCloud app "Rich Documents" and it fails. The failure leads to a hard abort of the installation process and renders a working appliance, but without ownCloud installed. As soon as an Internet connection exists, the ownCloud app can be installed.

I suggest to bundle the Rich Documents app inside the container and deliver it by default, because it is going to be installed during installation anyway.

For reproduction:

  1. Import the appliance in VirtualBox and set the network interface from "Bridged" to "Host Only" to cut off Internet connection.
  2. Start the virtual machine and run the system setup. It will succeed and provide a working UCS system.
  3. On the UCS portal page, the ownCloud icons are missing. Looking in the App Center, it is available for installation and not installed.

Current errors

Going to install ownCloud (10.0.8-20180428)
Password for Administrator: 
update LDAP schema...
E: object not found
Base configuration for ownCloud

enabling ldap app in docker setup script
checking occ commands

ownCloud is not installed - only a limited number of commands are available

  There are no commands defined in the "market" namespace.  

ownCloud is not installed - only a limited number of commands are available
                                      
  Command "app:enable" is not defined.  
                                        
  Did you mean this?                    
      app:check-code                    

user_ldap: Installing new app ...
user_ldap: App installed.
user_ldap enabled
user_ldap: App already installed and no update available
user_ldap enabled
Read base configs for ldap
creating new ldap config in docker setup script
Created new configuration with configID 's01'


enabling log log rotate
sed: can't read /config.php: No such file or directory
Configuring 4.2/owncloud=10.0.8-20180428
Executing interface configure for owncloud

App Settings Requests

Collection of requests what settings should be configurable in the app settings menu.

  • Make overwrite.cli.url configurable

The owncloud app automatically resets the overwrite.cli.url config parameter in /var/www/owncloud/config/config.php whenever a restart or upgrade happens.

This happens, like #33 and #34, due to a script, that's being executed when the container starts:

/etc/owncloud.d/45-cli.sh:4:occ config:system:set overwrite.cli.url --value http://${OWNCLOUD_DOMAIN}${OWNCLOUD_SUB_URL}

As with #33 and #34 it would be nice to have an App setting for the ownCloud app so administrators can set this value via the Management Console.

  • Make htaccess.RewriteBase configurable

The owncloud app automatically resets the htaccess.RewriteBase config parameter in /var/www/owncloud/config/config.php whenever a restart or upgrade happens.

This happens, like #33, due to a script, that's being executed when the container starts:

/etc/owncloud.d/90-services.sh:4:occ config:system:set htaccess.RewriteBase --value ${OWNCLOUD_SUB_URL}

As with #33 it would be nice to have an App setting for the ownCloud app so administrators can set this value via the Management Console.

  • Make default language configurable

The default language for new owncloud users logging into the web interface for the first time is currently always english. A customer wanted to change this, but there is no easy configurable place to do so. The language is set each time the container starts:

/etc/entrypoint.d/35-language.sh:
[[ -z "${OWNCLOUD_DEFAULT_LANGUAGE}" ]] && OWNCLOUD_DEFAULT_LANGUAGE="en"

/etc/owncloud.d/75-language.sh:
occ config:system:set default_language --value ${OWNCLOUD_DEFAULT_LANGUAGE}

An App setting should be defined for the ownCloud app so Administrators can set this value via the Management Console.

#33
#34
#35

Could not resolve host: updates.software-univention.de

I get this error very often when I start up my test appliance after I shut them down.

Is this a generic test environment error ? If yes - how can it be fixed?

My solution is to create a host entry in the /etc/hosts file from the hostname to the IP.

@edamrose

AD Join issues

I suggest removing this option as our appliance is targeted to people with low know how and this option is a great way to break the appliance or to cause support load in the future.

I have never seen it implemented without issues.

@edamrose @ngulden @felixboehm

changes from current docker image + scripts to new docker image + scripts

Here I would like to sum up changes that would make life easier for me and anyone who would maintain the appliance.

The plan is simple - create a separate docker image for the appliance, that we can adjust to the needs of the appliance.

The main reason at this moment is to include the richtdocuments app for collabora into the docker image. That way the installation would not fail if the appliance would be set up with no internet access.

Further reasons could be to move some content of the current appliance scripts from the univention directly in to the docker scripts. In order to migrate from the old appliance (9.x) to the new one (10.x) we had to make some changes so the user would keep their data, users and shares. That was achieved through complicated script in script scenarios. It would be much easier if we include this simply in the docker image.

Make default language configureable

The default language for new owncloud users logging into the web interface for the first time is currently always english. A customer wanted to change this, but there is no easy configurable place to do so. The language is set each time the container starts:

/etc/entrypoint.d/35-language.sh:
[[ -z "${OWNCLOUD_DEFAULT_LANGUAGE}" ]] && OWNCLOUD_DEFAULT_LANGUAGE="en"

/etc/owncloud.d/75-language.sh:
occ config:system:set default_language --value ${OWNCLOUD_DEFAULT_LANGUAGE}

An App setting should be defined for the ownCloud app so Administrators can set this value via the Management Console.

Update Notification

Steps to reproduce

  1. Setup a new ownCloud Appliance.
  2. Go to the web UI
  3. Click on the Bell Notification
  4. Click on the Update available for..
  5. URL Changes from

http://172.42.16.152/owncloud/index.php/apps/files/?dir=/

to

http://localhost/owncloud/index.php/apps/market/#/app/user_ldap

and gives you an error.

confirmed by @edamrose

10.0.8 Wrap up

Last steps before releasing new 10.0.8

  • Mark echos in scripts with the scriptname
  • rename "trying to enable" -> "enabling"
  • restore the mount of the setup-ldap.sh and test it
  • check how many 9.1 instances are there

Optional:

  • Put content of restore data after setup into docker script setup
  • check why the admin password is checked so late in the process.

Limit docker log size

After our customer had the stoppage of service because of lack of storage again, we need to implement a logging limitation on the docker container logs.

here is what I found.

@tboerger what do you think?


https://stackoverflow.com/questions/33017329/where-is-a-log-file-with-logs-from-a-container


Docker 1.8 and docker-compose 1.4 there is already exists a method to limit log size using docker compose log driver and log-opt max-size:

mycontainer:
  ...
  log_driver: "json-file"
  log_opt:
    # limit logs to 2MB (20 rotations of 100K each)
    max-size: "100k"
    max-file: "20"

In docker compose files of version '2' , the syntax changed a bit:

version: '2'
...
mycontainer:
  ...
  logging:
    #limit logs to 200MB (4rotations of 50M each)
    driver: "json-file"
    options:
      max-size: "50m"
      max-file: "4"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.