Code Monkey home page Code Monkey logo

wd-ultra-mycloud-owncloud's Introduction

OwnCloud on Docker on a Western Digital Ultra (MyCloud Home Server)

This project makes it possible to run OwnCloud on Docker on a Western Digital Ultra (inspired by a Respberry Pi idea).

This is a very experimental project and will certainly void your warranty on WD Ultra... but it is not intended to destroy your MyCloud, just to put OwnCloud working together (because it is an amazing software!).

Motivation Texts

community steps for Docker and Python

Make sure these are the latest from WD community, as these change over time.

Docker for Newer (OS5) Systems

Docker for Older (OS3?) Systems

Entware

Python (untested)

Install entware from WDCommunity.

opkg install python-pip

pip install --upgrade setuptools

pip install setuptools docker-compose

my installation (tested!)

These are the steps I have done to make ownCloud work successfully (please contribute if you think it's not good enough!):

  1. setup your MyCloud normally (including RAID), and log-in via SSH to your WD Ultra (mine is precisely WD My Cloud EX2 Ultra)

  2. discover where your data is stored with df -h (mine is on /mnt/HD/HD_a2)

  3. create a folder for storing your ownCloud software and data: mkdir /mnt/HD/HD_a2/owncloud_www

  4. Discover your docker version (mine is 1.7.0/API 1.19). Since it's a little old, it doesn't have command docker volume.. so, this is a workaround

4.1) Create a data volume container for your data: docker create -v /mnt/HD/HD_a2/owncloud_www:/var/www/ --name owncloud_www resin/rpi-raspbian:latest

4.2) Create a data volume container for your mysql data: docker create -v /mnt/HD/HD_a2/mysql_data:/var/lib/mysql --name mysql_data resin/rpi-raspbian:latest

  1. Download these Dockerfile and configurations (I used wget for this). I put these in a folder new_owncloud_docker, but it doesn't matter.

  2. Build Dockerfile (mine is slightly the same as the original): docker build -t comzone/rpi-owncloud:latest .

  3. Run owncloud daemon: docker run --restart=always --volumes-from owncloud_www --volumes-from mysql_data -d -i -t -p 4430:443 -p 8000:80 comzone/rpi-owncloud

7.1) Enter docker container (docker exec ..... /bin/bash). The mysql should have been destroyed too because of the data container volume... so, to rebuild it: mysql_install_db --user=mysql --ldata=/var/lib/mysql

/usr/bin/mysqladmin -u root password 'root123'

cat /etc/mysql/debian.cnf # view debian password, suppose its MPxBDvZrJKq99eJS

mysql -u root -p

GRANT ALL PRIVILEGES ON *.* TO 'debian-sys-maint'@'localhost' IDENTIFIED BY 'MPxBDvZrJKq99eJS';

service mysql start

7.2) Maybe you will need to chown -R mysql:mysql /var/lib/mysql at some point, I don't know exactly now...

  1. Enter docker container (docker exec ..... /bin/bash), go to /var/www folder and download owncloud: cd /var/www && wget -q -O - http://download.owncloud.org/community/owncloud-latest.tar.bz2 | tar jx -C . setup permissions too: chown -R www-data:www-data owncloud

  2. Exit docker container, and verify your owncloud data exists on local volume folder: ls -la /mnt/HD/HD_a2/owncloud_www/. It should display: drwxr-xr-x 13 33 33 4096 Sep 16 23:55 owncloud, or drwxr-xr-x 13 www-data www-data 4096 Sep 17 02:55 owncloud

  3. Go to your home system, suppose MyCloud is running at 192.168.1.102, so you can find owncloud at 192.168.1.102:8000. Create an admin password and select MariaDB. Default user can be root, password root123, database ownclouddb. Database is not being kept on volume for now, so take care of not destroying the database and container (this should be improved in the future).

Configuring no-ip

  1. You need a router that supports NAT Lookback: http://opensimulator.org/wiki/NAT_Loopback_Routers My D-LINK DIR-809 router did not support, so I changed to a TP-LINK 740 that supports it.

  2. Log-in to no-ip and register your dynamic domain. On the NAS device (via SSH), create a file on docker container root /noipupdater/noipupdater.sh, and create auxiliary folders noipupdater/configdir and noipupdater/logdir. Example is here: https://raw.githubusercontent.com/AntonioCS/no-ip.com-bash-updater. Use encoded email and password ([email protected] => x%40gmail.com), this may be useful: https://meyerweb.com/eric/tools/dencoder/

  3. Inside you docker owncloud container, apt install cron && service cron start. Add execution permission chmod +x /noipupdater/noipupdater.sh and add line to crontab -e: */15 * * * * /noipupdater/noipupdater.sh . It will refresh IP after 15 minutes, and only submit DNS request if IP changes. Ideally, cron should be supervised, perhaps editing /etc/supervisor/conf.d/lamp.conf:

[program:crontab]
command=/usr/bin/crontab
autorestart=true

3.1) NEVER, EVER try to edit MyCloud default XML file just to enter a native cronjob... I bricked my device this way! The native cronjob is volatile and hard to deal with, it's much easier to put that in your docker container that will be loaded automatically anyway ;)

  1. Edit vi owncloud_www/owncloud/config/config.php, and add to trusted_domains : 1 => 'xxx.ddns.net:8000, if your port is 800.

  2. Enter docker container and edit /etc/apache2/apache2.conf, adding ServerName xxx.ddns.net. For this to work, start.sh script must also use xxx.ddns.net instead of $(hostname), to generate a correct ssl certificate

  3. Forward port 8000 and 4430 to your NAS server, from your router

memcaching with apcu/redis (server becomes MUCH faster!!)

https://doc.owncloud.org/server/9.0/admin_manual/configuration_server/caching_configuration.html#id4

  1. Enter server with docker exec ... /bin/bash

  2. apt install redis-server php5-redis php5-apcu

  3. edit /var/www/owncloud/config/config.php (using Redis/APCu) and add:

   'memcache.locking' => '\OC\Memcache\Redis',
   'memcache.local' => '\OC\Memcache\APCu',
   'redis' => array(
     'host' => 'localhost',
     'port' => 6379,
      ),

3.1) UPDATE. My redis server keeps crashing (after 40 minutes or on heavy loads), with random messages (Uncaught exception 'RedisException' with message 'Connection lost'), or just segmentation faults. Perhaps my configuration is not good, so I'm using APCu for both caches (for now).

  1. Edit /etc/supervisor/conf.d/lamp.conf and add:
[program:redis]
command=/usr/bin/redis-server
autorestart=true
  1. Restart web server. The best way I found was: pkill start.sh. That killed my SSH session, but everything was restarted (I tried other ways, but didn't succeed in truly restarting all apache sessions).

limiting number of apache instances (MPM)

Apache was consuming too much memory (I only have 1GB) and leaving a lot of work to SWAP (even my SSH sessions suddently got slow...). So, default MPM was giving too much instances on memory (around 13, each with 180MB), so I limited that. Edit /etc/apache2/mods-enabled/mpm_prefork.conf, my values are (around 8 servers now):

<IfModule mpm_prefork_module>
	StartServers	          3
	MinSpareServers		  3
	MaxSpareServers		 5
	MaxRequestWorkers	  100
	MaxConnectionsPerChild   0
</IfModule>

Configuring OpenVPN (testing)

  1. Select a volume name for the openvpn data: OVPN_DATA="ovpn-data"

  2. Create the data volume container: docker create --name $OVPN_DATA -v /mnt/HD/HD_a2/openvpn_data:/etc/openvpn hypriot/armhf-busybox

  3. Create a public domain server. I'm using no-ip.com service, selected name xxxx.ddns.net

  4. Run config openvpn: docker run --volumes-from $OVPN_DATA --rm evolvedm/openvpn-rpi ovpn_genconfig -u udp://xxxx.ddns.net

OpenVPN - second try

Steps from: https://github.com/olivierguerriat/rpi-docker-openvpn

docker run -v /mnt/HD/HD_a2/$OVPN_DATA:/etc/openvpn -d -p 1194:1194/udp --cap-add=NET_ADMIN --name $OVPN_DATA mjenz/rpi-openvpn

docker run -v /mnt/HD/HD_a2/$OVPN_DATA:/etc/openvpn --rm -it arm32v6/alpine vi /etc/openvpn/openvpn.conf

-> Add this with vi: push "dhcp-option DNS 192.168.1.1"

docker create --name ovpn-rpi-mjenz -v /mnt/HD/HD_a2/ovpn-data:/etc/openvpn mjenz/rpi-openvpn

docker run -v /mnt/HD/HD_a2/$OVPN_DATA:/etc/openvpn --rm mjenz/rpi-openvpn ovpn_genconfig -u udp://xxxxx.ddns.net

docker run -v /mnt/HD/HD_a2/$OVPN_DATA:/etc/openvpn --rm -it mjenz/rpi-openvpn ovpn_initpki

Manually Adding Files do ownCloud

If you want to add GigaBytes of files, please don't use sync, it will take years!! Use SSH or USB to copy your files directly to /mnt/HD/.../owncloud_www/data/USERNAME/files/NEW_DIRECTORY. To index these files, perform a docker exec ... /bin/bash into your container, and execute: cd /var/www/owncloud, sudo -u www-data php occ files:scan --path "USERNAME/files/NEW_DIRECTORY"

setup Calendar (CalDav)

Enter admin user and install Calendar app. Install some Android Calendar (such as SimpleCalendar) and Android CalDav connector (such as OpenSync).

setup Collabora Office ("Google Docs")

First of all, that won't work in your MyCloud device, unfortunately... it seems the whole software is still not compatible with ARM (by 2018), it consumes several GB of RAM and occupies a lot of disk space. In the future, that may be compatible with Raspberry Pi, but for now, best thing is to install in on a 3rd party computer (it could be a Cloud Computer on DigitalOcean, for example).

  1. On x86_64 computer: docker run -t -d -p 9980:9980 -e "domain=xxx.ddns.net" -e "cert_domain=xxx.ddns.net" -e "username=admin" -e "password=S3cRet" --restart always --cap-add MKNOD collabora/code

  2. Wait a few minutes and try in this computer: curl -v https://localhost:9980. If answer is OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to localhost:9980, wait a little longer, until answer is: curl: (60) SSL certificate problem: self signed certificate in certificate chain

  3. Adjust your router to manage your domain and port 9980, then try curl -v https://xxx.ddns.net:9980 on your MyCloud device.

3.1) open https://xxx.ddns.net:9980 on your browser and make sure you accept the self-signed certificate (works on Firefox, but not too good on Chrome... it would be better to have a lets encrypt certificate)

  1. Install Collabora (richdocuments) with admin on owncloud. Go to Admin page, Additional, and configure domain: https://xxx.ddns.net:9980'. DO NOT EDIT MANUALLY: nano /var/www/owncloud/apps/richdocuments/lib/appconfig.php... that will bring integrity check problems.

  2. Get self-signed certificate on x86_64 machine: docker exec -it YOUR_CONTAINER cat /etc/loolwsd/ca-chain.cert.pem

  3. Add certificate on owncloud docker: nano /var/www/owncloud/resources/config/ca-bundle.crt, go to last line and add the contents of the certificate from last step. THIS WILL BREAK integrity checks... so it's best to adopt a global lets encrypt certificate. If you really need to do this (like me) you can whitelist ca-bundle file (edit owncloud/config/config.php):

  'integrity.excluded.files' =>
      array (
         'resources/config/ca-bundle.crt'
     ),
  1. Open owncloud Collabora (Rich Documents) in any user, and that should work.

install local sync for desktop and Android/LineageOS

#sync client
sudo apt install owncloud-client
#nautilus integration
sudo apt install owncloud-client-nautilus

For Android (or LineageOS) install owncloud app (it's paid, but less than a dollar). Also install FolderSync app to sync specific smartphone folders.

future advices

Should have started from noip configuration first! So all scripts are already done correctly with noip server. I had to do it twice, first locally, then realized it wouldn't work in the outside world... must think on this before everything starts.

Perhaps it was nice to have a volume also for apache certificates...

More information

Read about docker at http://docker.com

This project is inspired by:
git://github.com/comzone/rpi-owncloud.git http://dischord.org/2013/07/10/docker-and-owncloud/ http://dischord.org/2013/08/13/docker-and-owncloud-part-2/

wd-ultra-mycloud-owncloud's People

Contributors

igormcoelho avatar comzone avatar yankcrime avatar wookietreiber avatar vncoelho avatar

Stargazers

 avatar Tobias Reis Quinteiro avatar Ion Mincu avatar Ramon Buckland avatar  avatar Rafael K. avatar Michael Kalaf avatar camac avatar jiexian avatar Lᴉɐm avatar  avatar

Watchers

 avatar James Cloos avatar

Forkers

vncoelho

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.