cozy / cozy-controller Goto Github PK

View Code? Open in Web Editor NEW

5.0 15.0 13.0 2.15 MB

Cozy Module and Apps deployer

Home Page: cozy.io

License: GNU Affero General Public License v3.0

JavaScript 0.12% Shell 1.57% CoffeeScript 98.31%

cozy-controller's Introduction

Cozy Controller

The Cozy Controller is used to fetch and manage the applications in the Cozy Platform.

Requirement

Coffee script:

npm install coffee-script -g

Install

Installation:

npm install cozy-controller -g

Start:

cozy-controller

Contribution

You can contribute to the Cozy Controller in many ways:

Pick up an issue and solve it.
Add support for Python applications.

License

Cozy Controller is developed by Cozy Cloud and distributed under the AGPL v3 license.

What is Cozy?

Cozy is a platform that brings all your web services in the same private space. With it, your web apps and your devices can share data easily, providing you with a new experience. You can install Cozy on your own hardware where no one profiles you.

Community

You can reach the Cozy Community by:

Chatting with us on IRC #cozycloud on irc.freenode.net
Posting on our Forum
Posting issues on the Github repos
Mentioning us on Twitter

cozy-controller's People

Contributors

Stargazers

Watchers

Forkers

nledez m4dz obigroup kloadut fredericalix frankrousseau poupotte nono nicofrand pkdevbox felix-lambert brvier kosssi

cozy-controller's Issues

Couchdb port

Is it possible to change the couchdb port used ? It seems hardcoded in many parts of cozycloud, making it quite hard to change.

Possibly change branch when updating an app

So the use case is the following. There is an app K, which is hosted on github.

The app is in the registry, without any branch specification, that is: the app is in master.
People install and use the app.
K's authors decide that the release versions of the app should live in a particular branch called stable.

After some chatting with @poupotte on IRC, apparently users would need to uninstall and reinstall the app, so far. Could there be a way such that, when changing the branch key in the cozy registry JSON, the cozy-controller could be aware of this and change branch before updating its code?

Usage of git lib and npm lib

What do you think of using node.js libs instead of shell to handle Git and Npm operations?

Application directory should be passed through env variable

In the current state applications have no choice but to assume app_dir is the default /usr/local/var/cozy and application's name has not been changed.

It would be better if the controller passed a APPLICATION_PERSISTENT_DIRECTORY (or something shorter) env variable to each application. This way the application can path.join whatever it wants to this path.

Can't install an application with branch named like a folder in repository

Example:
I can't install cozy-home on branch test because there is a folder test on home repository.

Cause:
We use git checkout to change repository branch : https://github.com/cozy/cozy-controller/blob/master/server/lib/git.coffee#L71

"TypeError: Cannot set property 'old' of undefined" on cozy-controller startup

When I launch my cozy-controller I have an error:

# cozy-controller
info - ### FILE INITIALIZATION ###
info - init: source directory

/usr/lib/node_modules/cozy-controller/build/server/lib/conf.js:38
      data.old = require('/etc/cozy/.controller-backup.json');
               ^
TypeError: Cannot set property 'old' of undefined
    at readFile (/usr/lib/node_modules/cozy-controller/build/server/lib/conf.js:38:16)
    at Object.module.exports.init (/usr/lib/node_modules/cozy-controller/build/server/lib/conf.js:59:10)
    at initialize (/usr/lib/node_modules/cozy-controller/build/server/initialize.js:157:17)
    at Object.module.exports.init (/usr/lib/node_modules/cozy-controller/build/server/initialize.js:173:12)
    at module.exports (/usr/lib/node_modules/cozy-controller/build/server.js:37:10)
    at Object.<anonymous> (/usr/lib/node_modules/cozy-controller/bin/cozy-controller:7:43)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)

You can found bellow file content.

/etc/cozy/.controller-backup.json:

{
  "npm_registry": false,
  "npm_strict_ssl": false,
  "dir_log": "/usr/local/var/log/cozy",
  "dir_source": "/usr/local/cozy/apps",
  "env": {
    "global": false,
    "data-system": false,
    "home": {
      "MARKET": "private"
    },
    "proxy": false
  }
}

And an empty /etc/cozy/controller.json

Stack version is not updated correctly

I updated my stack from my Home, because the data system was out of date. It completed successfully (I now have the latest data-system version), but I still have the old version number.

Cannot remove application that can't start

I installed Konnectors. It seems that it fails to start. Here are the logs of the Controller:

[Mon, 20 Oct 2014 09:26:06 GMT] - POST /drones/konnectors/clean - - - - ms - -

It looks like the Controller never answers to that request. When the home calls for the controller to clean an app, it leads to a SOCKET HANG UP error (which sounds logical).

What leads to that problem?

Use or instead of ||

That's more coffeescript style!

I talk about that code: https://github.com/cozy/cozy-controller-new/blob/master/server/lib/conf.coffee#L58-L69

"Are you sure, you are root ?" should be reworded

This question is ambiguous, and can be understood in two mutually exclusive ways:

Are you sure that you are the root user?
Are you sure you want to run as root? (Implying that it can't run as root)

The third problem is that this is rhetoric question, process doesn't wait for an answer.

Better wording is: "cozy-controller should be run as root, current user is ''"

Die on uncaughtException

https://github.com/cozy/cozy-controller-new/blob/9f408c778c2ec4d1fa14ebc920eec2097617c009/server.coffee#L36

From what i have read, we should always exit after uncaughtException
/!\ flush stdout before exiting

Add queue management layer

In this PR we pint that apps could be able to register some delayed tasks to a global task layer that can execute the task in another timeframe (which can be just another thread called immediately or that can be delayed to another date).

The app should:

register a job (for itself or another app)
be woke up by the controller to execute a task

I suggest to rely on kue for the queue manager.

Add coffeescript requirement to the documentation

The application cannot run without Coffeescript. We should tell it in the README or make it a JS app.

Restore the autostop option for users with low memory

We should find a way for advanced users using devices with low memory to activate the autostop option, to stop applications when they are not used.
Si this thread

Patch to 2.0 failed on update

Here is the controller output on startup:

# cozy-controller
info - ### FILE INITIALIZATION ###
info - APPLY patch ...
info - Move old source directory ...
info - /usr/local/cozy/apps/tmp-webdav : Moved
info - /usr/local/cozy/apps/tmp-proxy : Moved
WARNING : 
{ [Error: ENOENT, no such file or directory '/usr/local/cozy/apps/notes/notes']
  errno: 34,
  code: 'ENOENT',
  path: '/usr/local/cozy/apps/notes/notes',
  syscall: 'readdir' }
Error: ENOENT, no such file or directory '/usr/local/cozy/apps/notes/notes'
  at Object.fs.readdirSync (fs.js:654:18)
  at getRepo (/usr/local/lib/node_modules/cozy-controller/build/server/lib/patch.js:22:13)
  at updateSourceDir (/usr/local/lib/node_modules/cozy-controller/build/server/lib/patch.js:71:14)
  at /usr/local/lib/node_modules/cozy-controller/build/server/lib/patch.js:87:28
  at ChildProcess.<anonymous> (/usr/local/lib/node_modules/cozy-controller/build/server/lib/patch.js:61:14)
  at ChildProcess.EventEmitter.emit (events.js:98:17)
  at maybeClose (child_process.js:743:16)
  at Process.ChildProcess._handle.onexit (child_process.js:810:5)

Process exit with code 0

Run the cozy-monitor patch command on startup

To migrate an install based on the current controller to an install with the new controller, it requires to run a patch from Cozy Monitor. It would be better if the patch was automatically started when the controller starts.

Use printit for logging

For later, we could use printit to make prettier console output.

Parse json files with require

You can use the require command to load a JSON config file.

SUDO_USER env variable

Are you sure it should be set?

https://github.com/cozy/cozy-controller/blob/master/server/lib/spawner.coffee#L28

Npm warning make stack update fails

Npm displays a deprecation warning during stack update. This prevent the update to complete and leave the stack stopped.

[Mon, 14 Dec 2015 00:49:46 GMT] - POST /apps/update-stack - - - - ms -  -
info - npm:install:success
info - monitor: update
info - monitor: update
info - monitor: update
error - Error during monitor update: npm WARN deprecated This version of npm lacks support for important features,
(…)


info - controller: update
info - controller: update
info - controller: update
error - Error during controller update: npm WARN deprecated This version of npm lacks support for important features,
(…)

Sending error to client: 
Error: Cannot update stack: Error during controller update: npm WARN deprecated This version of npm lacks support for important features,

    at /usr/lib/node_modules/cozy-controller/build/server/controllers/applications.js:298:19
    at /usr/lib/node_modules/cozy-controller/node_modules/async/lib/async.js:525:46
    at /usr/lib/node_modules/cozy-controller/node_modules/async/lib/async.js:254:17
    at /usr/lib/node_modules/cozy-controller/node_modules/async/lib/async.js:151:21
    at /usr/lib/node_modules/cozy-controller/node_modules/async/lib/async.js:251:21
    at /usr/lib/node_modules/cozy-controller/node_modules/async/lib/async.js:615:34
    at /usr/lib/node_modules/cozy-controller/node_modules/async/lib/async.js:516:25
    at /usr/lib/node_modules/cozy-controller/build/server/controllers/applications.js:47:18
    at ChildProcess.exithandler (child_process.js:656:7)
    at ChildProcess.emit (events.js:98:17)

Mixed content alert in firefox

Hei, my cozy instance shows a security warning for mixed content (Iceweasel 42, Cozy data-system 2.1.2).

What I expect is getting no security warning.

I believe the "Android app on google play" icon could be the reason for that, since it seems to be requested over http.
In any way, I don't feel really good with the idea that my cozy interface downloads an image from google.

Could a custom image hosted on the instance be used instead, or an option to remove that image ?

Thanks for your time,

Bad header for bash script

Bash script header should be #!/bin/bash instead of #/bin/bash

Request-json-light instead of request-json?

request-json-light comes with many less dependencies. Could we try to use it instead of request-json?

Wrong disk space shown if database is not default path

It has been reported that displayed disk space was not correct when not using the default path for the database.
See: https://clochix.cozycloud.cc/public/haste/ZnsErsPc

not upgrading to 2.0.32

Hi,
Cozy home is showing that controller is outdated (2.0.31). When I try to "update all", it says all have been updated but controller is still 2.0.31.

Sorry if that bug was supposed to be reported in cozy-home's project.

Thanks
J.

Improve exit logging

Exit logging is a little bit too minimalist:

https://github.com/cozy/cozy-controller-new/blob/master/server.coffee#L33

Do not override process object in start function

Hello,

The process variable describes the current process. It should not be overriden.
https://github.com/cozy/cozy-controller/blob/master/server/lib/spawner.coffee#L121

Find a way to display error logs when an app is crashing.

Currently, application logging is stopped immediately after an app is crashing. It's like the controller doesn't let the app to tell what's wrong.

Apps don't have the good rights on their installation folder

An app cannot do anything in its folder because all its files belong to root.

Example of issue that brings: the photos app can't upload photos anymore.

controller installed as an application show as "community"

While trying out the cozy command line I installed controller via cozy-monitor install controller

now it show up in the Platform column but also in the Applications column. When I try to upgrade it in the applications column I get :

Not very reassuring for a core component of cozy.

Error with supervisor configuration file

In the installation process, when I try to launch supervisor, I get the following error:

Error: Format string 'NODE_ENV=%(ENV_NODE_ENV)s' for 'environment' contains names which cannot be expanded

Use writeFile instead of open + write

What do you think of the writeFile function?

http://nodejs.org/api/fs.html#fs_fs_writefile_filename_data_options_callback
https://github.com/cozy/cozy-controller-new/blob/master/server/lib/conf.coffee#L96-L99

By the way async functions should always have and manage a callback parameter.

NPM install failed display undefind error

Once a NPM error occured during the installation, the message returned by the home is Error is undefined.

Here is the controller stack trace:

...
npm http 200 https://npm-registry.cozycloud.cc:5448/monocle/1.1.51
npm http GET https://npm-registry.cozycloud.cc:5448/monocle/-/monocle-1.1.51.tgz
npm http 200 https://npm-registry.cozycloud.cc:5448/monocle/-/monocle-1.1.51.tgz
npm http GET https://npm-registry.cozycloud.cc:5448/readdirp
npm http GET https://npm-registry.cozycloud.cc:5448/uglify-js
npm http GET https://npm-registry.cozycloud.cc:5448/uglify-js
npm http GET https://npm-registry.cozycloud.cc:5448/uglify-js
npm http GET https://npm-registry.cozycloud.cc:5448/promise
npm http GET https://npm-registry.cozycloud.cc:5448/css
npm http GET https://npm-registry.cozycloud.cc:5448/minimist/0.0.8
npm http 200 https://npm-registry.cozycloud.cc:5448/uglify-js
npm http 200 https://npm-registry.cozycloud.cc:5448/uglify-js
npm http 200 https://npm-registry.cozycloud.cc:5448/uglify-js
npm ERR! Error: failed to fetch from registry: uglify-js
npm ERR!     at RegClient.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-registry-client/lib/get.js:140:18)
npm ERR!     at cb (/usr/local/lib/node_modules/npm/node_modules/npm-registry-client/lib/request.js:26:9)
npm ERR!     at RegClient.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-registry-client/lib/request.js:101:16)
npm ERR!     at cb (/usr/local/lib/node_modules/npm/node_modules/npm-registry-client/lib/request.js:124:9)
npm ERR!     at RegClient.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-registry-client/lib/request.js:268:12)
npm ERR!     at Request.self.callback (/usr/local/lib/node_modules/npm/node_modules/request/request.js:123:22)
npm ERR!     at Request.EventEmitter.emit (events.js:98:17)
npm ERR!     at Request.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/request/request.js:893:14)
npm ERR!     at Request.EventEmitter.emit (events.js:117:20)
npm ERR!     at IncomingMessage.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/request/request.js:844:12)
npm ERR! If you need help, you may report this *entire* log,
npm ERR! including the npm and node versions, at:
npm ERR!     <http://github.com/npm/npm/issues>

npm ERR! System Linux 2.6.32-16-pve
npm ERR! command "/usr/local/bin/node" "/usr/local/bin/npm" "--production" "install" "--user" "cozy-emails"
npm ERR! cwd /usr/local/cozy/apps/emails
npm ERR! node -v v0.10.26
npm ERR! npm -v 1.4.3
npm http 200 https://npm-registry.cozycloud.cc:5448/promise
npm http GET https://npm-registry.cozycloud.cc:5448/async
npm http GET https://npm-registry.cozycloud.cc:5448/source-map/0.1.34
npm http GET https://npm-registry.cozycloud.cc:5448/uglify-to-browserify
npm http GET https://npm-registry.cozycloud.cc:5448/optimist
npm http 200 https://npm-registry.cozycloud.cc:5448/minimist/0.0.8
npm http GET https://npm-registry.cozycloud.cc:5448/minimist/-/minimist-0.0.8.tgz
npm http 200 https://npm-registry.cozycloud.cc:5448/async
npm http 200 https://npm-registry.cozycloud.cc:5448/source-map/0.1.34
npm http GET https://npm-registry.cozycloud.cc:5448/source-map/-/source-map-0.1.34.tgz
npm http 200 https://npm-registry.cozycloud.cc:5448/minimist/-/minimist-0.0.8.tgz
npm http 200 https://npm-registry.cozycloud.cc:5448/optimist
npm http 200 https://npm-registry.cozycloud.cc:5448/css
npm http 200 https://npm-registry.cozycloud.cc:5448/readdirp
npm http 200 https://npm-registry.cozycloud.cc:5448/source-map/-/source-map-0.1.34.tgz
npm http GET https://npm-registry.cozycloud.cc:5448/is-promise
npm http GET https://npm-registry.cozycloud.cc:5448/css-stringify/1.0.5
npm http GET https://npm-registry.cozycloud.cc:5448/css-parse/1.0.4
npm http GET https://npm-registry.cozycloud.cc:5448/minimatch
npm http GET https://npm-registry.cozycloud.cc:5448/source-map
npm http 200 https://npm-registry.cozycloud.cc:5448/uglify-to-browserify
npm http 200 https://npm-registry.cozycloud.cc:5448/is-promise
npm http 200 https://npm-registry.cozycloud.cc:5448/source-map
npm http GET https://npm-registry.cozycloud.cc:5448/wordwrap
npm http GET https://npm-registry.cozycloud.cc:5448/amdefine
npm http 200 https://npm-registry.cozycloud.cc:5448/css-stringify/1.0.5
npm http GET https://npm-registry.cozycloud.cc:5448/css-stringify/-/css-stringify-1.0.5.tgz
npm http 200 https://npm-registry.cozycloud.cc:5448/amdefine
npm http 200 https://npm-registry.cozycloud.cc:5448/css-parse/1.0.4
npm http 200 https://npm-registry.cozycloud.cc:5448/minimatch
npm http GET https://npm-registry.cozycloud.cc:5448/css-parse/-/css-parse-1.0.4.tgz
npm http GET https://npm-registry.cozycloud.cc:5448/lru-cache
npm http GET https://npm-registry.cozycloud.cc:5448/sigmund
npm http 200 https://npm-registry.cozycloud.cc:5448/css-stringify/-/css-stringify-1.0.5.tgz
npm http 200 https://npm-registry.cozycloud.cc:5448/css-parse/-/css-parse-1.0.4.tgz
npm http 200 https://npm-registry.cozycloud.cc:5448/sigmund
npm http 200 https://npm-registry.cozycloud.cc:5448/wordwrap
npm http 200 https://npm-registry.cozycloud.cc:5448/lru-cache
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /usr/local/cozy/a

Problem during controller update

When controller is updated, if there is a npm problem, cozy doesn't restart.
Two solutions to limite this issue:

Check if our cozy use our npm registry
Retry controller update if it receives an error.

missing things for docker compose

I need an option to listen 0.0.0.0

How can I setup indexer address and couchdb address?

An shell bang error still in build/server/lib/adduser.sh

A #/bin/bash still in build/server/lib/adduser.sh

Stack applications are duplicated in database

Unauthentified app + Install error won't stop the app from running

When installing Kresus on my hosted CC instance (benj.), I get some weird errors which look timely related. In particular, the installation aborts with "this app is broken, try to reinstall it". However, the logs are still filling up (with the installation of Weboob) !
Also, there are plenty of error messages in the log, due to the fact that the app isn't authentified:

error when saving temporary log: Error when reading setting weboob-log: Error: { error: 'Application is not authenticated' }

"Unfortunately", when I've retried, the error didn't show up anymore, so this might be an intermittent issue (who doesn't love race conditions!).

Update seems to not work on Raspberry Pi 2

An user (tchabaud) reported that he can't update his applications on the RPi2. Here is the log:
http://pastebin.com/1mcrtjfb

It looks like there is a bug in the application module. Here is the initial conversation: https://forum.cozy.io/t/feedback-needed-new-raspberry-pi-2-cozy-image/272/16

Start application in python

See forum

Change log dir to /usr/local/var/log

It would be proper to use the /usr/local/var/log dir to write logs. I think /var/log is dedicated to lower level modules.

Controller shouldn't crash if a package.json is corrupted

After an update, the package.json of an app was broken (git merge conflicts). The controller crash every time he tries to start the application. Then supervisoctl restart the controller, which try to start again the broken app, crashes…

Allow installing Debian packages at install time

Weboob keeps growing its dependency base (before it included libffi-dev, now libjpeg-dev as well), and users don't understand why Kresus isn't installing anymore on their hosted Cozy. It'd be nice if we could add some Debian-only packages dependencies in the app manifest, and if these could be installed at install time only if the host is Debian. That would spare @nledez and me some annoyances with this issue :)

Unexplained error in the logs

Moved cozy/cozy-home#469

Hello, I have regularly this can of error in the controller log (self hosted instance) :


ications.js:129:13                                                              
    at Object.module.exports.stop (/usr/local/lib/node_modules/cozy-controller/b
uild/server/lib/controller.js:284:12)                                           
    at Object.module.exports.stop [as handle] (/usr/local/lib/node_modules/cozy-
controller/build/server/controllers/applications.js:126:21)                     
    at next_layer (/usr/local/lib/node_modules/cozy-controller/node_modules/amer
icano/node_modules/express/lib/router/route.js:103:13)                          
    at Object.module.exports.check [as handle] (/usr/local/lib/node_modules/cozy
-controller/build/server/middlewares/token.js:31:16)                            
    at next_layer (/usr/local/lib/node_modules/cozy-controller/node_modules/amer
icano/node_modules/express/lib/router/route.js:103:13)                          
    at Route.dispatch (/usr/local/lib/node_modules/cozy-controller/node_modules/
americano/node_modules/express/lib/router/route.js:107:5)                       
    at /usr/local/lib/node_modules/cozy-controller/node_modules/americano/node_m
odules/express/lib/router/index.js:213:24                                       
    at param (/usr/local/lib/node_modules/cozy-controller/node_modules/americano
/node_modules/express/lib/router/index.js:303:14)                               
    at param (/usr/local/lib/node_modules/cozy-controller/node_modules/americano
/node_modules/express/lib/router/index.js:319:14)

The problem occurs sometimes on updating an app (but I don't know the status of the app).

I think there is a thing here :

https://github.com/cozy/cozy-controller/blob/master/server/lib/controller.coffee#L313

The update is done even if an error occurs during stop of application.

Improve handling of npm start script

For now, when there's a npm start script, we just split the line and use the second word as main program name. Trying to port to Cozy third party applications, I found lot of cases that didn't work:

node server.js --prod: without the option, the application won't run as expected;
node app.js start: the application is expecting a command line subcommand;
ENV=prod node server.js: here we'll try to start node;
node server: this will start server.js, but, as server itself doen't exist, our controller is unable to start the right file.

So I was wondering why couldn't we just exec npm start?

Option to start cozy-stack not as subprocess

To get some of the memory advantages of cozy-light, may be we could add an option to the cozy-controller for it to start home / proxy / DS by just requiring them and starting them (keep them in the same process than the cozy-controller)

Can install an invalid application

STR:

As the cozy user, run this command: cozy-monitor install https://github.com/bnjbvr/kresus
You should see some error telling you it is an invalid application
Then run : cozy-monitor status
=> You should see https://github.com/bnjbvr/kresus

NOOOOO !

Florent

seems like using the same file for standard and error logs doesn't always work great: sometime, logs are mixed / truncated, and we loose some importants informations ;
it would make monitoring errors easiers ;
it would encourage us to remove false error logs ;