deepkit / deepkit-ml Goto Github PK

The collaborative real-time open-source machine learning devtool and training suite: Experiment execution, tracking, and debugging. With server and project management tools.

Home Page: https://ml.deepkit.io

License: MIT License

Dockerfile 0.15% Makefile 0.78% Objective-C 0.56% Shell 1.09% Batchfile 0.01% JavaScript 1.34% TypeScript 86.44% CSS 0.83% HTML 3.96% Python 0.10% SCSS 4.74%

machine-learning deep-learning experiments gui desktop artificial-intelligence devtools

deepkit-ml's Introduction

Deepkit

Deepkit is an open-source platform and cross-platform desktop application to execute, track, and debug modern machine learning experiments. With integrated server management tool to execute experiments on any Linux server with a simple click or cli command.

Download - Example Python

Features

Experiment execution using uniform experiment definition
Execute experiments locally or in the cloud, seamlessly
Pipeline support to increase execution time and reduce hardware costs
Layer debugger for Keras2 and Pytorch
Automatic source code versioning
Experiment artifacts (aka output: weights, snapshots, etc)
Experiment execution via Docker container with auto-build
Experiment comparison: Side by side, file diff, metrics diff
Server job scheduling with resource limitation and assignment (and NVIDIA Docker GPU support)
Real-time experiment tracking: Log metrics, text, images, numpy arrays, and more
Python SDK
Project management: Issue tracker, Kanban board, and notes
Real-time desktop (and web) app with analytical tools to monitor and organize experiments
Optional centralized server for teams

Development

Deepkit is written entirely in Typescript and consists of following main packages:

packages/cli: command line interface tools
packages/deepkit: The GUI, desktop and web app interface
packages/server: The server component that stores all the data using the filesystem and mongodb
packages/website: The website as SSR Angular 9 SPA app
packages/core: Common models, functions, services
packages/core-node: Common models, functions, services for node environment
packages/electron: The electron bootstrapper for the GUI

Following open-source packages have been created for Deepkit that wouldn't exist otherwise and were absolutely necessary:

Marshal.ts - The fastest universal Typescript data serializer and validator
Glut.ts - A reactive real-time server framework with distributed entity/file abstraction and syncing based on ReactJS
angular-desktop-ui - A GUI library for native-like GUI widget based on Angular 9+

Development app

cd deepkit
npm run bootstrap

cd deepkit
npm-local-development

cd deepkit
make angular-watch

cd deepkit
server-watch

# Open in browser the angular dist
# e.g. file:///Users/marc/bude/deepkit/packages/deepkit/dist/deepkit/index.html
# or electron
cd deepkit
make electron-build
make electron

Development website

Watcher

cd packages/website
ng run website:server:production --watch

Server

cd packages/website
npm run serve:ssr

deepkit-ml's People

Contributors

Stargazers

Watchers

deepkit-ml's Issues

Experiment log output cannot be copied on Windows

The STDOUT output of experiments cannot be copied (either by right clicking -> no menu) or by Ctrl-C. This would be handy for bug tracking/reporting.

Windows 10
Deepkit Release 2020.1.5

Registering with Keras fails

I have tried several combinations (Docker and other PC and environment), but the following fails

import deepkit
...
experiment = deepkit.experiment()
experiment.watch_keras_model(model)
deepkit_callback = experiment.create_keras_callback(model)
...
model.fit(generator,epochs=90, callbacks=[deepkit_callback])

with

Traceback (most recent call last):
  File "forecast-with-lstm.py", line 58, in <module>
    model.fit(generator,epochs=90,callbacks=[deepkit_callback])
  File "/usr/local/lib/python3.8/site-packages/deepkit/experiment.py", line 532, in fit
    debugger.set_input(x)
  File "/usr/local/lib/python3.8/site-packages/deepkit/keras_tf.py", line 433, in set_input
    self.model_input = np.array([x[0]] if self.is_batch else x)
ValueError: could not broadcast input array from shape (6,12,1) into shape (6)

Running Python 3.8.3 (on OSX) and these dependencies from requirements.txt:

deepkit
pandas
matplotlib
statsmodels
sklearn
keras
scipy==1.4.1
tensorflow

Any clue of what goes wrong?

Docker run error

I tried to use the deepkit with clusters. After configuring the deepkit.yml as follows:

image: xxx
command: 
    - echo hell

I run it on the server, but got the error:

Success fully built 47df7b7037f5
Successfully tagged deepkit_ test_ 3980a55c32a6d7a01ac7 7537ff8495dc: latest
start docker deepkit test 3980a5 5c32a6d7a01ac77537ff8495dc : echo hell
run
failed Error: ( HTTP code 400 ) unexpected
invalid environment variable :
( HTTP code 400 )
unexpected
invalid environment variable:

Can u help me to figure out how to fix it?

please windows version ?

I want it very much.

How about the project activity state?

Hello, I found this wonderful project.
The repository seems no active activity from 3 years. Is it abandoned?

How about the future plan of this project? Is there any difficulty?
It seems only one people in the deepkit ORG.Are you too busy to maintain this project?

Dark theme not available on Linux

deepkit link error

I have installed deepkit using snap in ubuntu and created folder as shown in the website(getting started), but when I am trying "deepkit link" an error is showing like

Error: No account found for localhost.
at Je.getLocalAccount (/snap/deepkit/12/resources/app/deepkit-cli/main.js:441:216380)
at /snap/deepkit/12/resources/app/deepkit-cli/main.js:441:345337
at async Ai.run (/snap/deepkit/12/resources/app/deepkit-cli/main.js:441:345279)
at async Ai._run (/snap/deepkit/12/resources/app/deepkit-cli/main.js:174:138701)
at async h.runCommand (/snap/deepkit/12/resources/app/deepkit-cli/main.js:174:89044)
at async i.run (/snap/deepkit/12/resources/app/deepkit-cli/main.js:174:210359)
at async i._run (/snap/deepkit/12/resources/app/deepkit-cli/main.js:174:138701)

Application hangs after project deletion

The application fully not responsible after project delete action and can be the only killed

video_recording.zip

Invalid yaml causes GUI to freeze

If yaml in deepkit.yml contains syntax errors, the GUI freezes when trying to start a new experiment and displays "Loading.." while becoming unresponsive. Closing and restarting fixes this, but it would be nice to get a more subtle error message that was less of an interruption to workflows.

Passing additional options to docker

Natively, Docker accepts a a long list of options when running containers where the CLI syntax looks like docker run [OPTIONS] IMAGE [COMMAND] [ARG...]. It doesn't appear that this is possible in DeepKit.

Would it be possible to add options passing in the config for jobs?

It is often the case that system specific options are needed to run containers and allow access to things like networking on the host machine. The ability to add a flag such as --network=host may be the difference between being able to use DeepKit at all with some networking setups.

Naming of "Default" experiment list

I found it misleading that the default list is actually called "default" but appears written "Default" - when I wanted to actively place my experiment there using the yml's list: Default syntax, DeepKit ended up generating a duplicate list called "Default".

Is it possible to download ?

There is no code\binaries to download ?

Automatic provisioning of VM instances

Checking the configuration (deepkit.yml) documentation I do not see an appropriate method to provision VM instances (E.g: MS Azure) as a node through a REST API.

A current solution is to:

Start a DL VM instance and provision the instance through deepkit as node.
Start an experiment.
When the experiment had ended or shut down per user request stop the instance.

However this does not prevent unnecessary idle time of the instance which will add up to the costs if one does not stop the instance directly after an experiment.

Preferred functionality would be:

Define a target VM instance within deepkit.yml experiment file through API.
Let deepkit start the instance, provision it automatically when the experiment has been started by the user.
When the experiment has ended stop the the instance.

Example config file:

vm: <API: start the instance>
image: tensorflow/tensorflow:1.15.2-gpu-py3
command: python model.py
vm_post: <API: stop the instance>

If you have a different work around in place I would be happy to hear about it!

Is it possible to run this without docker installed on the server?

I dont have the ability to install docker on my server. Can I still use deepkit? It fits everything I need otherwise.

Docker Mounts

I have tried the most recent version of DeepKit and cant get Docker mounts to work. I go to the node where I want to use them, change the settings in mounts to /home/anthony/data:/data, this works locally but using DeepKit this mount does not appear.

I get the error:

run failed Error: (HTTP code 400) unexpected - invalid environment variable:  
(HTTP code 400) unexpected - invalid environment variable:

Links in README are no longer working

https://deepkit.ai/ is redirected to https://appmaster.io/

Scalar plots are empty, log_scalar values only appear on experiment overview

Plots of any log_metric are empty, though the latest logged scalars do appear in the experiment overview table and the compare panel as single float values.

Setup:

Windows 10
Deepkit Release 2020.1.5
Deepkit SDK 1.0.1
Python 3.7.x (running local or remote)

Sample per second metric always 0

[Feature request] Distributed training

Due to the fact that cluster entity already in place, can you please add Distributed training across all cluster nodes as described below:
https://lambdalabs.com/blog/tensorflow-2-0-tutorial-05-distributed-training-multi-node/

Project name of "deepkit" changes nested directory behavior

A logical name to keep DeepKit configuration files within a project would be nested in a subdirectory called deepkit. However, using this convention on a DeepKit server causes inconsistency with deepkit.yml config values.

Server Example

For example, say we're training a model in a parent repo of https://github.com/username/MyNetwork.git

If we create a directory ./deepkit and put deepkit.yml there, DeepKit can't seem to find the configuration. Notice the [object Object] placeholder value.

./MyNetwork
└──  deepkit
    └── deepkit.yml

But if we duplicate the same deepkit.yml file and nest it under ./deepkit/train, DeepKit is able to find the config

./MyNetwork
└──  deepkit
    └── train
        └── deepkit.yml

GUI Example

This also happens when using the GUI on localhost to manage a Project - but it is affected one level higher in the directory structure (so that the [object Object] placeholder value appears if ./deepkit.yml is in the root of the project:

./MyNetwork
└──  deepkit.yml

Summary

One solution is to make sure that DeepKit configuration files are not organized under a directory named deepkit . However, unless the directory name deepkit needs to be reserved, it might cause less confusion to correct this behavior so that experiments are not affected by this.

Is this alive?

Docker build fails

Hello,

I'm trying to build the docker locally, but it fails on leana:

Step 14/29 : RUN cd /app && npm run bootstrap
 ---> Running in acaa693b066c

> root@ bootstrap /app
> lerna bootstrap --nohoist \* --no-ci

lerna notice cli v3.22.1
lerna info Bootstrapping 4 packages
lerna info Installing external dependencies
lerna ERR! npm install exited 1 in '@deepkit/server'
lerna ERR! npm install stdout:

> [email protected] install /app/packages/server/node_modules/bson-ext
> (node-gyp rebuild 2> builderror.log) || (exit 0)


> [email protected] install /app/packages/server/node_modules/nodegit
> node lifecycleScripts/preinstall && node lifecycleScripts/install

[nodegit] Running pre-install script
[nodegit] Running install script
Failed to execute '/usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/app/packages/server/node_modules/nodegit/build/Release/nodegit.node --module_name=nodegit --module_path=/app/packages/server/node_modules/nodegit/build/Release --napi_ver
sion=4 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v67' (1)

lerna ERR! npm install stderr:
node-pre-gyp
WARN Using request for node-pre-gyp https download
node-pre-gyp
WARN Tried to download(404): https://axonodegit.s3.amazonaws.com/nodegit/nodegit/nodegit-v0.26.4-node-v67-linux-x64.tar.gz
node-pre-gyp WARN Pre-built binaries not found for [email protected] and [email protected] (node-v67 ABI, musl) (falling back to source compile with node-gyp)

/bin/sh: krb5-config: not found
gyp: Call to 'krb5-config gssapi --libs' returned exit status 127 while in binding.gyp. while trying to load binding.gyp
gyp

ERR! configure error

gyp
ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack
at ChildProcess.onCpExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:345:16)
gyp ERR!
stack     at ChildProcess.emit (events.js:193:13)
gyp ERR!
stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:255:12)
gyp
ERR! System Linux 4.15.0-99-generic
gyp
ERR! command
"/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--module=/app/packages/server/node_modules/nodegit/build/Release/nodegit.node" "--module_name=nodegit" "--module_path=/app/packages/server/node_modules/nodegit/build/Release" "--napi_version=4" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v67"
gyp ERR! cwd
/app/packages/server/node_modules/nodegit
gyp ERR!
node -v v11.15.0
gyp
ERR! node-gyp -v v3.8.0
....

Apparently this:
WARN Tried to download(404): https://axonodegit.s3.amazonaws.com/nodegit/nodegit/nodegit-v0.26.4-node-v67-linux-x64.tar.gz

is missing.

Any ideas?

Thanks!

Job Failure

I tried to run a job using deepkit and it never starts up, I get the bellow error on the node:

Start job task 127.0.0.1 6c49f921-fa20-476e-aac3-cbb8821923d9 main 0
Job errored 6c49f921-fa20-476e-aac3-cbb8821923d9 main 0 Error: Command failed with exit code 1 (EPERM): /home/anthony/deepkit/cli/bin/node /home/anthony/deepkit/cli/main.js start 127.0.0.1 8961 6c49f921-fa20-476e-aac3-cbb8821923d9 81cb8bd7-2bcf-4219-913c-c3ddb668b72c main 0 eyJlbnYiOltdLCJkb2NrZXJCaW5kcyI6W10sImhvc3RFeGVjdXRpb25BbGxvd2VkIjpmYWxzZSwiY3VzdG9tTW91bnRzQWxsb3dlZCI6ZmFsc2V9
    at e.exports (/home/anthony/deepkit/cli/main.js:180:176343)
    at /home/anthony/deepkit/cli/main.js:174:73052
    at processTicksAndRejections (internal/process/task_queues.js:85:5) {
  command: '/home/anthony/deepkit/cli/bin/node /home/anthony/deepkit/cli/main.js start 127.0.0.1 8961 6c49f921-fa20-476e-aac3-cbb8821923d9 81cb8bd7-2bcf-4219-913c-c3ddb668b72c main 0 eyJlbnYiOltdLCJkb2NrZXJCaW5kcyI6W10sImhvc3RFeGVjdXRpb25BbGxvd2VkIjpmYWxzZSwiY3VzdG9tTW91bnRzQWxsb3dlZCI6ZmFsc2V9',
  exitCode: 1,
  exitCodeName: 'EPERM',
  stdout: "args['start-config-base64'] eyJlbnYiOltdLCJkb2NrZXJCaW5kcyI6W10sImhvc3RFeGVjdXRpb25BbGxvd2VkIjpmYWxzZSwiY3VzdG9tTW91bnRzQWxsb3dlZCI6ZmFsc2V9\n" +
    'jobFiles /home/anthony/.deepkit/cli/job/6c49f921-fa20-476e-aac3-cbb8821923d9/instance-checkout/main/0\n' +
    'Checkout files',
  stderr: '(node:11487) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.\n' +
    'Error: unknown operator: $or\n' +
    '    at Object.t.getUnserializedError (~/deepkit/cli/main.js:50:70725)\n' +
    '    at t._next (~/deepkit/cli/main.js:174:277534)\n' +
    '    at t.__tryOrUnsub (~/deepkit/cli/main.js:16:61380)\n' +
    '    at t.next (~/deepkit/cli/main.js:16:60525)\n' +
    '    at t._next (~/deepkit/cli/main.js:16:59593)\n' +
    '    at t.next (~/deepkit/cli/main.js:16:59264)\n' +
    '    at c.t.next (~/deepkit/cli/main.js:16:74263)\n' +
    '    at Object.replies.<computed> (~/deepkit/cli/main.js:174:267843)\n' +
    '    at t.SocketClient.onDecodedMessage (~/deepkit/cli/main.js:174:263759)\n' +
    '    at t.Batcher.messageComplete (~/deepkit/cli/main.js:180:6014)\n' +
    '    at ORIGIN (action job.getJobFiles)\n' +
    '    at e.exports.<anonymous> (/app/main.js:2:1330616)\n' +
    '    at A (/app/main.js:2:1322204)\n' +
    '    at Socket.<anonymous> (/app/main.js:2:1325896)\n' +
    '    at Object.t.asyncOperation (~/deepkit/cli/main.js:113:127820)\n' +
    '    at t.SocketClient.stream (~/deepkit/cli/main.js:174:266228)\n' +
    '    at Proxy.<anonymous> (~/deepkit/cli/main.js:174:263459)\n' +
    '    at ~/deepkit/cli/main.js:441:344226',
  all: '(node:11487) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.\n' +
    "args['start-config-base64'] eyJlbnYiOltdLCJkb2NrZXJCaW5kcyI6W10sImhvc3RFeGVjdXRpb25BbGxvd2VkIjpmYWxzZSwiY3VzdG9tTW91bnRzQWxsb3dlZCI6ZmFsc2V9\n" +
    'jobFiles /home/anthony/.deepkit/cli/job/6c49f921-fa20-476e-aac3-cbb8821923d9/instance-checkout/main/0\n' +
    'Checkout files\n' +
    'Error: unknown operator: $or\n' +
    '    at Object.t.getUnserializedError (~/deepkit/cli/main.js:50:70725)\n' +
    '    at t._next (~/deepkit/cli/main.js:174:277534)\n' +
    '    at t.__tryOrUnsub (~/deepkit/cli/main.js:16:61380)\n' +
    '    at t.next (~/deepkit/cli/main.js:16:60525)\n' +
    '    at t._next (~/deepkit/cli/main.js:16:59593)\n' +
    '    at t.next (~/deepkit/cli/main.js:16:59264)\n' +
    '    at c.t.next (~/deepkit/cli/main.js:16:74263)\n' +
    '    at Object.replies.<computed> (~/deepkit/cli/main.js:174:267843)\n' +
    '    at t.SocketClient.onDecodedMessage (~/deepkit/cli/main.js:174:263759)\n' +
    '    at t.Batcher.messageComplete (~/deepkit/cli/main.js:180:6014)\n' +
    '    at ORIGIN (action job.getJobFiles)\n' +
    '    at e.exports.<anonymous> (/app/main.js:2:1330616)\n' +
    '    at A (/app/main.js:2:1322204)\n' +
    '    at Socket.<anonymous> (/app/main.js:2:1325896)\n' +
    '    at Object.t.asyncOperation (~/deepkit/cli/main.js:113:127820)\n' +
    '    at t.SocketClient.stream (~/deepkit/cli/main.js:174:266228)\n' +
    '    at Proxy.<anonymous> (~/deepkit/cli/main.js:174:263459)\n' +
    '    at ~/deepkit/cli/main.js:441:344226',
  failed: true,
  timedOut: false,
  isCanceled: false,
  killed: false,
  signal: undefined