unable to generate new projects and admin credentials when running a local server about clearml-server HOT 6 CLOSED

Hi @amirkforesightauto , it seems your web-browser cannot reach the API server (hence the "X Fetching projects failed X" error messages)

There are currently two optional deployments:

• Single IP ports: 8080/8008/8081 for Web/API/File-storage
• Subdomain deployment: app.* / api.* / files.*
  (all subdomains use either http or https, mixing is not supported)

What is your specific setup?

from clearml-server.

I having difficulty understanding what you mean by the above.
I followed the step by step instruction at
https://github.com/allegroai/trains-server
"After the trains-server Dockers are up, the following are available:

API server on port 8008
Web server on port 8080
File server on port 8081

"

from clearml-server.

Try these instructions for stand alone Linux Ubuntu installation:

• Install docker https://docs.docker.com/install/linux/docker-ce/ubuntu/

• Install docker-compose https://docs.docker.com/compose/install/:

sudo curl -L "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

• Cleanup previous installations (Notice! this will clear all previous TRAINS databases)

$ sudo rm -R /opt/trains/

• Create local directories for DB & storage

$ sudo mkdir -p /opt/trains/data/elastic
$ sudo mkdir -p /opt/trains/data/mongo/db
$ sudo mkdir -p /opt/trains/data/mongo/configdb
$ sudo mkdir -p /opt/trains/logs
$ sudo mkdir -p /opt/trains/data/fileserver
$ sudo chown -R $(whoami):$(whoami) /opt/trains

• Clone the trains-server repository

$ git clone https://github.com/allegroai/trains-server.git
$ cd trains-server

• Run docker-compose with the unified docker
  Note: This is the only command you will have to repeat after rebooting your machine

$ HOSTIP=$(ip route get 8.8.8.8 | head -1 | cut -d' ' -f8) /usr/local/bin/docker-compose -f docker-compose-unified.yml up

You should now have a server running on http://localhost:8080 🤞

from clearml-server.

I still get the same error as before.
After writing the last line I get the following output:

trains-server | Loading config from /opt/trains/fileserver/config/default
trains-server | Loading config from file /opt/trains/fileserver/config/default/logging.conf
trains-server | Loading config from file /opt/trains/fileserver/config/default/fileserver.conf
trains-server | Loading config from /opt/trains/config
trains-server | * Serving Flask app "fileserver" (lazy loading)
trains-server | * Environment: production
trains-server | WARNING: This is a development server. Do not use it in a production deployment.
trains-server | Use a production WSGI server instead.
trains-server | * Debug mode: off
trains-server | [2019-07-30 06:48:52,239] [26] [INFO] [trains.schema] loading schema from cache
trains-server | [2019-07-30 06:48:52,245] [26] [INFO] [trains.es_factory] Using override elastic host uid
trains-server | [2019-07-30 06:48:52,247] [26] [INFO] [trains.server] ################ API Server initializing #####################
trains-server | [2019-07-30 06:48:52,247] [26] [INFO] [trains.database] Initializing database connections
trains-server | [2019-07-30 06:48:52,247] [26] [INFO] [trains.database] Using override mongodb host uid
trains-server | [2019-07-30 06:48:52,248] [26] [INFO] [trains.database] Registering connection to auth-db (mongodb://uid:27017/auth)
trains-server | [2019-07-30 06:48:52,249] [26] [INFO] [trains.database] Registering connection to backend-db (mongodb://uid:27017/backend)
trains-server | [2019-07-30 06:48:52,250] [26] [INFO] [trains.init_data] Applying mappings to host: http://uid:9200
trains-server | [2019-07-30 06:48:52,257] [26] [WARNING] [urllib3.connectionpool] Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3ba76e8470>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /_template/events
trains-server | [2019-07-30 06:48:53,262] [26] [WARNING] [urllib3.connectionpool] Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3ba76e8748>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /_template/events
trains-server | [2019-07-30 06:48:55,267] [26] [WARNING] [urllib3.connectionpool] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3ba76e88d0>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /_template/events
trains-server | 172.18.0.1 - - [30/Jul/2019:06:48:56 +0000] "GET /plus.7098a939feb676c834dc.svg HTTP/1.1" 200 182 "http://localhost:8080/dashboard" "Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
trains-server | 172.18.0.1 - - [30/Jul/2019:06:48:58 +0000] "GET /plus-hover.322712abe2971266ec37.svg HTTP/1.1" 200 182 "http://localhost:8080/dashboard" "Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0" "-"
trains-server | [2019-07-30 06:48:59,273] [26] [WARNING] [urllib3.connectionpool] Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3ba76e8a58>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /_template/events
trains-server | [2019-07-30 06:49:07,285] [26] [WARNING] [urllib3.connectionpool] Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3ba76e8c18>: Failed to establish a new connection: [Errno -2] Name or service not known',)': /_template/events
trains-server | Loading config from /opt/trains/server/config/default
trains-server | Loading config from file /opt/trains/server/config/default/apiserver.conf
trains-server | Loading config from file /opt/trains/server/config/default/logging.conf
trains-server | Loading config from file /opt/trains/server/config/default/secure.conf
trains-server | Loading config from file /opt/trains/server/config/default/hosts.conf
trains-server | Loading config from file /opt/trains/server/config/default/services/tasks.conf
trains-server | Loading config from file /opt/trains/server/config/default/services/events.conf
trains-server | Loading config from /opt/trains/config
trains-server | Traceback (most recent call last):
trains-server | File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn
trains-server | (self._dns_host, self.port), self.timeout, **extra_kw)
trains-server | File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 57, in create_connection
trains-server | for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
trains-server | File "/usr/lib64/python3.6/socket.py", line 745, in getaddrinfo
trains-server | for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
trains-server | socket.gaierror: [Errno -2] Name or service not known
trains-server |
trains-server | During handling of the above exception, another exception occurred:
trains-server |
trains-server | Traceback (most recent call last):
trains-server | File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 603, in urlopen
trains-server | chunked=chunked)
trains-server | File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 355, in _make_request
trains-server | conn.request(method, url, **httplib_request_kw)
trains-server | File "/usr/lib64/python3.6/http/client.py", line 1239, in request
trains-server | self._send_request(method, url, body, headers, encode_chunked)
trains-server | File "/usr/lib64/python3.6/http/client.py", line 1285, in _send_request
trains-server | self.endheaders(body, encode_chunked=encode_chunked)
trains-server | File "/usr/lib64/python3.6/http/client.py", line 1234, in endheaders
trains-server | self._send_output(message_body, encode_chunked=encode_chunked)
trains-server | File "/usr/lib64/python3.6/http/client.py", line 1026, in _send_output
trains-server | self.send(msg)
trains-server | File "/usr/lib64/python3.6/http/client.py", line 964, in send
trains-server | self.connect()
trains-server | File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 183, in connect
trains-server | conn = self._new_conn()
trains-server | File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 169, in _new_conn
trains-server | self, "Failed to establish a new connection: %s" % e)
trains-server | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f3ba76e8dd8>: Failed to establish a new connection: [Errno -2] Name or service not known
trains-server |
trains-server | During handling of the above exception, another exception occurred:
trains-server |
trains-server | Traceback (most recent call last):
trains-server | File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
trains-server | timeout=timeout
trains-server | File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 670, in urlopen
trains-server | **response_kw)
trains-server | File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 670, in urlopen
trains-server | **response_kw)
trains-server | File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 670, in urlopen
trains-server | **response_kw)
trains-server | [Previous line repeated 2 more times]
trains-server | File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 641, in urlopen
trains-server | _stacktrace=sys.exc_info()[2])
trains-server | File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 399, in increment
trains-server | raise MaxRetryError(_pool, url, error or ResponseError(cause))
trains-server | urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='uid', port=9200): Max retries exceeded with url: /_template/events (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3ba76e8dd8>: Failed to establish a new connection: [Errno -2] Name or service not known',))
trains-server |
trains-server | During handling of the above exception, another exception occurred:
trains-server |
trains-server | Traceback (most recent call last):
trains-server | File "server.py", line 31, in
trains-server | init_es_data()
trains-server | File "/opt/trains/server/init_data.py", line 34, in init_es_data
trains-server | res = apply_mappings_to_host(host)
trains-server | File "/opt/trains/server/elastic/apply_mappings.py", line 37, in apply_mappings_to_host
trains-server | _send_mapping(f) for f in p.iterdir() if f.is_file() and f.suffix == ".json"
trains-server | File "/opt/trains/server/elastic/apply_mappings.py", line 37, in
trains-server | _send_mapping(f) for f in p.iterdir() if f.is_file() and f.suffix == ".json"
trains-server | File "/opt/trains/server/elastic/apply_mappings.py", line 27, in _send_mapping
trains-server | session.delete(url)
trains-server | File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 615, in delete
trains-server | return self.request('DELETE', url, **kwargs)
trains-server | File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
trains-server | resp = self.send(prep, **send_kwargs)
trains-server | File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
trains-server | r = adapter.send(request, **kwargs)
trains-server | File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
trains-server | raise ConnectionError(e, request=request)
trains-server | requests.exceptions.ConnectionError: HTTPConnectionPool(host='uid', port=9200): Max retries exceeded with url: /_template/events (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f3ba76e8dd8>: Failed to establish a new connection: [Errno -2] Name or service not known',))
trains-server exited with code 1

from clearml-server.

With the help of dear friend we got to the following solution:
"Instructions for stand alone Linux Ubuntu installation:

Install docker https://docs.docker.com/install/linux/docker-ce/ubuntu/

Install docker-compose https://docs.docker.com/compose/install/:

sudo curl -L "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

sudo chmod +x /usr/local/bin/docker-compose

Cleanup previous installations (Notice! this will clear all previous TRAINS databases)

$ sudo rm -R /opt/trains/

Create local directories for DB & storage

$ sudo mkdir -p /opt/trains/data/elastic

$ sudo mkdir -p /opt/trains/data/mongo/db

$ sudo mkdir -p /opt/trains/data/mongo/configdb

$ sudo mkdir -p /opt/trains/logs

$ sudo mkdir -p /opt/trains/data/fileserver

$ sudo chown -R $(whoami):$(whoami) /opt/trains

Clone the trains-server repository

$ git clone https://github.com/allegroai/trains-server.git

$ cd trains-server

Find which internal ip is available in the docker :
$ifconfig
............................
...... inet 172.18.0.1................

Run docker-compose with the unified docker with the internal ip address:

HOSTIP=172.18.0.1 /usr/local/bin/docker-compose -f docker-compose-unified.yml up

from clearml-server.

@amirkforesightauto great news!
I'll make sure we add it to the FAQ :)
Closing issue

from clearml-server.