I'm not super experienced in docker but was wondering where the location of the logfiles are in the container? What I'm looking for is to be able to get whatever output I get from docker logs into a file, which I can then map to a volume so I can monitor it from outside the container.

From poking around I can see that there's /data/hist/LOG and /data/utxo/LOG, but neither of those seem to be what I get from docker logs, which looks like this:

INFO:Prefetcher:new block height 510,317 hash 000000000000000000551fc6797a215b85d73e1720a67f64ab726bb5d53543b7
INFO:BlockProcessor:flush #499 took 0.0s.  Height 510,317 txs: 300,813,334
INFO:BlockProcessor:processed 1 block in 1.4s
INFO:Controller:ID     Flags            Client Proto  Reqs   Txs    Subs    Recv Recv KB    Sent Sent KB      Time                  Peer
INFO:Controller:{"daemon": "10.0.0.1:8332/", "daemon_height": 510317, "db_height": 510317, "closing": 0, "errors": 0, "groups": 0, "logged": 0, "paused": 0, "pid": 1, "peers": {"bad": 0, "good": 0, "never": 13, "stale": 0, "total": 13}, "requests": 0, "sessions": 0, "subs": 0, "txs_sent": 0, "uptime": "22h 26m 56s"}
INFO:MemPool:3,454 txs touching 41,798 addresses
INFO:MemPool:3,753 txs touching 43,869 addresses
INFO:MemPool:4,044 txs touching 44,715 addresses
INFO:MemPool:4,376 txs touching 45,914 addresses
INFO:MemPool:4,654 txs touching 46,797 addresses
INFO:MemPool:4,921 txs touching 47,657 addresses
INFO:MemPool:5,187 txs touching 49,301 addresses
INFO:MemPool:5,440 txs touching 50,289 addresses
INFO:MemPool:5,661 txs touching 51,034 addresses
INFO:MemPool:5,882 txs touching 51,787 addresses
INFO:MemPool:6,148 txs touching 52,682 addresses
INFO:MemPool:6,412 txs touching 53,832 addresses
INFO:MemPool:6,685 txs touching 54,814 addresses
INFO:MemPool:6,947 txs touching 55,803 addresses
INFO:MemPool:7,181 txs touching 56,555 addresses
INFO:Prefetcher:new block height 510,318 hash 000000000000000000020172d45a5714ff668ae93addce97139922c94dceafa3
INFO:MemPool:5,251 txs touching 48,569 addresses
INFO:BlockProcessor:flush #500 took 0.1s.  Height 510,318 txs: 300,815,511
INFO:BlockProcessor:processed 1 block in 1.6s
INFO:Prefetcher:new block height 510,319 hash 0000000000000000000eb8583d49dd419d71979ac14d387ed85c1be00e3bbe75

Cheers

Could you supply simple instructions on how a user of lukechilds/electrumx can execute the compact_history command?

I've been using the lukechilds/electrumx docker image successfully for many months. However recently my ElectrumX server raises exceptions as reported in kyuupichan/electrumx#185

INFO:BlockProcessor:our height: 657,506 daemon: 658,515 UTXOs 266MB hist 386MB
INFO:BlockProcessor:processed 16 blocks size 20.12 MB in 8.2s
INFO:BlockProcessor:processed 8 blocks size 10.87 MB in 4.5s
INFO:BlockProcessor:processed 15 blocks size 18.34 MB in 7.3s
INFO:BlockProcessor:processed 17 blocks size 20.65 MB in 9.2s
INFO:BlockProcessor:our height: 657,563 daemon: 658,515 UTXOs 306MB hist 449MB
INFO:DB:flushed filesystem data in 0.03s
INFO:Prefetcher:cancelled; prefetcher stopping
INFO:BlockProcessor:flushing to DB for a clean shutdown...
INFO:DB:flushed filesystem data in 0.00s
INFO:SessionManager:closing down server for rpc://0.0.0.0:8000
ERROR:root:task crashed: <Task finished coro=<Controller.serve() done, defined at /electrumx/electrumx/server/controller.py:81> exception=error("'H' format requires 0 <= number <= 65535")>
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/aiorpcX-0.18.3-py3.7.egg/aiorpcx/util.py", line 309, in check_task
    task.result()
  File "/electrumx/electrumx/server/controller.py", line 134, in serve
    await group.spawn(wait_for_catchup())
  File "/usr/local/lib/python3.7/site-packages/aiorpcX-0.18.3-py3.7.egg/aiorpcx/curio.py", line 242, in __aexit__
    await self.join()
  File "/usr/local/lib/python3.7/site-packages/aiorpcX-0.18.3-py3.7.egg/aiorpcx/curio.py", line 211, in join
    raise task.exception()
  File "/electrumx/electrumx/server/block_processor.py", line 669, in fetch_and_process_blocks
    await self.flush(True)
  File "/electrumx/electrumx/server/block_processor.py", line 342, in flush
    await self.run_in_thread_with_lock(flush)
  File "/electrumx/electrumx/server/block_processor.py", line 198, in run_in_thread_with_lock
    return await asyncio.shield(run_in_thread_locked())
  File "/electrumx/electrumx/server/block_processor.py", line 197, in run_in_thread_locked
    return await run_in_thread(func, *args)
  File "/usr/local/lib/python3.7/site-packages/aiorpcX-0.18.3-py3.7.egg/aiorpcx/curio.py", line 68, in run_in_thread
    return await get_event_loop().run_in_executor(None, func, *args)
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/electrumx/electrumx/server/block_processor.py", line 341, in flush
    self.estimate_txs_remaining)
  File "/electrumx/electrumx/server/db.py", line 210, in flush_dbs
    self.flush_history()
  File "/electrumx/electrumx/server/db.py", line 279, in flush_history
    self.history.flush()
  File "/electrumx/electrumx/server/history.py", line 133, in flush
    flush_id = pack_be_uint16(self.flush_count)
struct.error: 'H' format requires 0 <= number <= 65535

I understand from the aforementioned bug that I'm supposed to run something like this:

$ docker-compose exec electrumx electrumx_compact_history

However, when I try to run docker-compose, it complains thus:

ERROR: 
        Can't find a suitable configuration file in this directory or any
        parent. Are you in the right directory?

        Supported filenames: docker-compose.yml, docker-compose.yaml

I am a novice at docker, this is the only container that I run.

Hello,

Thanks for a great Docker image.

Upon startup I see the following in the logs:

WARNING lowered maximum sessions from 1,000 to 674 because your open file limit is 1,024  [util.py:60]

This brings to mind, from ElectrumX (https://github.com/kyuupichan/electrumx/blob/master/docs/HOWTO.rst#process-limits):

You must ensure the ElectrumX process has a large open file limit. During sync it should not need more than about 1,024 open files. When serving it will use approximately 256 for LevelDB plus the number of incoming connections. It is not unusual to have 1,000 to 2,000 connections being served, so I suggest you set your open files limit to at least 2,500.

Note that setting the limit in your shell does NOT affect ElectrumX unless you are invoking ElectrumX directly from your shell. If you are using :command:systemd, you need to set it in the :file:.service file (see contrib/systemd/electrumx.service).

1. Is there a specific way anyone else here has raised that limit for themselves?
2. Would this be considered a bug? If so, would it be easily fixable nearly universally, and therefore a good candidate for a pull request?

Thanks,
James

Please let me know, Ill attempt to do on my own if you have no time. Thank you!

Hi, I tried a lot of stuff now to get this running.

I have bitcoind installed, synced, reindexed and want to run electrumx on the same server.
Everytime I try to run the docker command it seems to spin up the server and then just forever say

ERROR:Daemon:connection problem - is your daemon running? Retrying occasionally...

So it can somehow not connect to bitcoind, right?

I have configured a rpc user and password there and included this in the docker commands variables.
I am running the docker command like this:

docker run -v /mnt/bitcoinserver2/electrumx:/data -e DAEMON_URL=http://coindancer:[email protected] -e COIN=BitcoinSegwit -p 50002:50002 elec:stream

(elec:stream are the name and tag of the image that I built. It also does not work with lukechilds/electrumx)

Everything is working now except for the banner file.
this is what I type:
sudo docker run --net=host -v /media/user/hd/electrumx/data:/data -e DAEMON_URL="username:[email protected]:8332" -e BANNER_FILE="/home/user/banner.txt" -e REPORT_HOST_TOR="address.onion" -e COIN=BitcoinSegwit -p 50002:50002 lukechilds/electrumx:v1.8.7
and the error I get is ERROR:ElectrumX:[0] reading banner file /home/user/banner.txt: FileNotFoundError(2, 'No such file or directory')`
but I'm pretty sure the file is there.
Thanks for the help!

hi, bitcoin core is sync but electrumx take a long time. What i can do. Thanks.

this is my docker file
services: electrumx: image: lukechilds/electrumx restart: unless-stopped container_name: electrumx volumes: - ./data:/data environment: - DAEMON_URL=http://user:[email protected]:8332 - COIN=BitcoinSegwit - PEER_DISCOVERY=on - MAX_SUBS=500000 - MAX_SESSION_SUBS=100000 ports: - 50002:50002 - 5000:5000

I'm running Electrum 3.2.4 (also tried 3.3.8) on a fully up to date Ubuntu 18.04 using this command:

electrum/electrum gui -v --oneserver --server <myhost>:50002:s

I'm getting this output in the console, the connection is immediately dropped and my wallets are not updated:

[Network] connecting to <myhost>:50002:s as new interface
[myhost] connected
[Network] switching to <myhost>:50002:s
[Network] sending subscriptions to <myhost>:50002:s 26 0
[myhost] connection closed remotely
[profiler] Standard_Wallet.get_full_history 0.0007
[profiler] HistoryList.on_update 0.0011

In electrumx-docker, I only see

electrumx_1  | INFO:ElectrumX:[23] SSL <myip>:34316, 0 total

sometimes (but not always) followed by

electrumx_1  | INFO:ElectrumX:[5] changing task concurrency from 10 to 6
electrumx_1  | INFO:ElectrumX:[5] disconnected whilst throttled

I can

$ telnet <myhost> 50002
Trying <myip>...
Connected to <myhost>.
Escape character is '^]'.

or

$ openssl s_client -connect <myhost>:50002
CONNECTED(00000005)
depth=0 O = ElectrumX
verify error:num=18:self signed certificate
verify return:1
depth=0 O = ElectrumX
verify return:1
[...]

and is stays connected for a while.

Both bitcoind and electrumx are fully synched, earlier in the log I see

electrumx_1  | INFO:SessionManager:SSL server listening on electrumx:50002

My host is a fully up to date Ubuntu Server 18.04. This is my docker-electrumx config:

$ cat docker-compose.yaml 
version: "3"
services:
  bitcoind:
    image: nicolasdorier/docker-bitcoin:0.18.1
    volumes:
      - "<mybitcoinddata>:/data"
    ports:
      - "<myip>:8333:8333"
    environment:
      BITCOIN_EXTRA_ARGS: |
        txindex=1
        disablewallet=1
        rpcuser=electrumx
        rpcpassword=<mypassword>
        [main]
        rpcbind=bitcoind
  electrumx:
    image: lukechilds/electrumx:v1.13.0
    volumes:
      - "<myelectrumxdata>:/data"
    ports:
      - "<myip>:50002:50002"
    environment:
      - DAEMON_URL=http://electrumx:<mypassword>@bitcoind
      - COIN=BitcoinSegwit
      - SERVICES=ssl://electrumx:50002
      - LOG_LEVEL=debug

I tried the same with testnet; same results.

(<myip>, <myhost>, <mypassword>, <mybitcoinddata> and <myelectrumxdata> obviously have real values in the real config)

I think it would be useful to document the necessary configuration for if you want to connect ElectrumX to…

• a bitcoind running in a second container.
• a bitcoind running on the host machine (e.g. via the Debian package).

Hi there,

Thanks again for all your help. I have a quick question about docker resource usage. I'm getting about 1000 connections per minute incoming from the DDoS attack and y machine doesn't appear to be bottle necking anywhere so I'm wondering if its possible docker is limiting resources? I was told by default it did not limit resources, but I wondered if maybe you had changed a setting in that regard (an off chance, I know). Anyway I'm right now in the process of moving the server from a quad core laptop to a 32 core xeon with 64GB RAM (hyperV so actually 64 theoretical cores). Hopefully the overkill of this server coupled with being on a fiber connection will help overpower the DDoSer.

That gets me to the main point of this message. E-X has been upgraded to limit resources based on IP address to help mitigate (and hopefully end) this DDoS. Think you could spare a few minutes to update to version 0.10.1 which was just released 18 hours ago? I've already sent a tip to get your attention, and you know how I roll after the fact ; )

edit: Also, any idea why nothing is showing up under releases anymore?

Attempting to connect to Polar's regtest node with:

docker run \
  -v /Users/user/electrumx:/data \
  -e DAEMON_URL=polaruser:[email protected]:18443 \
  -e COIN=BitcoinSegwit \
  -e NET=regtest \
  -p 50002:50002 \
  -p 127.0.0.1:8000:8000 \
  lukechilds/electrumx

Returns the error:

INFO:electrumx:ElectrumX server starting
INFO:electrumx:logging level: INFO
INFO:Controller:Python version: 3.7.9 (default, Dec 18 2020, 05:56:48)  [GCC 9.3.0]
INFO:Controller:software version: ElectrumX 1.16.0
INFO:Controller:aiorpcX version: 0.18.7
INFO:Controller:supported protocol versions: 1.4-1.4.2
INFO:Controller:event loop policy: <uvloop.EventLoopPolicy object at 0x7f4968ff9990>
INFO:Controller:reorg limit is 8,000 blocks
INFO:Daemon:daemon #1 at 127.0.0.1:18443/ (current)
INFO:DB:switching current directory to /data
INFO:DB:using leveldb for DB backend
ERROR:Daemon:connection problem - check your daemon is running.  Retrying occasionally...

Steps to Reproduce:

• Install and run v1.3.0 of Polar
• Create and start the default regtest node in Polar.
• Attempt to connect docker-electrumx using Polar's credentials with:

docker run \
  -v /Users/user/electrumx:/data \
  -e DAEMON_URL=polaruser:[email protected]:18443 \
  -e COIN=BitcoinSegwit \
  -e NET=regtest \
  -p 50002:50002 \
  -p 127.0.0.1:8000:8000 \
  lukechilds/electrumx

Hi,

I'm running the following docker version:

Client: Docker Engine - Community
 Version:           19.03.8
 API version:       1.40
 Go version:        go1.12.17
 Git commit:        afacb8b
 Built:             Wed Mar 11 01:26:27 2020
 OS/Arch:           linux/arm64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.8
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.17
  Git commit:       afacb8b
  Built:            Wed Mar 11 01:24:59 2020
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

The docker is running on Odroid N2 with Ubuntu 18.04.3, kernel 4.9.219-72 and aarch64 architecture.
uname -m:
aarch64
dpkg --print-architecture:
arm64

When I try to run the container with:
docker run --net=host -v ~/electrumx:/data -e DAEMON_URL="electrumx:[email protected]:8332" -e COIN=BitcoinSegwit -e NET=mainnet -e RPC_HOST=127.0.0.1 -e RPC_PORT=8332 -p 50002:50002 lukechilds/electrumx

I receive the following error:
standard_init_linux.go:211: exec user process caused "exec format error"

Would be great to support more architectures like arm for raspberry pi. I run BTCPay server on an rpi 4 and it fails there because only linux/amd64 is supported.

I have no idea how but could investigate give it a try if you do not know/find any straight forward solution to this.

My setup is as follows:

Ubuntu Server 18.04 - hostname: myhost and IP 192.168.1.3 with fully synched full node and ElectrumX hosted (and the router has public IP - let's say with public IP1.2.3.4 with port forwarding for 50002 "on", and pointing to 192.168.1.3 )

Mac with Electrum wallet - IP 192.168.1.4 (same network as above, same router)

From my mac, I can connect to from Electrum wallet and use it perfecting using server as myhost:50002 but CANNOT connect with 192.168.1.3:50002 or 1.2.3.4:50002

Is there something I miss or some way to diagnose the issue? From home network, all is ok as I can just use the hostname that works, but I cannot connect from same network with IP or from external network with public IP. Please advise how I could troubleshoot this, I cant find any connection error in the logs of ElectrumX.

Docker file:

version: "3"

services:
  electrumx:
    image: lukechilds/electrumx
    ports:
      - "50002:50002"
    volumes:
      - /some/drive/zElectrumX_data:/data
    environment:
      - DAEMON_URL=un:pw:43782
      - COIN=BitcoinSegwit

Hi,
first time using docker and I think I did a mistake.
What I did was running the command:
sudo docker run --net=host -v /media/user/hd/electrumx/data -e ..........
basically I skipped the : after electrumx
no wonder why my data folder is empty...
Do you know where am I saving all the data right now? is there a default directory?
I see that electrumx is syncing and saving data on my ssd but I don't know where.
Thanks for the help

That way we can just call them directly.

It simplifies the init script and simplifies using RPC with docker exec.

Hi,
i cannot access / ping it from outsite the server.

hain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy DROP)
target prot opt source destination
DOCKER-USER all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-1 all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

Chain DOCKER (1 references)
target prot opt source destination
ACCEPT tcp -- anywhere 172.17.0.2 tcp dpt:50002
ACCEPT tcp -- anywhere 172.17.0.2 tcp dpt:50001
ACCEPT tcp -- anywhere 172.17.0.2 tcp dpt:8000

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
RETURN all -- anywhere anywhere

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target prot opt source destination
DROP all -- anywhere anywhere
RETURN all -- anywhere anywhere

Chain DOCKER-USER (1 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere
RETURN all -- anywhere anywhere

Any Ideas?

I am on a Fedora 28 machine. I'm creating the container, as root:

docker run \
 -v /home/rspigler/electrumx:/data \
 -e DAEMON_URL=http://RPCUser:RPCPassword@localhost:8332 \
 -e COIN=BitcoinSegwit \
 -e ALLOW_ROOT=1 \
 -e BANNER_FILE=/home/rspigler/Documents/BannerFile.txt \
 -e ANON_LOGS=1 \
 -e MAX_SUBS=500000 \
 -e MAX_SESSION_SUBS=100000 \
 -e PEER_DISCOVERY=on \
 -e PEER_ANNOUNCE=1 \
 -e FORCE_PROXY=1 \
 -e TOR_PROXY_PORT=9050 \
 -e REPORT_HOST_TOR=l3e2obqp26fr2n5r.onion \
 -p 50002:50002 \
 lukechilds/electrumx

RPCUser and RPCPassword matches what's set in my bitcoin.conf file, and the onion address was generated by running cat /var/lib/tor/electrum-service/hostname after placing

HiddenServiceDir /var/lib/tor/electrum-service/
HiddenServicePort 50002 127.0.0.1:50002

in my Torrc file and restarting Tor.

I get the following error:

Generating a 2048 bit RSA private key
............................................+++
....................................................+++
unable to write 'random state'
writing new private key to '/data/electrumx.key'
/data/electrumx.key: Permission denied
140254660733836:error:0200100D:system library:fopen:Permission >denied:bss_file.c:406:fopen('/data/electrumx.key','w')
140254660733836:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:408:
INFO:root:ElectrumX server starting
INFO:Controller:event loop policy: None
INFO:Daemon:daemon #1 at localhost:8332/ (current)
INFO:BlockProcessor:switching current directory to /data
INFO:BlockProcessor:using leveldb for DB backend
Traceback (most recent call last):
File "/electrumx/electrumx_server.py", line 25, in main
controller = Controller(Env())
File "/electrumx/server/controller.py", line 84, in init
self.bp = self.coin.BLOCK_PROCESSOR(env, self, self.daemon)
File "/electrumx/server/block_processor.py", line 145, in init
super().init(env)
File "/electrumx/server/db.py", line 68, in init
self.open_dbs()
File "/electrumx/server/db.py", line 116, in open_dbs
self.utxo_db = self.db_class('utxo', for_sync)
File "/electrumx/server/storage.py", line 31, in init
self.open(name, create=self.is_new)
File "/electrumx/server/storage.py", line 83, in open
max_open_files=mof)
File "plyvel/_plyvel.pyx", line 247, in plyvel._plyvel.DB.init
File "plyvel/_plyvel.pyx", line 88, in plyvel._plyvel.raise_for_status
plyvel._plyvel.IOError: b'IO error: utxo/LOCK: No such file or directory'
CRITICAL:root:ElectrumX server terminated abnormally

Thank you for any help in advance.

Electrumx doesn't listen on port 50001. I have a readiness probe and I am getting a connection refused message. I think this started happening on the 1.16 version.

I cant work out what to do here, I keep getting this, every time the server starts..
Please advise if you have encountered this.

root@ssd:~# docker logs -f generated_electrumx_1
INFO:electrumx:ElectrumX server starting
INFO:electrumx:logging level: DEBUG
INFO:Controller:Python version: 3.7.9 (default, Dec 18 2020, 05:56:48)  [GCC 9.3.0]
INFO:Controller:software version: ElectrumX 1.16.0
INFO:Controller:aiorpcX version: 0.18.5
INFO:Controller:supported protocol versions: 1.4-1.4.2
INFO:Controller:event loop policy: None
INFO:Controller:reorg limit is 200 blocks
INFO:Daemon:daemon #1 at bitcoind:43782/ (current)
INFO:DB:switching current directory to /data
INFO:DB:using leveldb for DB backend
ERROR:Daemon:connection problem - check your daemon is running.  Retrying occasionally...
INFO:Daemon:running normally
INFO:DB:opened UTXO DB (for sync: True)
INFO:DB:UTXO DB version: 8
INFO:DB:coin: BitcoinSegwit
INFO:DB:network: mainnet
INFO:DB:height: 696,707
INFO:DB:tip: 0000000000000000000042feb97d359861fda2bcb306e3d17220d8b2425ae44d
INFO:DB:tx count: 664,111,409
INFO:DB:flushing DB cache at 2,000 MB
INFO:History:history DB version: 1
INFO:History:flush count: 65,535
INFO:SessionManager:RPC server listening on 0.0.0.0:8000
INFO:Prefetcher:catching up to daemon height 697,011 (304 blocks behind)
INFO:BlockProcessor:our height: 696,717 daemon: 697,012 UTXOs 16MB hist 18MB
INFO:BlockProcessor:processed 10 blocks size 14.20 MB in 149.3s
INFO:BlockProcessor:our height: 696,724 daemon: 697,014 UTXOs 28MB hist 31MB
INFO:BlockProcessor:processed 7 blocks size 10.50 MB in 31.7s
INFO:BlockProcessor:our height: 696,738 daemon: 697,014 UTXOs 47MB hist 56MB
INFO:BlockProcessor:processed 14 blocks size 21.33 MB in 58.3s
INFO:BlockProcessor:our height: 696,751 daemon: 697,014 UTXOs 63MB hist 78MB
INFO:BlockProcessor:processed 13 blocks size 18.93 MB in 27.3s
INFO:BlockProcessor:processed 7 blocks size 10.56 MB in 14.9s
INFO:BlockProcessor:our height: 696,772 daemon: 697,014 UTXOs 86MB hist 108MB
INFO:BlockProcessor:processed 14 blocks size 18.68 MB in 29.3s
INFO:BlockProcessor:our height: 696,814 daemon: 697,014 UTXOs 104MB hist 136MB
INFO:BlockProcessor:processed 42 blocks size 28.92 MB in 32.8s
INFO:BlockProcessor:our height: 696,848 daemon: 697,014 UTXOs 120MB hist 158MB
INFO:BlockProcessor:processed 34 blocks size 23.53 MB in 34.3s
INFO:BlockProcessor:processed 25 blocks size 18.77 MB in 18.1s
INFO:BlockProcessor:our height: 696,887 daemon: 697,014 UTXOs 146MB hist 192MB
INFO:BlockProcessor:processed 14 blocks size 13.97 MB in 13.9s
INFO:BlockProcessor:processed 10 blocks size 13.38 MB in 15.0s
INFO:BlockProcessor:our height: 696,913 daemon: 697,014 UTXOs 163MB hist 218MB
INFO:BlockProcessor:processed 16 blocks size 14.90 MB in 12.5s
INFO:BlockProcessor:processed 26 blocks size 19.40 MB in 18.3s
INFO:BlockProcessor:our height: 696,974 daemon: 697,014 UTXOs 190MB hist 257MB
INFO:BlockProcessor:processed 35 blocks size 23.44 MB in 22.7s
INFO:BlockProcessor:our height: 697,009 daemon: 697,014 UTXOs 202MB hist 275MB
INFO:BlockProcessor:processed 35 blocks size 23.32 MB in 30.5s
INFO:BlockProcessor:processed 5 blocks size 4.92 MB in 10.1s
INFO:BlockProcessor:caught up to height 697014
INFO:DB:flushed filesystem data in 0.04s
INFO:Prefetcher:cancelled; prefetcher stopping 
INFO:SessionManager:closing down server for rpc://0.0.0.0:8000
INFO:Controller:shutting down
INFO:Controller:shutdown complete
ERROR:electrumx:ElectrumX server terminated abnormally
Traceback (most recent call last):
  File "/electrumx/electrumx_server", line 35, in main
    asyncio.run(controller.run())
  File "/usr/local/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
    return future.result()
  File "/electrumx/electrumx/lib/server_base.py", line 129, in run
    await server_task
  File "/electrumx/electrumx/lib/server_base.py", line 102, in serve
    await self.serve(shutdown_event)
  File "/electrumx/electrumx/server/controller.py", line 134, in serve
    await group.spawn(wait_for_catchup())
  File "/usr/local/lib/python3.7/site-packages/aiorpcX-0.18.5-py3.7.egg/aiorpcx/curio.py", line 242, in __aexit__
    await self.join()
  File "/usr/local/lib/python3.7/site-packages/aiorpcX-0.18.5-py3.7.egg/aiorpcx/curio.py", line 211, in join
    raise task.exception()
  File "/electrumx/electrumx/server/block_processor.py", line 702, in fetch_and_process_blocks
    await group.spawn(self._process_prefetched_blocks())
  File "/usr/local/lib/python3.7/site-packages/aiorpcX-0.18.5-py3.7.egg/aiorpcx/curio.py", line 242, in __aexit__
    await self.join()
  File "/usr/local/lib/python3.7/site-packages/aiorpcX-0.18.5-py3.7.egg/aiorpcx/curio.py", line 211, in join
    raise task.exception()
  File "/electrumx/electrumx/server/block_processor.py", line 654, in _process_prefetched_blocks
    await self._first_caught_up()
  File "/electrumx/electrumx/server/block_processor.py", line 670, in _first_caught_up
    await self.flush(True)
  File "/electrumx/electrumx/server/block_processor.py", line 358, in flush
    await self.run_in_thread_with_lock(flush)
  File "/electrumx/electrumx/server/block_processor.py", line 212, in run_in_thread_with_lock
    return await asyncio.shield(run_in_thread_locked())
  File "/electrumx/electrumx/server/block_processor.py", line 211, in run_in_thread_locked
    return await run_in_thread(func, *args)
  File "/usr/local/lib/python3.7/site-packages/aiorpcX-0.18.5-py3.7.egg/aiorpcx/curio.py", line 68, in run_in_thread
    return await get_event_loop().run_in_executor(None, func, *args)
  File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/electrumx/electrumx/server/block_processor.py", line 357, in flush
    self.estimate_txs_remaining)
  File "/electrumx/electrumx/server/db.py", line 244, in flush_dbs
    self.flush_history()
  File "/electrumx/electrumx/server/db.py", line 313, in flush_history
    self.history.flush()
  File "/electrumx/electrumx/server/history.py", line 160, in flush
    flush_id = pack_be_uint16(self.flush_count)
struct.error: 'H' format requires 0 <= number <= 65535

Had issues getting this to work after it fully sync'd to wallet daemon, pip install websockets in the build was the fix.

Hello,

Thank you for a great Docker image.

Upon start-up, I get the following:

ERROR   RPC server failed to listen on localhost:8000 :[Errno 99] error while attempting to bind on address ('::1', 8000, 0, 0): address not available [controller.py:238]

If it matters, I am running docker run without any non-default --net or related arguments.

I cannot seem to remedy this. Has anyone else here encountered this, and found a fix?

Thanks,
James

username@electrum:~/docker-electrumx$ sudo docker run lukechilds/electrumx
Generating a RSA private key
...........................................................+++++
...............................+++++
writing new private key to '/data/electrumx.key'

INFO:electrumx:ElectrumX server starting
ERROR:electrumx:ElectrumX server terminated abnormally
Traceback (most recent call last):
File "/electrumx/electrumx_server", line 31, in main
env = Env()
File "/electrumx/electrumx/server/env.py", line 44, in init
self.daemon_url = self.required('DAEMON_URL')
File "/electrumx/electrumx/lib/env_base.py", line 40, in required
raise cls.Error('required envvar {} not set'.format(envvar))
electrumx.lib.env_base.EnvBase.Error: required envvar DAEMON_URL not set

Hi all,

I need to stop the ElectrumX server within the docker container in order to run the electrumx_compact_history script from within the container.

When I use the command electrumx_rpc stop, it says
cannot connect - is ElectrumX catching up, not running, or is 18443 the wrong RPC port?
I already tried different ports.

Do you have any experience in running that script in order to compact the history?
The reason to run this script is if the flush_count of the database reaches 65535.

Just tried upgrading my Electrum client to the latest 3.1.2, but got this error with docker logs about not having the right SSL protocol version (also happened with 3.1.0 and 3.1.1):

INFO:Controller:SSL server listening on :50002
INFO:ElectrumX:[0] SSL 10.10.99.105:50312, 1 total
INFO:ElectrumX:[0] unsupported protocol version request 1.2
INFO:ElectrumX:[0] unsupported protocol version: 1.2

Not sure if this is upstream or not?

Cheers

Getting error with docker run

`docker run -v /home/ubuntu/electrumx:/data -e DAEMON_URL=http://x:[email protected]:8332 -e COIN=BitcoinSegwit -p 50002:50002 lukechilds/electrumx
Generating a RSA private key
....................+++++
......................................................................................................................................................+++++
writing new private key to '/data/electrumx.key'

INFO:electrumx:ElectrumX server starting
INFO:electrumx:logging level: INFO
INFO:Controller:Python version: 3.7.2 (default, Jan 30 2019, 23:45:31) [GCC 6.4.0]
INFO:Controller:software version: ElectrumX 1.12
INFO:Controller:aiorpcX version: 0.18.1
INFO:Controller:supported protocol versions: 1.4-1.4.2
INFO:Controller:event loop policy: None
INFO:Controller:reorg limit is 200 blocks
INFO:Daemon:daemon #1 at 11.11.12.11:8332/ (current)
INFO:DB:switching current directory to /data
INFO:DB:using leveldb for DB backend
ERROR:Daemon:Forbidden
ERROR:root:task crashed: <Task finished coro=<Controller.serve() done, defined at /electrumx/electrumx/server/controller.py:81> exception=DaemonError('Forbidden')>
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/aiorpcX-0.18.1-py3.7.egg/aiorpcx/util.py", line 309, in check_task
task.result()
File "/electrumx/electrumx/server/controller.py", line 121, in serve
await daemon.height()
File "/electrumx/electrumx/server/daemon.py", line 274, in height
self._height = await self._send_single('getblockcount')
File "/electrumx/electrumx/server/daemon.py", line 172, in _send_single
return await self._send(payload, processor)
File "/electrumx/electrumx/server/daemon.py", line 130, in _send
result = await self._send_data(data)
File "/electrumx/electrumx/server/daemon.py", line 107, in _send_data
raise DaemonError(text)
electrumx.server.daemon.DaemonError: Forbidden`

Only adding --net=host option helps. Could you help to find out why I get the error?

I'm trying to setup the ElectrumX server on windows, but receive the following errors currently:

docker run --net=host -v ~/electrumx:/data -e DAEMON_URL="rpcuser:[email protected]:8332" -e COIN=BitcoinSegwit -e NET=testnet -e RPC_HOST=127.0.0.1 -e RPC_PORT=8332 -p 5
0002:50002 lukechilds/electrumx
WARNING: Published ports are discarded when using host network mode
INFO:root:ElectrumX server starting
INFO:Controller:event loop policy: None
INFO:Daemon:daemon #1 at 127.0.0.1:8332/ (current)
INFO:BlockProcessor:switching current directory to /data
INFO:BlockProcessor:using leveldb for DB backend
Traceback (most recent call last):
  File "/electrumx/electrumx_server.py", line 25, in main
    controller = Controller(Env())
  File "/electrumx/server/controller.py", line 84, in __init__
    self.bp = self.coin.BLOCK_PROCESSOR(env, self, self.daemon)
  File "/electrumx/server/block_processor.py", line 145, in __init__
    super().__init__(env)
  File "/electrumx/server/db.py", line 68, in __init__
    self.open_dbs()
  File "/electrumx/server/db.py", line 116, in open_dbs
    self.utxo_db = self.db_class('utxo', for_sync)
  File "/electrumx/server/storage.py", line 31, in __init__
    self.open(name, create=self.is_new)
  File "/electrumx/server/storage.py", line 83, in open
    max_open_files=mof)
  File "plyvel/_plyvel.pyx", line 247, in plyvel._plyvel.DB.__init__
  File "plyvel/_plyvel.pyx", line 88, in plyvel._plyvel.raise_for_status
plyvel._plyvel.IOError: b'IO error: utxo: Invalid argument'
CRITICAL:root:ElectrumX server terminated abnormally

Any hints on what might cause this?

Hi there!

I know your busy, but its more important than ever to upgrade your electrumx server because of the massive phishing attack. I like your new tip page. I've pre-tipped you $20 with more to follow the updated docker ;)

Your the best!

As opposed to running as root by default.

What database does Docker use by default?
rocksdb? leveled?
thanks

Hi all,

I want to build 2 containers which separately contain bitcoind and Electrumx and connect ElectrumX to bitcoind.
Currently trying this on Regtest.

The bitcoind node is running and I can communicate via bitcoin-cli and curl commands
curl --user U:P --data '{"method": "getinfo"}' http://127.0.0.1:5000

When starting the ElectrumX container, after quite some time a Daemon connection error appears:

I used this command to start the ElectrumX container:
docker run \ -v ~/Documents/Crypto/AndGo/regtest02/data:/electrumx/data \ -e DAEMON_URL=http://U:P@[IP address of bitcoind container]:18443 \ -e COIN=BitcoinSegwit \ -e NET=regtest \ -e RPC_HOST=127.0.0.1 \ -e RPC_PORT=18443 \ -e HOST=electrumx \ -p 50002:50002 \ lukechilds/electrumx

I am still quite new to all this so I am not sure what to look into.
In bicoin.conf I set:
rpcallowip=0.0.0.0/0
regtest.rpcport=18443

Your help is much appreciated!

• Fabian

docker run -v /home/username/electrumx:/data -e DAEMON_URL=http://u:[email protected]:8332 -e RPC_HOST=127.0.0.1 -e COIN=BitcoinSegwit -p 50002:50002 --net=host lukechilds/electrumx:1.8

Unable to find image 'lukechilds/electrumx:1.8' locally docker: Error response from daemon: manifest for lukechilds/electrumx:1.8 not found.

I need to use the lastest version electrumX server

Version on docker image:

INFO:BlockProcessor:software version: ElectrumX 1.4 INFO:BlockProcessor:supported protocol versions: 0.9-1.2

How to perform clean shutdown without corrupting database?

@kyuupichan

Thank you.

Worked on amd64 systems but not on arm32. docker installed correctly.
Log of the build command docker build -t electrumx_img .

I have ElectrumX running nicely in docker using the full bitcoin node of BTCPay Server. Works well. I have submitted PR here:
btcpayserver/btcpayserver-docker#217

In case you see some improvements or polishing possible, please let me know.

Can be used really easily with docker exec.

Checking out a version tag allows upstream attacks where the repo can be modified to contain malicious code and update the tags. We will checkout the version tag containing the malicious code and build a vulnerably Docker image.

Specifying the commit hash acts as a checksum, so if the upstream repo is modified, we won't checkout the new malicious code.

Hi,

My ElectrumX server keeps crashing randomly

Here are the logs,

INFO:DB:sync time: 3d 18h 57m ETA: 23h 18m 38s

INFO:Prefetcher:cancelled; prefetcher stopping

INFO:Controller:shutting down
INFO:Controller:shutdown complete
ERROR:electrumx:ElectrumX server terminated abnormally
Traceback (most recent call last):
File "/electrumx/electrumx_server", line 35, in main
asyncio.run(controller.run())

INFO:History:DB shut down uncleanly. Scanning for excess history flushes...