Code Monkey home page Code Monkey logo

plugins's Introduction

Plugins for Core-Lightning

Community curated plugins for Core-Lightning.

Integration Tests (latest) Nightly Integration Tests (master)

Available plugins

Name Short description
backup A simple and reliable backup plugin
btcli4j A Bitcoin Backend to enable safely the pruning mode, and support also rest APIs.
circular A smart rebalancing plugin for Core Lightning routing nodes
clearnet A plugin that can be used to enforce clearnet connections when possible
cln-ntfy Core Lightning plugin for sending ntfy alerts.
currencyrate A plugin to convert other currencies to BTC using web requests
donations A simple donations page to accept donations from the web
event-websocket Exposes notifications over a Websocket
feeadjuster Dynamic fees to keep your channels more balanced
go-lnmetrics.reporter Collect and report of the lightning node metrics
graphql Exposes the Core-Lightning API over graphql
holdinvoice Holds htlcs for invoices until settle or cancel is called (aka Hodlinvoices) via RPC/GRPC
invoice-queue Listen to lightning invoices from multiple nodes and send to a redis queue for processing
lightning-qt A bitcoin-qt-like GUI for lightningd
listmempoolfunds Track unconfirmed wallet deposits
monitor helps you analyze the health of your peers and channels
nloop Generic Lightning Loop for boltz
paythrough Pay an invoice through a specific channel, regardless of better routes
persistent-channels Maintains a number of channels to peers
poncho Turns CLN into a hosted channels provider
pruning This plugin manages pruning of bitcoind such that it can always sync
rebalance Keeps your channels balanced
reckless An experimental plugin manager (search/install plugins)
sauron A Bitcoin backend relying on Esplora's API
sitzprobe A Lightning Network payment rehearsal utility
sling Rebalance your channels with smart rules and built-in background tasks
sparko RPC over HTTP with fine-grained permissions, SSE and spark-wallet support
summars Print configurable summary of node, channels and optionally forwards, invoices, payments
trustedcoin Replace your Bitcoin Core with data from public block explorers
watchtower-client Watchtower client for The Eye of Satoshi
webhook Dispatches webhooks based from event notifications
zmq Publishes notifications via ZeroMQ to configured endpoints

Archived plugins

The following is a list of archived plugins that are no longer maintained and reside inside the 'archived' subdirectory. Any plugins that fail CI will be archived.

If you like a plugin from that list, feel free to update and fix it, so we can un-archive it.

Name Short description
autopilot An autopilot that suggests channels that should be established
commando This plugin allows to send commands between nodes
drain Draining, filling and balancing channels with automatic chunks.
helpme This plugin is designed to walk you through setting up a fresh Core-Lightning node
historian Archiving the Lightning Network
jitrebalance The JITrebalance plugin
noise Chat with your fellow node operators
paytest A plugin to benchmark the performance of the pay plugin
probe Regularly probes the network for stability
prometheus Lightning node exporter for the prometheus timeseries server
summary Print a nice summary of the node status

Installation

To install and activate a plugin you need to stop your lightningd and restart it with the plugin argument like this:

lightningd --plugin=/path/to/plugin/directory/plugin_file_name.py

Notes:

  • The plugin_file_name.py must have executable permissions: chmod a+x plugin_file_name.py
  • A plugin can be written in any programming language, as it interacts with lightningd purely using stdin/stdout pipes.

Automatic plugin initialization

Alternatively, especially when you use multiple plugins, you can copy or symlink all plugin directories into your ~/.lightning/plugins directory. The daemon will load each executable it finds in sub-directories as a plugin. In this case you don't need to manage all the --plugin=... parameters.

Dynamic plugin initialization

Most of the plugins can be managed using the RPC interface. Use

lightning-cli plugin start /path/to/plugin/directory/plugin_file_name

to start it, and

lightning-cli plugin stop /path/to/plugin/directory/plugin_file_name

to stop it.

As a plugin developer this option is configurable with all the available plugin libraries, and defaults to true.

PYTHONPATH and pyln

To simplify plugin development you can rely on pyln-client for the plugin implementation, pyln-proto if you need to parse or write lightning protocol messages, and pyln-testing in order to write tests. These libraries can be retrieved in a number of different ways:

  • Using pip tools: pip3 install pyln-client pyln-testing
  • Using the PYTHONPATH environment variable to include your clightning's shipped pyln-* libraries:
export PYTHONPATH=/path/to/lightnind/contrib/pyln-client:/path/to/lightnind/contrib/pyln-testing:$PYTHONPATH

Writing tests

The pyln-testing library provides a number of helpers and fixtures to write tests. While not strictly necessary, writing a test will ensure that your plugin is working correctly against a number of configurations (both with and without DEVELOPER, COMPAT and EXPERIMENTAL_FEATURES), and more importantly that they will continue to work with newly release versions of Core-Lightning.

Writing a test is as simple as this:

  • The framework will look for unittest filenames starting with test_.
  • The test functions should also start with test_.
from pyln.testing.fixtures import *

pluginopt = {'plugin': os.path.join(os.path.dirname(__file__), "YOUR_PLUGIN.py")}

def test_your_plugin(node_factory, bitcoind):
    l1 = node_factory.get_node(options=pluginopt)
    s = l1.rpc.getinfo()
    assert(s['network'] == 'regtest') # or whatever you want to test

Tests are run against pull requests, all commits on master, as well as once ever 24 hours to test against the latest master branch of the Core-Lightning development tree.

Running tests locally can be done like this: (make sure the PYTHONPATH env variable is correct)

pytest YOUR_PLUGIN/YOUR_TEST.py

Python plugins specifics

Additional dependencies

Additionally, some Python plugins come with a requirements.txt which can be used to install the plugin's dependencies using the pip tools:

pip3 install -r requirements.txt

Note: You might need to also specify the --user command line flag depending on your environment.

Minimum supported Python version

The minimum supported version of Python for this repository is currently 3.8.x (14 Oct 2019). Python plugins users must ensure to have a version >= 3.8. Python plugins developers must ensure their plugin to work with all Python versions >= 3.8.

More Plugins from the Community

Plugin Builder Resources

plugins's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

plugins's Issues

Pretty-print for monitor

Latest version just outputs a giant block of text, but I figured out a solution, maybe add to readme?

lightning-cli -J monitor | sed 's/"channels": "/"channels": /' | sed 's/"]}"/"]}/' | sed 's/\\t/ /g' | sed 's/\\//g' | jq

A simple blacklist plugin

Just blacklist some nodes by node id. I have an implementation already but if a new contributor wants to jump on it before i get to pr it i can review :). Hint: use the peer_connected hook.

jitrebalance: shouldn't it be static ?

I just crashed my node by stopping jitrebalance via RPC. I could have give it a thought before issuing the command, but making the plugin static would avoid the footgun in the first place : what do you think ?

2020-05-25T09:23:37.816Z INFO plugin-jitrebalance.py: Killing plugin: /home/bitcoin/clightning/plugins/jitrebalance/jitrebalance.py stopped by lightningd via RPC
2020-05-25T09:23:38.356Z **BROKEN** lightningd: Already in transaction from lightningd/io_loop_with_timers.c:31
2020-05-25T09:23:38.970Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v0.8.2)
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: common/daemon.c:44 (send_backtrace) 0x5586d65ea826
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: common/daemon.c:52 (crashdump) 0x5586d65ea876
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7fda1762783f
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7fda176277bb
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7fda17612534
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: lightningd/log.c:819 (fatal) 0x5586d65bc454
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: wallet/db.c:798 (db_begin_transaction_) 0x5586d6622d08
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: lightningd/plugin_hook.c:206 (plugin_hook_callback) 0x5586d65df213
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: lightningd/plugin_hook.c:134 (plugin_hook_killed) 0x5586d65def81
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:240 (notify) 0x5586d664f8d4
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:402 (del_tree) 0x5586d664fdc3
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:412 (del_tree) 0x5586d664fe15
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: ccan/ccan/tal/tal.c:486 (tal_free) 0x5586d665014f
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: lightningd/plugin_control.c:232 (clear_plugin) 0x5586d65de3c6
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: lightningd/plugin_control.c:248 (plugin_dynamic_stop) 0x5586d65de46b
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: lightningd/plugin_control.c:318 (json_plugin_control) 0x5586d65de7bb
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:587 (command_exec) 0x5586d65b610c
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:695 (rpc_command_hook_callback) 0x5586d65b65a4
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: lightningd/plugin_hook.c:277 (plugin_hook_call_) 0x5586d65df5a6
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:772 (plugin_hook_call_rpc_command) 0x5586d65b69c4
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: lightningd/jsonrpc.c:779 (call_rpc_command_hook) 0x5586d65b69ed
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: common/timeout.c:39 (timer_expired) 0x5586d65fac9d
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:32 (io_loop_with_timers) 0x5586d65b4055
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:984 (main) 0x5586d65b99fb
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7fda1761409a
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x5586d659e9b9
2020-05-25T09:23:38.970Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffffffffffffff

Lightning Websocket and Invoice Queue plugins

Hello
I've made 2 plugins and I thought I share them here. It could be useful for people.

Redis queue for paid invoice

This is used for managing multiple nodes. When an invoice is paid the invoice id and the node pub is added to queue, so the paid invoice can be processed. (Planning to also implement "new_invoice" creation event as well)
https://github.com/rbndg/Lightning-Invoice-Queue

C-Lightning Websocket Events

Simple WebSocket events for C-Lightning.
https://github.com/rbndg/c-lightning-events

probe: “plugin-probe.pyBROKEN: The plugin failed to respond to "getmanifest" in time, terminating....

Probe plugin doesn't seem to load.
If I try through the lightning-cli plugin start plugins/probe/probe.py command I get:

lightning-cli plugin start ~/plugins/probe/probe.py
{
   "plugins": [
      {
         "name": "/usr/local/bin/../libexec/c-lightning/plugins/autoclean",
         "active": true
      },
      {
         "name": "/usr/local/bin/../libexec/c-lightning/plugins/pay",
         "active": true
      },
...

      {
         "name": "/home/gbd/plugins/probe/probe.py",
         "active": false
      }
   ]
}

The crash log says:

+2342.535486570 plugin-managerDEBUG: started(8830) /home/gbd/plugins/probe/probe.py
...
+2403.487756863 plugin-probe.pyBROKEN: The plugin failed to respond to "getmanifest" in time, terminating.
+2403.487993615 lightningd(8279):BROKEN: Can't recover from plugin failure, terminating.
+2403.509562589 lightningd(8279):BROKEN: FATAL SIGNAL 6 (version v0.7.2.1-66-g76f27f4)
+2403.509639080 lightningd(8279):BROKEN: backtrace: common/daemon.c:45 (send_backtrace) 0x557784ab299e
+2403.509655829 lightningd(8279):BROKEN: backtrace: common/daemon.c:53 (crashdump) 0x557784ab29db
+2403.509671700 lightningd(8279):BROKEN: backtrace: (null):0 ((null)) 0x7fa257e3c05f
+2403.509686324 lightningd(8279):BROKEN: backtrace: (null):0 ((null)) 0x7fa257e3bfff
+2403.509699474 lightningd(8279):BROKEN: backtrace: (null):0 ((null)) 0x7fa257e3d429
+2403.509712953 lightningd(8279):BROKEN: backtrace: lightningd/log.c:639 (fatal) 0x557784a99da4
+2403.509727484 lightningd(8279):BROKEN: backtrace: lightningd/plugin.c:802 (plugin_manifest_timeout) 0x557784aaa788
+2403.509742979 lightningd(8279):BROKEN: backtrace: common/timeout.c:39 (timer_expired) 0x557784ab91ce
+2403.509757689 lightningd(8279):BROKEN: backtrace: lightningd/io_loop_with_timers.c:32 (io_loop_with_timers) 0x557784a945c6
+2403.509772888 lightningd(8279):BROKEN: backtrace: lightningd/lightningd.c:840 (main) 0x557784a983a1
+2403.509787573 lightningd(8279):BROKEN: backtrace: (null):0 ((null)) 0x7fa257e292e0
+2403.509800687 lightningd(8279):BROKEN: backtrace: (null):0 ((null)) 0x557784a881e9
+2403.509812391 lightningd(8279):BROKEN: backtrace: (null):0 ((null)) 0xffffffffffffffff

I'm using also the dbbackup plugin that takes 2 minutes to start. Maybe this could be relevant.

JITrebalance plugin improvements

Just a summary to keep a trace of the discussion on IRC :

  • Don't log onions
  • Don't try again channels that were tried some seconds ago
  • Maybe exhaust all possible routes instead of stopping at 5 ?

autopilot: too many values to unpack

I don't want autopilot to do anything, and documentation was unclear. I believe this should be the default mode, and you should have to specify something to make it actually take over your node.

Anyway, my config was:

autopilot-percent=0
autopilot-num-channels=0

Resulting in:

2019-07-26T04:27:40.397Z INFO lightningd(1263): RPC method 'autopilot-run-once' does not have a docstring.
2019-07-26T04:29:10.127Z INFO plugin-autopilot.py No input specified download graph from peers
2019-07-26T04:29:10.129Z INFO plugin-autopilot.py Instantiated networkx graph to store the lightning network
2019-07-26T04:29:10.131Z INFO plugin-autopilot.py Attempt RPC-call to download nodes from the lightning network
2019-07-26T04:33:10.600Z INFO plugin-autopilot.py Number of nodes found and added to the local networkx graph: 4452
2019-07-26T04:33:10.602Z INFO plugin-autopilot.py Attempt RPC-call to download channels from the lightning network
2019-07-26T04:44:12.382Z INFO plugin-autopilot.py Number of retrieved channels: 64728
2019-07-26T05:29:38.483Z INFO plugin-autopilot.py I'd like to open -10 new channels with 0 satoshis each
2019-07-26T05:29:38.507Z UNUSUAL plugin-autopilot.py 2019-07-26 05:29:38,498 - lib-autopilot - INFO - running the autopilot on a graph with 4453 nodes and 29049 edges.
2019-07-26T05:29:38.512Z UNUSUAL plugin-autopilot.py INFO:lib-autopilot:running the autopilot on a graph with 4453 nodes and 29049 edges.
2019-07-26T05:29:38.519Z UNUSUAL plugin-autopilot.py 2019-07-26 05:29:38,506 - lib-autopilot - INFO - GENERATE CANDIDATES: Try to generate up to -10 nodes with 4 strategies: (random, central, network Improvement, liquidity)
2019-07-26T05:29:38.524Z UNUSUAL plugin-autopilot.py INFO:lib-autopilot:GENERATE CANDIDATES: Try to generate up to -10 nodes with 4 strategies: (random, central, network Improvement, liquidity)
2019-07-26T05:29:38.529Z UNUSUAL plugin-autopilot.py 2019-07-26 05:29:38,507 - lib-autopilot - INFO - DECREASE DIAMETER: Generating probability density function
2019-07-26T05:29:38.534Z UNUSUAL plugin-autopilot.py INFO:lib-autopilot:DECREASE DIAMETER: Generating probability density function
2019-07-26T05:30:58.978Z INFO plugin-autopilot.py Traceback (most recent call last):
2019-07-26T05:30:58.980Z INFO plugin-autopilot.py   File \"/home/rusty/lightning/contrib/pylightning/lightning/plugin.py\", line 363, in _dispatch_request
2019-07-26T05:30:58.982Z INFO plugin-autopilot.py     result = self._exec_func(method.func, request)
2019-07-26T05:30:58.985Z INFO plugin-autopilot.py   File \"/home/rusty/lightning/contrib/pylightning/lightning/plugin.py\", line 347, in _exec_func
2019-07-26T05:30:58.988Z INFO plugin-autopilot.py     return func(*ba.args, **ba.kwargs)
2019-07-26T05:30:58.989Z INFO plugin-autopilot.py   File \"/home/rusty/plugins/autopilot/autopilot.py\", line 148, in run_once
2019-07-26T05:30:58.992Z INFO plugin-autopilot.py     percentile=0.5
2019-07-26T05:30:58.997Z INFO plugin-autopilot.py   File \"/home/rusty/plugins/autopilot/lib_autopilot.py\", line 382, in find_candidates
2019-07-26T05:30:59.000Z INFO plugin-autopilot.py     res = self.__create_pdfs()
2019-07-26T05:30:59.003Z INFO plugin-autopilot.py   File \"/home/rusty/plugins/autopilot/lib_autopilot.py\", line 290, in __create_pdfs
2019-07-26T05:30:59.008Z INFO plugin-autopilot.py     res[\"path\"] = self.__get_long_path_pdf()
2019-07-26T05:30:59.013Z INFO plugin-autopilot.py   File \"/home/rusty/plugins/autopilot/lib_autopilot.py\", line 245, in __get_long_path_pdf
2019-07-26T05:30:59.016Z INFO plugin-autopilot.py     for node, paths in all_pair_shortest_path_lengths:
2019-07-26T05:30:59.026Z INFO plugin-autopilot.py ValueError: too many values to unpack (expected 2)

JIT rebalance plugin crash

2020-02-02T00:29:43.059Z **BROKEN** lightningd: Plugin for htlc_accepted returned non-result response {"jsonrpc": "2.0", "id": 34, "error": "Error while processing htlc_accepted: ValueError('Millisatoshi must be >= 0')"}
2020-02-02T00:29:43.461Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v0.8.0-136-gf3600d2)
2020-02-02T00:29:43.461Z **BROKEN** lightningd: backtrace: common/daemon.c:46 (send_backtrace) 0x56132d104049
2020-02-02T00:29:43.461Z **BROKEN** lightningd: backtrace: common/daemon.c:54 (crashdump) 0x56132d104099
2020-02-02T00:29:43.461Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f515e4030ff
2020-02-02T00:29:43.461Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f515e403081
2020-02-02T00:29:43.461Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f515e3ee534
2020-02-02T00:29:43.461Z **BROKEN** lightningd: backtrace: lightningd/log.c:819 (fatal) 0x56132d0d6b83
2020-02-02T00:29:43.461Z **BROKEN** lightningd: backtrace: lightningd/plugin_hook.c:83 (plugin_hook_callback) 0x56132d0f8b4f
2020-02-02T00:29:43.461Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:241 (plugin_response_handle) 0x56132d0f4d33
2020-02-02T00:29:43.461Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:339 (plugin_read_json_one) 0x56132d0f4edb
2020-02-02T00:29:43.462Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:371 (plugin_read_json) 0x56132d0f502e
2020-02-02T00:29:43.462Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x56132d157eee
2020-02-02T00:29:43.462Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x56132d158a6f
2020-02-02T00:29:43.462Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x56132d158aad
2020-02-02T00:29:43.462Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:445 (io_loop) 0x56132d15ac79
2020-02-02T00:29:43.462Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:24 (io_loop_with_timers) 0x56132d0cd38c
2020-02-02T00:29:43.462Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:871 (main) 0x56132d0d4176
2020-02-02T00:29:43.462Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7f515e3efbba
2020-02-02T00:29:43.462Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x56132d0b84a9
2020-02-02T00:29:43.462Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffffffffffffff

Ideas for a plugin

  • A plugin that generates a QR code out of a newaddr to fund lightningd's wallet from a mobile wallet

  • A Bitcoin backend plugin for your favourite data source, you can checkout Sauron as an example.

JITrebalance: assertion failed

Just re-tested JIT-rebalance, and I got an assertion error (did not dive in it yet so I post the issue as a reminder) :

2020-05-25T11:31:24.787Z UNUSUAL plugin-jitrebalance.py: Exception in thread Thread-48:
2020-05-25T11:31:24.787Z UNUSUAL plugin-jitrebalance.py: Traceback (most recent call last):
2020-05-25T11:31:24.787Z UNUSUAL plugin-jitrebalance.py:   File \"/usr/lib/python3.5/threading.py\", line 914, in _bootstrap_inner
2020-05-25T11:31:24.787Z UNUSUAL plugin-jitrebalance.py:     self.run()
2020-05-25T11:31:24.787Z UNUSUAL plugin-jitrebalance.py:   File \"/usr/lib/python3.5/threading.py\", line 862, in run
2020-05-25T11:31:24.787Z UNUSUAL plugin-jitrebalance.py:     self._target(*self._args, **self._kwargs)
2020-05-25T11:31:24.787Z UNUSUAL plugin-jitrebalance.py:   File \"/home/bitcoin/clightning/plugins/jitrebalance/jitrebalance.py\", line 66, in try_rebalance
2020-05-25T11:31:24.787Z UNUSUAL plugin-jitrebalance.py:     route = get_circular_route(scid, chan, amt, peer, exclusions, request)
2020-05-25T11:31:24.787Z UNUSUAL plugin-jitrebalance.py:   File \"/home/bitcoin/clightning/plugins/jitrebalance/jitrebalance.py\", line 20, in get_circular_route
2020-05-25T11:31:24.787Z UNUSUAL plugin-jitrebalance.py:     assert(len(reverse_chan) == 2)
2020-05-25T11:31:24.787Z UNUSUAL plugin-jitrebalance.py: AssertionError
2020-05-25T11:31:24.787Z UNUSUAL plugin-jitrebalance.py:

backup sqlite file grows fast, crashes lightningd when reaching 2GB

10 days ago I configures backup. I remember the backup file being twice the size of the ~/.lighting/bitcoin/lightningd.sqlite file (75 vs 148MB). Just today I noticed that my lightingd kept crashing, as the backup file was at 2GB and exceeded some limit. I removed it, inited again, and now it's at twice the size again. What is the exptected size/growth?

I have 16 channels open, if this is relevant.

rebalance creating incorrect routes

When trying around with the rebalance plugin I discovered that it sometimes constructs / tries incorrect routes in a way that the payment gets delivered to one of my channels (no money lost), but it gets shifted into the wrong channel.

As the logs show below, my node (02a2c53bc475cb92e4ab2f38a5bca56df695034ce90ad78c2f47c05911e3f79e41) appears twice in the route. The rebalanced amount is shifted to the first channel occurence of my node along the route (574212x277x1), not the destination (567180x2077x1).

2019-05-06T21:27:34.579Z plugin-rebalance.py Estimating optimal amount 708740000msat
2019-05-06T21:27:34.785Z plugin-rebalance.py Invoice payment_hash: b1cc486a533865738b13f3662ec7218083e32b8c2575b3ee7d5e12b6d678e87c
2019-05-06T21:27:34.821Z plugin-rebalance.py Sending 708747066msat over 5 hops to rebalance 708740000msat
2019-05-06T21:27:34.821Z plugin-rebalance.py Node: 030995c0c0217d763c2274aa6ed69a0bb85fa2f7d118f93631550f3b6219a577f5, channel: 566627x2814x0, 708747066msat
2019-05-06T21:27:34.821Z plugin-rebalance.py Node: 03dab87ff8635982815c4567eb58af48f9944d11c56beb12b91e1049aaea06e187, channel: 567237x2798x0, 708746358msat
2019-05-06T21:27:34.821Z plugin-rebalance.py Node: 02a2c53bc475cb92e4ab2f38a5bca56df695034ce90ad78c2f47c05911e3f79e41, channel:  574212x277x1, 708744941msat
2019-05-06T21:27:34.821Z plugin-rebalance.py Node: 0214ec84c84827dd4911de56d2ecb77d367c6f24c658b8acfe4826b01968e45594, channel: 567180x2077x1, 708741707msat
2019-05-06T21:27:34.821Z plugin-rebalance.py Node: 02a2c53bc475cb92e4ab2f38a5bca56df695034ce90ad78c2f47c05911e3f79e41, channel: 567180x2077x1, 708740000msat
2019-05-06T21:27:34.825Z lightningd(23061): Sending 708747066msat over 5 hops to deliver 708740000msat
2019-05-06T21:27:58.052Z lightningd(23061): Resolved invoice 'Rebalance-5f20fcab-fbab-4cb6-8913-df49b423d03e' with amount 708740000msat

Same here:

2019-05-06T21:20:17.644Z plugin-rebalance.py Estimating optimal amount 708739000msat
2019-05-06T21:20:17.819Z plugin-rebalance.py Invoice payment_hash: 99ea0a4780439704a8b906a004d12d4771bee0b0c0e25a7ae40c59105d323185
2019-05-06T21:20:17.859Z plugin-rebalance.py Sending 708746066msat over 5 hops to rebalance 708739000msat
2019-05-06T21:20:17.859Z plugin-rebalance.py Node: 030995c0c0217d763c2274aa6ed69a0bb85fa2f7d118f93631550f3b6219a577f5, channel: 566627x2814x0, 708746066msat
2019-05-06T21:20:17.859Z plugin-rebalance.py Node: 02247d9db0dfafea745ef8c9e161eb322f73ac3f8858d8730b6fd97254747ce76b, channel: 567267x1939x0, 708745358msat
2019-05-06T21:20:17.859Z plugin-rebalance.py Node: 02a2c53bc475cb92e4ab2f38a5bca56df695034ce90ad78c2f47c05911e3f79e41, channel: 573948x2183x0, 708743941msat
2019-05-06T21:20:17.859Z plugin-rebalance.py Node: 0214ec84c84827dd4911de56d2ecb77d367c6f24c658b8acfe4826b01968e45594, channel: 567180x2077x1, 708740707msat
2019-05-06T21:20:17.859Z plugin-rebalance.py Node: 02a2c53bc475cb92e4ab2f38a5bca56df695034ce90ad78c2f47c05911e3f79e41, channel: 567180x2077x1, 708739000msat
2019-05-06T21:20:17.862Z lightningd(23061): Sending 708746066msat over 5 hops to deliver 708739000msat
2019-05-06T21:20:31.089Z lightningd(23061): Resolved invoice 'Rebalance-2ea19a98-27e7-4b7e-a484-69d396f53995' with amount 708739000msat

autopilot: ZeroDivisionError

Testing on jsarenik/plugins@34e1ad6 (the only difference to current master, 89c95e4, is an added requirements.txt file).

$ lightningd --version
v0.7.0-382-g8ef6623

See #34 on how I run the daemon. This is my ~/.lightning/config:

network=testnet
log-level=debug

On the client side, same for both with and without dryrun, I get this:

$ lightning-cli autopilot-run-once dryrun
"Error while processing autopilot-run-once: ZeroDivisionError('float division by zero')"

On the daemon side:

...
2019-05-28T09:29:19.666Z plugin-autopilot.py I'd like to open 8 new channels with 942187 satoshis each
2019-05-28T09:29:19.667Z plugin-autopilot.py 2019-05-28 11:29:19,666 - lib-autopilot - INFO - running the autopilot on a graph with 2 nodes and 1 edges.
2019-05-28T09:29:19.667Z plugin-autopilot.py INFO:lib-autopilot:running the autopilot on a graph with 2 nodes and 1 edges.
2019-05-28T09:29:19.667Z plugin-autopilot.py 2019-05-28 11:29:19,667 - lib-autopilot - INFO - GENERATE CANDIDATES: Try to generate up to 8 nodes with 4 strategies: (random, central, network Improvement, liquidity)
2019-05-28T09:29:19.667Z plugin-autopilot.py INFO:lib-autopilot:GENERATE CANDIDATES: Try to generate up to 8 nodes with 4 strategies: (random, central, network Improvement, liquidity)
2019-05-28T09:29:19.668Z plugin-autopilot.py 2019-05-28 11:29:19,667 - lib-autopilot - INFO - DECREASE DIAMETER: Generating probability density function
2019-05-28T09:29:19.668Z plugin-autopilot.py INFO:lib-autopilot:DECREASE DIAMETER: Generating probability density function
2019-05-28T09:29:19.668Z plugin-autopilot.py 2019-05-28 11:29:19,668 - lib-autopilot - INFO - DECREASE DIAMETER: probability density function created
2019-05-28T09:29:19.668Z plugin-autopilot.py INFO:lib-autopilot:DECREASE DIAMETER: probability density function created
2019-05-28T09:29:19.669Z plugin-autopilot.py 2019-05-28 11:29:19,668 - lib-autopilot - INFO - manipulate_pdf: Skewing the probability density function
2019-05-28T09:29:19.669Z plugin-autopilot.py INFO:lib-autopilot:manipulate_pdf: Skewing the probability density function
2019-05-28T09:29:19.669Z plugin-autopilot.py 2019-05-28 11:29:19,669 - lib-autopilot - INFO - CENTRALITY_PDF: Try to generate a PDF proportional to centrality scores
2019-05-28T09:29:19.669Z plugin-autopilot.py INFO:lib-autopilot:CENTRALITY_PDF: Try to generate a PDF proportional to centrality scores
2019-05-28T09:29:19.671Z plugin-autopilot.py Traceback (most recent call last):
2019-05-28T09:29:19.672Z plugin-autopilot.py   File \"/home/n/src/plugins/autopilot/venv/lib/python3.7/site-packages/lightning/plugin.py\", line 352, in _dispatch_request
2019-05-28T09:29:19.672Z plugin-autopilot.py     result = self._exec_func(method.func, request)
2019-05-28T09:29:19.672Z plugin-autopilot.py   File \"/home/n/src/plugins/autopilot/venv/lib/python3.7/site-packages/lightning/plugin.py\", line 336, in _exec_func
2019-05-28T09:29:19.672Z plugin-autopilot.py     return func(*ba.args, **ba.kwargs)
2019-05-28T09:29:19.672Z plugin-autopilot.py   File \"./autopilot.py\", line 148, in run_once
2019-05-28T09:29:19.672Z plugin-autopilot.py     percentile=0.5
2019-05-28T09:29:19.672Z plugin-autopilot.py   File \"/home/n/src/plugins/autopilot/lib_autopilot.py\", line 382, in find_candidates
2019-05-28T09:29:19.672Z plugin-autopilot.py     res = self.__create_pdfs()
2019-05-28T09:29:19.672Z plugin-autopilot.py   File \"/home/n/src/plugins/autopilot/lib_autopilot.py\", line 291, in __create_pdfs
2019-05-28T09:29:19.672Z plugin-autopilot.py     res[\"centrality\"] = self.__get_centrality_pdf()
2019-05-28T09:29:19.672Z plugin-autopilot.py   File \"/home/n/src/plugins/autopilot/lib_autopilot.py\", line 165, in __get_centrality_pdf
2019-05-28T09:29:19.672Z plugin-autopilot.py     pdf = {k:v/cumsum for k, v in pdf.items()}
2019-05-28T09:29:19.672Z plugin-autopilot.py   File \"/home/n/src/plugins/autopilot/lib_autopilot.py\", line 165, in <dictcomp>
2019-05-28T09:29:19.672Z plugin-autopilot.py     pdf = {k:v/cumsum for k, v in pdf.items()}
2019-05-28T09:29:19.672Z plugin-autopilot.py ZeroDivisionError: float division by zero
2019-05-28T09:29:19.672Z plugin-autopilot.py

@renepickhardt Please have a look. Thanks!

Unexpected and unexplainable error

First of all: sendinvoiceless is a really cool and useful feature that I've implemented in my lightning game

I run 2 nodes like this: myshtery <-> Bitkoins.nl <-> 7 other channels

Myshtery is only connect to Bitkoins.nl. I have paid "Ship of Theseus" via Bitkoins.nl a few day ago 1000 satoshi with sendinvioiceless and it went ok over 4 hops and a small fee.

I had to pay "Ship of Theseus" again today but it didn't work while it's a very well connected node and it worked before. Error message:

error: {'message': 'Sending failed'}\",)"

I tried increasing the fee but nothing worked. I tested paying another node ID with Bitkoins.nl and that went fine.

I then decided to try to pay with myshtery but since myshtery is only connected to Bitkoins.nl it should have made it harder not easier to pay. However, the payment went smooth (via Bitkoins.nl obviously) over 6 hops and a small fee. After that I tried to pay again via Bitkoins.nl but it didn't work. Restarted Bitkoins.nl but that also didn't help.

What on Earth could cause this situation?

c-lightning: "v0.8.0-1-gb14b2b0"

Prometheus error and no data after update to lightning 0.8.0

Everything works fine with lightning 0.7.1

After changed to lightning 0.8.0 I got the following error on prometheus.

level=warn ts=2019-12-31T15:32:39.600Z caller=scrape.go:930 component="scrape manager" scrape_pool=lightningd target=http://clightning_tobg:9900/metrics msg="append failed" err="expected equal, got "INVALID""

With curl I get all the metrics.

Everything runs on docker.

drain: test_setbalance fails if EXPERIMENTAL_FEATURES=1

Recently the test has started failing with the following error:

727        # set and test some 70/30 specific balancing
728        assert(l1.rpc.setbalance(scid12, 30))
729        wait_for_all_htlcs(nodes)
730        ours_after = get_ours(l1, scid12)
731>       assert(ours_after < ours_before * 0.33)
732E       assert 335290000msat < (1000000000msat * 0.33)
733
734drain/test_drain.py:141: AssertionError

Full stacktrace:

_______________________________ test_setbalance ________________________________
690[gw4] linux -- Python 3.7.6 /opt/python/3.7.6/bin/python3.7
691
692node_factory = <pyln.testing.utils.NodeFactory object at 0x7f9f18bbad50>
693bitcoind = <pyln.testing.utils.BitcoinD object at 0x7f9f18da36d0>
694
695    @unittest.skipIf(not DEVELOPER, "slow gossip, needs DEVELOPER=1")
696    def test_setbalance(node_factory, bitcoind):
697        # SETUP: a basic circular setup to run setbalance tests
698        #
699        #   l1---l2
700        #    |    |
701        #   l4---l3
702        #
703    
704        l1, l2, l3, l4 = node_factory.line_graph(4, opts=pluginopt)
705        l4.rpc.connect(l1.info['id'], 'localhost', l1.port)
706        nodes = [l1, l2, l3, l4]
707    
708        scid12 = l1.get_channel_scid(l2)
709        scid23 = l2.get_channel_scid(l3)
710        scid34 = l3.get_channel_scid(l4)
711        scid41 = l4.fund_channel(l1, 10**6)
712    
713        # wait for each others gossip
714        bitcoind.generate_block(6)
715        for n in nodes:
716            for scid in [scid12,scid23,scid34,scid41]:
717                n.wait_channel_active(scid)
718    
719        # test auto 50/50 balancing
720        ours_before = get_ours(l1, scid12)
721        assert(l1.rpc.setbalance(scid12))
722        ours_after = wait_ours(l1, scid12, ours_before)
723        # TODO: can we fix/change/improve this to be more precise?
724        assert(ours_after < ours_before * 0.52)
725        assert(ours_after > ours_before * 0.48)
726    
727        # set and test some 70/30 specific balancing
728        assert(l1.rpc.setbalance(scid12, 30))
729        wait_for_all_htlcs(nodes)
730        ours_after = get_ours(l1, scid12)
731>       assert(ours_after < ours_before * 0.33)
732E       assert 335290000msat < (1000000000msat * 0.33)
733
734drain/test_drain.py:141: AssertionError

Full logs: https://gist.github.com/b40deadd7f5fdb6c4e9a721e4f735199

backup: version_count is never increased

The header of the FileBackend backup file contains a version_count metadata field but this is never increased from 0 (It's also never used, so this issue doesn't affect anything in practice).
I suppose it is meant to be increased in add_change?

jitrebalance crash

2020-09-10T06:33:28.150Z INFO plugin-jitrebalance.py: Got an incoming HTLC htlc={'amount': '674395289msat', 'cltv_expiry': 647860, 'cltv_expiry_relative': 294, 'payment_hash': 'ddb6cf9d71dddf198a0f0b44cefd919d6d00521915ec9c7dd0caec893722a7cd'}
2020-09-10T06:33:28.173Z **BROKEN** lightningd: Plugin for htlc_accepted returned non-result response {"jsonrpc": "2.0", "id": 63253, "error": {"code": -32600, "message": "Error while processing htlc_accepted: 'NoneType' object is not subscriptable"}}
2020-09-10T06:33:30.264Z **BROKEN** lightningd: FATAL SIGNAL 6 (version v0.9.0-1)

No traceback. It restrained lightningd from restarting for hours as it would crash on startup, the above is the first crash log.

autopilot breaking when no peers present. Missing dns requirement

Hi, When autopilot is run with no peers present -

for nodeid in random.shuffle(self.__get_seed_keys()):

This call fails, due to a missing import of the random module. Upon importing random, the process fails at a further point due to no module "dns" being present.

srv_records = dns.resolver.query(domain,"SRV")

It does not appear that there is any such dependency specified in the requirements of the autopilot plugin.

What is the "dns" module autopilot is expecting to use? I believe adding this requirement and corresponding import statement should make initial seed key retrieval complete. Allowing autopilot to run with a node with no peers connected.

I'd be glad to open a PR to resolve this, I will attempt to find which dns module we are expecting as well.

Let autopilot adjust routing fees

In this stack exchange question https://bitcoin.stackexchange.com/questions/87488/why-is-my-lighting-node-not-routing-any-transaction/87490#87490 a lnd user assumed that the lnd autopilot would also automatically set the best routing fees.

As I wanted to change the centrality measure of lib_autopilot.py anyway to be estimated on the fee graph. While doing this the autopilot plugin could adjust/suggest the fees for each channel.

Actually it could also be a separate plugin.

feeadjuster: assert can trigger

Description

I've seen a curious line in the lightningd log:

2020-12-06T20:32:06.150Z **BROKEN** plugin-feeadjuster.py: Adjusting fees:

I've investigated the situation and figured out what happened. It caused by the assert at line 122 in feeadjuster/feeadjuster.py:

assert 0 <= percentage and percentage <= 1

Steps to reproduce

  1. Let's assume I have a channel with a total size of 100,000 Sats. 20,000 Sats are mine in that channel.
  2. I call a feeadjust. It saves my balance in plugin.adj_balances[scid] at line 198.
  3. Then an incoming payment happens: 30,000 Sats arrive into the channel. (In my case it was actually a circular payment, rebalance, but it does not matter.)
    At this point I have 50,000 Sats in the channel, but plugin.adj_balances[scid] still stores the 20,000 Sats value. The feeadjuster is not aware of the incoming payments.
  4. After these a forward event occurs: 40,000 Sats leave the channel.
    The feeadjuster is subscribed to the forward event, so it reduces my stored balance by 40,000 Sats at line 173. The result will be negative: -20,000 Sats. This will trigger the assert.

A possible solution

The easy way: it will not happen again if I turn off the automatic fee updates in feeadjuster.

plugin idea: YA Bitcoin backend plugin

We could have a plugin that completely reproduce bcli's behaviour (hit bitcoind) but which would in addition fall back to other block sources if bitcoind can't serve that block (for example if it's been pruned already). Here is an example use case of nix-bitcoin.

We could fall back to a randomly-picked explorer from a list, cross-check its hash with bitcoind and serve it to lightningd.

There could also be a fallback to the Blockstream satellite :).

backup: cli tool failed to restore backup

Setup:

  • network testnet
  • docker - alpine:3.11 (amd64)
  • c-lighning v0.9.1
  • plugins version df4b422
  • Python 3.8.2
  • pip freeze:

Click==7.0
flaky==3.7.0
Mako==1.1.0
MarkupSafe==1.1.1
psutil==5.7.3
pyln-client==0.8.2
tqdm==4.51.0

SQLite

  • lib sqlite-dev 3.30.1
  • python sqlite3.sqlite_version: 3.30.1

Steps to reproduce:

  • backup-cli init performed
  • backup plugin is working (db version >2000)
  • lightningd is stopped
  • file /var/lib/lightning/testnet/lightningd.sqlite3 is removed.
backup-cli restore file:///var/lib/lightning/backup/testnet.db /var/lib/lightning/testnet/lightningd.sqlite3

Error:

The lightningd.sqlite3 file is not restored, backup-cli crashed with error:

unrecognized token: "1891733WHERE"

Traceback:

662it [01:24,  7.85it/s]
Traceback (most recent call last):
  File "/usr/local/libexec/c-lightning/plugins/backup/backup-cli", line 76, in <module>
    cli()
  File "/usr/lib/python3.8/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.8/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.8/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/libexec/c-lightning/plugins/backup/backup-cli", line 64, in restore
    backend.restore(restore_destination)
  File "/usr/local/libexec/c-lightning/plugins/backup/backup.py", line 122, in restore
    self._restore_transaction(c.transaction)
  File "/usr/local/libexec/c-lightning/plugins/backup/backup.py", line 101, in _restore_transaction
    cur.execute(q.decode('UTF-8'))
sqlite3.OperationalError: unrecognized token: "1891733WHERE"

Error when starting lightningd with plugin path

File "/home/user/lightning/plugins/plugins/rebalance/rebalance.py", line 2, in
from lightning import Plugin, Millisatoshi, RpcError
ModuleNotFoundError: No module named 'lightning'

Does it matter where the plugin is located?

Prometheus plugin improvements

Where suggested by brian-brazil in prometheus/docs#1737:

Some suggestions to improve your exporter:
_total is a suffix for counters, it shouldn't be used with gauges. Some metrics like lightning_channel_in_msatoshi_offered also sound like they might be counters rather than gauges.
Units are inconsistent, sometimes missing and conflicting with the help text. You should consistently use satoshis, and also include satoshi as the unit in all relevant metric names for clarity.
id is a bit of a generic label, peer or peer_id would be clearer.

feeadjuster: a blacklist option

Just a reminder for me to implement it, but if a new contributor wants to jump on it i can (and Michael too probably) offer guidance

helpme: 'helpme channels' leads to error

I managed to complete stages 1 (funds) and 2 (peers), but calling stage 3 (channels) fails:

$ ./cli/lightning-cli helpme channels
"Error while processing helpme: UnboundLocalError(\"local variable 'inv' referenced before assignment\",)"

Apparently the function give_channel_advice(plugin, *args) accesses the dictionary inv near the end:

plugins/helpme/helpme.py

Lines 767 to 772 in 41e4405

best = None
# Don't even bother if they don't have $10.
best_score = Millisatoshi("0.001btc") * 2
for p in peers:
# FIXME: Filter out giant fee channels...
channels = plugin.rpc.listchannels(source=inv['payee'])['channels']

but this variable was never defined. It would only get initialized if the length of args is 1, but it is always called without arguments...

plugins/helpme/helpme.py

Lines 921 to 922 in 41e4405

elif command == "channels":
return give_channel_advice(plugin)

autopilot: Timeout responding to 'init'

Description

The autopilot plugin consistently times out and terminates while lightningd waits for response to 'init'.

2021-01-19T20:36:12.455Z INFO    plugin-autopilot.py: RPC method 'autopilot-run-once' does not have a docstring.
2021-01-19T20:36:15.926Z INFO    plugin-bcli: bitcoin-cli initialized and connected to bitcoind.
2021-01-19T20:36:29.607Z INFO    plugin-autopilot.py: No input specified download graph from peers
2021-01-19T20:36:29.607Z INFO    plugin-autopilot.py: Instantiated networkx graph to store the lightning network
2021-01-19T20:36:29.607Z INFO    plugin-autopilot.py: Attempt RPC-call to download nodes from the lightning network
2021-01-19T20:36:29.613Z INFO    lightningd: --------------------------------------------------
2021-01-19T20:36:29.613Z INFO    lightningd: Server started with public key xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx, alias Xxxxxx (color #ffffff) and lightningd v0.9.2
2021-01-19T20:36:30.498Z INFO    plugin-autopilot.py: peering with node: yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
2021-01-19T20:37:33.477Z INFO    plugin-autopilot.py: Killing plugin: failed to respond to 'init' in time, terminating.

Steps to reproduce

Trial and error with various values of autopilot-num-channels (including 0) does not change this behaviour, plugin still consistently terminates ~60 seconds after peering with first node.

Possible workaround

From experimenting I see that timeout does not occur if lightningd is built with a larger value for PLUGIN_MANIFEST_TIMEOUT in file lightningd/plugin.c. This is an observation rather than a solution or workaround.
Possible cause may be too low or zero funds available although setting autopilot-min-channel-size-msat to 0 does not solve issue either.

Update

From further investigation it would seem that this timeout may be a result of using lightningd over Tor. When the autopilot plugin initializes the following messages start to appear in the Tor log:

Jan 23 20:28:54 tor Tor[59]: Bootstrapped 100% (done): Done
Jan 23 20:39:42 tor Tor[59]: Have tried resolving or connecting to address '[scrubbed]' at 3 different places. Giving up.
Jan 23 20:40:22 tor Tor[59]: We tried for 15 seconds to connect to '[scrubbed]' using exit $30DBF7A773F4EE3BF6D1B7407DAECF40220AC8CD~relayon0337 at 185.220.101.205. Retrying on a new circuit.
Jan 23 20:40:37 tor Tor[59]: We tried for 15 seconds to connect to '[scrubbed]' using exit $45E9240AD4ECE01793A1977C1260503B2C2C861F~apx1 at 185.107.47.215. Retrying on a new circuit.
Jan 23 20:40:52 tor Tor[59]: We tried for 15 seconds to connect to '[scrubbed]' using exit $E8AD8C4FDC3FE152150C005BB2EAA6A0990B74DF~F3Netze at 185.220.100.243. Retrying on a new circuit.
Jan 23 20:41:08 tor Tor[59]: We tried for 15 seconds to connect to '[scrubbed]' using exit $98F793C7320CE3C15A45353AFCC165747A40366D~F3Netze at 185.220.100.255. Retrying on a new circuit.
Jan 23 20:41:22 tor Tor[59]: We tried for 15 seconds to connect to '[scrubbed]' using exit $25EEFFF278E1D9C4FCBED1B664B11ECB33532C76~Unnamed at 188.127.251.245. Retrying on a new circuit.
Jan 23 20:41:37 tor Tor[59]: We tried for 15 seconds to connect to '[scrubbed]' using exit $AFF2FC5C6F793B6E147EB93C1897D6DDA49E54FD~Wix at 95.211.230.211. Retrying on a new circuit.
Jan 23 20:41:52 tor Tor[59]: We tried for 15 seconds to connect to '[scrubbed]' using exit $C00C28C2B1A7D8038517626CECA9BCB23B0A31D2~relayon0224 at 185.220.101.129. Retrying on a new circuit.
Jan 23 20:41:52 tor Tor[59]: Tried for 125 seconds to get a connection to [scrubbed]:9735. Giving up.
.
.
.

This may indicate a more general error with lightning over Tor rather than the autopilot plugn.

feeadjuster: error when trying to run c-lightning with feeadjuster plugin

  • c-lightning 0.9.2
  • Python 3.8.5
  • pyln-client 0.8.2
  • Bitcoin Core 0.20.1

When I try to run lightningd with the feeadjuster plugin I get the following error

billy@AcerUbuntu:~/.lightning$ lightningd
import-im6.q16: attempt to perform an operation not allowed by the security policy `PS' @ error/constitute.c/IsCoderAuthorized/408.
import-im6.q16: attempt to perform an operation not allowed by the security policy `PS' @ error/constitute.c/IsCoderAuthorized/408.
from: can't read /var/mail/pyln.client
/home/billy/.lightning/plugins/feeadjuster.py: 8: Syntax error: "(" unexpected

Not sure if related, but the python alias is for Python 2 and I use python3 for Python 3.

Also not sure if there is supposed to be something in /var/mail/ but that directory is empty. When I run pip3 install pyln-client I get the following message:

Requirement already satisfied: pyln-client in /home/billy/.local/lib/python3.8/site-packages (0.8.2)

Using the autoreload plugin hides error messages

@renepickhardt and I are working on a new plugin and were using for this the autoreload plugin. In the new plugin we are using hooks. With the autoreload plugin active our coding errors were not visible in the lightning-cli log file. After removing it and manually loading our own plugin via --plugin our errors were logged in the log file.

tagging @darosior

Edit: Just saw that this was somewhat explained in the Readme.md of the autoreload plugin. Still, this poses a problem and should be fixed.

backup: Backup plugin gets out of sync if `init` doesn't complete

Occasionally I get an emergency shutdown from the backup plugin:

2020-11-11T10:20:18.023Z INFO    plugin-backup.py: Comparing backup version 1308057 versus first write version 1308060
2020-11-11T10:20:18.023Z INFO    plugin-backup.py: Backup is out of date, we cannot continue safely. Emergency shutdown.
2020-11-11T10:20:18.034Z INFO    plugin-summary.py: Plugin summary.py initialized
2020-11-11T10:20:19.026Z INFO    plugin-backup.py: Killing process lightningd (22862)

This is caused by an incomplete startup, in which the plugin started
buffering the pre-init db_writes but then couldn't flush them on
init. Since 97c7027 we are writing the backup.lock file which
contains all the necessary information for us to start writing the
changes to the backup right away and don't have to wait for the init
message (previously needed to get CLI arguments).

We should really just be reading this file and start writing the
changes to the file without buffering.

Sauron plugin error "bad response to getchaininfo"

Hi all,

I'm trying to use this plugin because I need to make a test on bitcoin testnet, I try to use Sauron plugin, but I receive the error below

plugins/sauron/sauron.py error: bad response to getchaininfo (bad 'result' field), response was {"jsonrpc": "2.0", "id": 7, "error": {"code": -32600, "message": "Error while processing getchaininfo: Missing dependencies for SOCKS support."}}

I think that the RPC method getchaininfo returns a wrong response inside the plugin, but I'm not sure and I want make a double-check with someone that knows already the plugin code (maybe @darosior ?) because I can make some error inside the procedure.

My procedure is

  • clone the project && pip install requirements.
  • Setting the plugin inside the file conf and remove the bitcoin conf inside it.
  • Ran the following command
    lightningd --lightning-dir=/media/vincent/Maxtor/C-lightning/node/ --disable-plugin bcli --sauron-api-endpoint https://blockstream.info/testnet/api

RTL crash when parsing JSON-RPC response

I was trying to use the Rebalance plugin but after restarting Lightningd, RTL stops working.
Below there is the Systemctl status output.
Before adding the Rebalance plugin this line for RTL was present:
├─35732 node /home/go/c-lightning-REST/plugin.js
So it seems that there is a conflict between the above and this one:
├─35208 python3 /home/go/.lightning/plugins/rebalance/rebalance.py

● lightningd.service - C-Lightning daemon
     Loaded: loaded (/etc/systemd/system/lightningd.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2020-08-18 23:02:04 CDT; 39s ago
    Process: 35200 ExecStart=/usr/local/bin/lightningd --daemon --pid-file=/run/lightningd/lightningd.pid (code=exited, status=0/SUCCESS)
   Main PID: 35201 (lightningd)
      Tasks: 16 (limit: 14187)
     Memory: 274.1M
     CGroup: /system.slice/lightningd.service
             ├─35201 /usr/local/bin/lightningd --daemon --pid-file=/run/lightningd/lightningd.pid
             ├─35202 /usr/local/bin/../libexec/c-lightning/plugins/autoclean
             ├─35203 /usr/local/bin/../libexec/c-lightning/plugins/bcli
             ├─35204 /usr/local/bin/../libexec/c-lightning/plugins/fundchannel
             ├─35205 /usr/local/bin/../libexec/c-lightning/plugins/keysend
             ├─35206 /usr/local/bin/../libexec/c-lightning/plugins/pay
             ├─35208 python3 /home/go/.lightning/plugins/rebalance/rebalance.py
             ├─35223 /usr/local/libexec/c-lightning/lightning_hsmd
             ├─35224 /usr/local/libexec/c-lightning/lightning_connectd
             ├─35236 /usr/local/libexec/c-lightning/lightning_gossipd
             ├─35255 /usr/local/libexec/c-lightning/lightning_channeld
             ├─35256 /usr/local/libexec/c-lightning/lightning_channeld
             ├─35258 /usr/local/libexec/c-lightning/lightning_channeld
             ├─35259 /usr/local/libexec/c-lightning/lightning_channeld
             └─35263 /usr/local/libexec/c-lightning/lightning_channeld

Aug 18 23:02:10 BTCPayServer lightningd[35207]: **Error: Unexpected "T" at position 10 in state STOP**
Aug 18 23:02:10 BTCPayServer lightningd[35207]:     at Parser.proto.charError (/home/go/c-lightning-REST/node_modules/jsonparse/jsonparse.js:90:16)
Aug 18 23:02:10 BTCPayServer lightningd[35207]:     at Parser.proto.write (/home/go/c-lightning-REST/node_modules/jsonparse/jsonparse.js:267:27)
Aug 18 23:02:10 BTCPayServer lightningd[35207]:     at LightningClient._handledata (/home/go/c-lightning-REST/lightning-client-js.js:143:29)
Aug 18 23:02:10 BTCPayServer lightningd[35207]:     at Socket.LightningClient.client.on.data (/home/go/c-lightning-REST/lightning-client-js.js:77:46)
Aug 18 23:02:10 BTCPayServer lightningd[35207]:     at Socket.emit (events.js:198:13)
Aug 18 23:02:10 BTCPayServer lightningd[35207]:     at addChunk (_stream_readable.js:288:12)
Aug 18 23:02:10 BTCPayServer lightningd[35207]:     at readableAddChunk (_stream_readable.js:269:11)
Aug 18 23:02:10 BTCPayServer lightningd[35207]:     at Socket.Readable.push (_stream_readable.js:224:10)
Aug 18 23:02:10 BTCPayServer lightningd[35207]:     at Pipe.onStreamRead [as onread] (internal/stream_base_commons.js:94:17)

REST API Plugin

I have the need to access and administer my Lightning Node remotely in a secure, and authenticated manner (possibly with multi-user and some rbac).

I was unable to find any existing repositories which provided a robust REST API. And certainly none that are c-lightning plugins.

So i started one:

https://github.com/dfresh613/C-Lightning-REST

My goal is to support a large subset of methods available to the LightningRPC class of PyLightning so they will also be executable through a REST api with authentication and TLS.

I hope to have it ready to be a plugin within the next 2 weeks or so. I was just wondering if there was already anything else like this out there, or any explicit reason why we wouldn't want a REST API on c-lightning?

Add setup.py

Would be nice to have a setup.py for each plugin, so dependencies can be listed there. I see unspecified dependencies now (sqlalchemy etc) and it's not clear which ones are required.

Cannot Rebalance without msatoshi argument

I would like to rebalance my channels 50/50 which should work if I do not provide the msatoshi argument. Looks like msatoshi should be a whole number and not have a decimal point.

"Error while processing rebalance: 
RpcError('RPC call failed: method: invoice, payload: 
{
\\'msatoshi\\' : 8388607500.0msat, 
\\'label\\': \\'Rebalance-23a20dc6-356f-423e-8a0c-6a835a061847\\', 
\\'description\\ ': \\'1516934x52x0 to 1517537x39x0\\', 
\\'expiry\\': 120
}, 
error: {
\\'code\\': -32602, 
\\'message\\': \ "\\'msatoshi\\' should be millisatoshis or \\'any\\', not \\'8388607500.0msat\\'\"
}',)"

How to create a channel_update with a plugin using the htlc_accepted hook?

I am writing a plugin.

It uses the htlc_accepted hook.

In some cases I want to fail the forwarding of a htlc with a temporary_channel_failure but that requires me to create a channel_update. I am not sure if I have enough info inside a plugin to do that. It seems to me that I should parse the payload field in the onion, to get the info needed (like the short_channel_id).

Is my assumption correct? Are there helpers for parsing onion payloads and/or creating channel updates?

Thanks for any help or pointers.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.