hoytech / strfry Goto Github PK

View Code? Open in Web Editor NEW

447.0 17.0 90.0 469 KB

a nostr relay

License: GNU General Public License v3.0

Makefile 0.18% C++ 87.91% C 0.09% Perl 10.53% Dockerfile 1.18% Nix 0.11%

nostr

strfry's Issues

"bad event id" on actually valid object.

Long story short:

Jun 15 00:19:28 birb strfry[1775541]: 2023-06-15 00:19:28.688 ( 146.073s) [Ingester 1      ]INFO| [1] dumpInEvent: ["EVENT",{"id":"0bc534823aa6c247766d598ad42cc0726911b183145340887a26e3bccd2cf28c","pubkey":"5e336907a3dda5cd58f11d162d8a4c9388f9cfb2f8dc4b469c8151e379c63bc9","created_at":1686759536,"kind":7,"content":"+","tags":[["e","caeb5711dc51a534b1b599c4b34820262003d7e8367f91dff51d84dc07021672"],["p","89d1ce9164f1f172daaa9c784153178cb1dec7912bf55f5dc07e0f1dabe40e6c"]],"sig":"9b1e2d07682f8e14658c78124537ae2bd9d4851cc1c988075ef043071200b6f0cd167bc35661511dd4da292ac3b6148c976d12be694aae2819f7fdc5b85832dc"}]
Jun 15 00:19:28 birb strfry[1775541]: 2023-06-15 00:19:28.688 ( 146.073s) [Ingester 1      ]INFO| Rejected invalid event: bad event id

But, the ID looks perfectly fine.

Anything I missed or can do?

Config:

db = "/srv/strfry/db/"
dbParams {
    maxreaders = 256
    mapsize = 10995116277760
}
relay {
    bind = "127.0.0.1"
    port = 7777
    nofiles = 0
    realIpHeader = "x-forwarded-for"
    info {
        name = "Just a relay"
        description = "Just a relay I host for fun."
        pubkey = "npub1tcekjparmkju6k83r5tzmzjvjwy0nnajlrwyk35us9g7x7wx80ys9hjmky"
        contact = "@ingwiephoenix:ingwie.me"
    }
    maxWebsocketPayloadSize = 131072
    autoPingSeconds = 55
    enableTcpKeepalive = true
    queryTimesliceBudgetMicroseconds = 10000
    maxFilterLimit = 500
    maxSubsPerConnection = 20
    writePolicy {
        plugin = ""
        lookbackSeconds = 0
    }
    compression {
        enabled = true
        slidingWindow = true
    }
    logging {
        dumpInAll = false
        dumpInEvents = true # <- I changed this to see the logs.
        dumpInReqs = false
        dbScanPerf = false
    }
    numThreads {
        ingester = 3
        reqWorker = 3
        reqMonitor = 3
        yesstr = 1
    }
}
events {
    maxEventSize = 65536
    rejectEventsNewerThanSeconds = 900
    rejectEventsOlderThanSeconds = 94608000
    rejectEphemeralEventsOlderThanSeconds = 60
    ephemeralEventsLifetimeSeconds = 300
    maxNumTags = 2000
    maxTagValSize = 1024
}

I am using Caddy as a reverse proxy to handle TLS/SSL. Strfry is running on Ubuntu 22.04 arm64.

Anything I can do to fix that?

stream ws://SERVER_IP:PORT --dir up makes strfry not to receive events

Running strfry in docker container, with "relay" it works well, with "stream" it doesn't receive events.

console with stream:

date       time         ( uptime  ) [ thread name/id ]   v|
2023-03-04 03:19:09.991 (   0.016s) [main thread     ]INFO| arguments: /app/strfry stream ws://SERVER_IP:PORT --dir up
2023-03-04 03:19:09.991 (   0.016s) [main thread     ]INFO| Current dir: /app
2023-03-04 03:19:09.991 (   0.016s) [main thread     ]INFO| stderr verbosity: 0
2023-03-04 03:19:09.991 (   0.016s) [main thread     ]INFO| -----------------------------------
2023-03-04 03:19:09.991 (   0.016s) [main thread     ]INFO| CONFIG: Loading config from file: /etc/strfry.conf
2023-03-04 03:19:09.996 (   0.020s) [main thread     ]INFO| CONFIG: successfully installed
2023-03-04 03:19:09.997 (   0.021s) [main thread     ]INFO| Attempting to connect to ws://SERVER_IP:PORT
2023-03-04 03:19:09.998 (   0.022s) [main thread     ]INFO| Connected to SERVER_IP

Client (noscl) says: error opening websocket to ws://STRFRY_SERVER_IP:PORT read tcp balabala read: connection reset by peer.

Subscription id too long

I started a relayer wss://nos.lol

https://snort.social is having problems: [wss://nos.lol] NOTICE: ERROR: bad req: subscription id too long

https://astral.ninja seems to work better.

golpe uses private git path to loguru

$ make setup-golpe
cd golpe && git submodule update --init
Submodule 'external/PEGTL' (https://github.com/taocpp/PEGTL.git) registered for path 'external/PEGTL'
Submodule 'external/config' (https://github.com/taocpp/config.git) registered for path 'external/config'
Submodule 'external/docopt.cpp' (https://github.com/docopt/docopt.cpp.git) registered for path 'external/docopt.cpp'
Submodule 'external/hoytech-cpp' (https://github.com/hoytech/hoytech-cpp.git) registered for path 'external/hoytech-cpp'
Submodule 'external/json' (https://github.com/taocpp/json.git) registered for path 'external/json'
Submodule 'external/lmdbxx' (https://github.com/hoytech/lmdbxx.git) registered for path 'external/lmdbxx'
Submodule 'external/loguru' ([email protected]:emilk/loguru.git) registered for path 'external/loguru'
Submodule 'external/quadrable' (https://github.com/hoytech/quadrable.git) registered for path 'external/quadrable'
Submodule 'external/rasgueadb' (https://github.com/hoytech/rasgueadb.git) registered for path 'external/rasgueadb'
Submodule 'external/uWebSockets' (https://github.com/hoytech/uWebSockets.git) registered for path 'external/uWebSockets'
Cloning into '/home/e24/strfry/golpe/external/PEGTL'...
Cloning into '/home/e24/strfry/golpe/external/config'...
Cloning into '/home/e24/strfry/golpe/external/docopt.cpp'...
Cloning into '/home/e24/strfry/golpe/external/hoytech-cpp'...
Cloning into '/home/e24/strfry/golpe/external/json'...
Cloning into '/home/e24/strfry/golpe/external/lmdbxx'...
Cloning into '/home/e24/strfry/golpe/external/loguru'...
The authenticity of host 'github.com (140.82.121.4)' can't be established.
ECDSA key fingerprint is SHA256:p2QAMXNIC1TJYWeIOttrVc98/R1BUFWu3/LiyKgUfQM.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'github.com,140.82.121.4' (ECDSA) to the list of known hosts.
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:emilk/loguru.git' into submodule path '/home/e24/strfry/golpe/external/loguru' failed
Failed to clone 'external/loguru'. Retry scheduled
Cloning into '/home/e24/strfry/golpe/external/quadrable'...
Cloning into '/home/e24/strfry/golpe/external/rasgueadb'...
Cloning into '/home/e24/strfry/golpe/external/uWebSockets'...
Cloning into '/home/e24/strfry/golpe/external/loguru'...
Warning: Permanently added the ECDSA host key for IP address '140.82.121.3' to the list of known hosts.
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:emilk/loguru.git' into submodule path '/home/e24/strfry/golpe/external/loguru' failed
Failed to clone 'external/loguru' a second time, aborting
make: *** [golpe/rules.mk:54: setup-golpe] Error 1

I had to change loguru's url to a public one:

cd golpe
git submodule set-url external/loguru https://github.com/emilk/loguru.git
git submodule update
cd ..
make setup-golpe

Turn "Write policy plugins" into a NIP

Basically take this: https://github.com/hoytech/strfry/blob/master/docs/plugins.md

Slap a NIP-one-hundred-something on it, add some words at the top and bottom and submit to nostr-protocol/nips

Build error on FreeBSD

uWebsockets doesn't directly have kqueue support. We'll need to use its libuv wrapper I believe.

Astral.ninja integration

Brave browser log shows:

This does not happen on snort.social

add the #nostr tag to this repo

Would it be possible to add the #nostr tag to this repo

I'll show up in this nice list here:

https://github.com/topics/nostr

g++: fatal error: Killed signal terminated program cc1plus

g++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
make: *** [golpe/rules.mk:28: build/config.o] Error 1
make: *** Waiting for unfinished jobs....

Trying to install on an Ubuntu 22. This occurs when running make -j4

I ran sudo apt update and sudo apt upgrade followed by the commands in the docs. Any ideas?

Write policy applied to sync?

I created a whitelist policy, because I dont want spammers to be able to post using my instance.

But now when I run sync or stream those also fail to add anything because the events are not whitelisted.

So how do you make a policy that doesnt allow external users to use the relay, but still accept incoming data from sync or stream?

make -j4 vis make -j2

I have no idea if this is important...

On the home page of the project (https://github.com/hoytech/strfry) the instructions say to "make -j4"

On the deployment page (https://github.com/hoytech/strfry/blob/master/docs/DEPLOYMENT.md) it says "make -j2"

Since 4 is more than 2, I used 4 -- LOL.

Just a small typo in the setup instructions...

Whitelist / Blacklist pubkeys

Is there, or could there be, a way to configure strfry for whitelisting or blacklisting pubkeys allowing or disallowing posting to the instance?

Error running sync

I just migrated nostr.lu.ke to strfry. I have my old nostream instance exposed at nostream.lu.ke. I tried to migrate the DB with ./strfry sync wss://nostream.lu.ke but I'm getting:

date       time         ( uptime  ) [ thread name/id ]   v| 
2023-03-27 09:35:19.848 (   0.031s) [main thread     ]INFO| arguments: ./strfry sync wss://nostream.lu.ke
2023-03-27 09:35:19.848 (   0.031s) [main thread     ]INFO| Current dir: /app
2023-03-27 09:35:19.848 (   0.031s) [main thread     ]INFO| stderr verbosity: 0
2023-03-27 09:35:19.848 (   0.031s) [main thread     ]INFO| -----------------------------------
2023-03-27 09:35:19.848 (   0.031s) [main thread     ]INFO| CONFIG: Loading config from file: /etc/strfry.conf
2023-03-27 09:35:19.857 (   0.040s) [main thread     ]INFO| CONFIG: successfully installed
2023-03-27 09:35:19.906 (   0.089s) [main thread     ]INFO| Filter matched 34213 local events
terminate called without an active exception

Loguru caught a signal: SIGABRT
Stack trace:
13      0x56229354aca5 ./strfry(+0x2eca5) [0x56229354aca5]
12      0x7face7b2fe40 __libc_start_main + 128
11      0x7face7b2fd90 /lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7face7b2fd90]
10      0x56229354a720 ./strfry(+0x2e720) [0x56229354a720]
9       0x56229356b83a ./strfry(+0x4f83a) [0x56229356b83a]
8       0x562293546e44 ./strfry(+0x2ae44) [0x562293546e44]
7       0x562293849554 ./strfry(+0x32d554) [0x562293849554]
6       0x7face7ee32b7 /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae2b7) [0x7face7ee32b7]
5       0x7face7ee324c /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae24c) [0x7face7ee324c]
4       0x7face7ed7bbe /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa2bbe) [0x7face7ed7bbe]
3       0x7face7b2e7f3 abort + 211
2       0x7face7b48476 raise + 22
1       0x7face7b9ca7c pthread_kill + 300
0       0x7face7b48520 /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7face7b48520]
2023-03-27 09:35:19.907 (   0.090s) [main thread     ]FATL| Signal: SIGABRT
Aborted (core dumped)

I'm running e5ec135 built from the provided Dockerfile. Any ideas?

Core dump when run multiple in stream mode

I am collecting events from relays by running multiple instances of strfry in stream mode and this happens like in every few hours.

terminate called after throwing an instance of 'std::runtime_error'
what(): duplicate insert into Event

Loguru caught a signal: SIGABRT
Stack trace:
13 0x7f3c179f0a00 /lib/x86_64-linux-gnu/libc.so.6(+0x126a00) [0x7f3c179f0a00]
12 0x7f3c1795eb43 /lib/x86_64-linux-gnu/libc.so.6(+0x94b43) [0x7f3c1795eb43]
11 0x7f3c17cd52b3 /lib/x86_64-linux-gnu/libstdc++.so.6(+0xdc2b3) [0x7f3c17cd52b3]
10 0x5639ff8e37c3 ./strfry(+0x2b97c3) [0x5639ff8e37c3]
9 0x5639ff8e3110 ./strfry(+0x2b9110) [0x5639ff8e3110]
8 0x5639ff64d76d ./strfry(+0x2376d) [0x5639ff64d76d]
7 0x7f3c17ca7518 /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae518) [0x7f3c17ca7518]
6 0x7f3c17ca72b7 /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae2b7) [0x7f3c17ca72b7]
5 0x7f3c17ca724c /lib/x86_64-linux-gnu/libstdc++.so.6(+0xae24c) [0x7f3c17ca724c]
4 0x7f3c17c9bbbe /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa2bbe) [0x7f3c17c9bbbe]
3 0x7f3c178f27f3 abort + 211
2 0x7f3c1790c476 raise + 22
1 0x7f3c17960a7c pthread_kill + 300
0 0x7f3c1790c520 /lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f3c1790c520]
2023-01-11 08:15:48.106 (2499.091s) [Writer ]FATL| Signal: SIGABRT
Aborted (core dumped)

Using a Reverse Proxy like e.g. NGINX

The README mentions, there'll be details soon, on how the NGINX configuration should look like. However, #50 makes me wonder, whether there need to be some special settings enabled, to make this fully work.

Can I use a 101 NGINX Proxy configuration, as is used by most services?
How can I make sure, i.e. test, the service works fine & all functionalities are working?

I suggest the README gets a sections about NGINX.

Implement nip 42 client authentication

Please implement client auth as defined in nip 42. This is also needed for #17.

I am primarily interested in my relay requiring auth at some point. Only serve REQ from

paying users
their follows
their follows follows
everybody else. (no auth would count as everybody else)

But ... as clients don't support nip42, the very first step is to let users auth without caring with which key they auth and maybe notify them that auth failed without consequences, to get client devs moving on this topic.

As mentioned in the comments below, full nip-42 support could also mean support for sync and stream.

strfry relay supports nip42
strfry stream supports nip42
strfry sync supports nip42

Please provide an api for "Read Policy Plugins"

A relay can be strained by spam it gets to store forever and that can be addressed with the Write Policy Plugin.
Equally a relay can be strained by spam requests. Here, strfry at this moment provides very limited tools and I would like to better control the kind of queries clients may direct at the relay. Ideally in combination with #47 .

nip-11 pubkey as bech32 should be rejected or warned or well documented to avoid

On too many relays, people are putting npub keys into the 'pubkey' field of nip-11, such as wss://relay.mostr.pub/, wss://relay.current.fyi/, wss://nostr.fmt.wiz.biz/, wss://nostr.mom/ (some of those are nostream, I'll file a separate bug for that).

NIP-11 states:

An administrative contact may be listed with a pubkey, in the same format as Nostr events (32-byte hex for a secp256k1 public key)

Could you (1) validate this and error or warn, or (2) make the comments/documentation very clear on this point so people don't make this mistake?

The gossip client is using strict typing and fails to deserialize these NIP-11 JSON objects, meaning the client is not providing any interesting relay information to the user, and gossip is presuming these clients do not support NIP-11 and not utilizing them fully.

Of course I could accept npub, but I fall into the camp of people who believe we should hold the line and defend the standards lest they proliferate into too many defacto variants.

Logging and DDOS protection

This morning I’m getting a whole lot (thousands) of this error message repeated in my strfry logs:

too many concurrent REQs

Unfortunately, that error is logged without the IP address that is intentionally or unintentionally DDOSing my instance.

So I turned on verbose REQ logging and found the IP address in the initial connect and manually blocked it, but that’s obviously not a sustainable workflow, so I’m thinking that maybe it would be good for certain error messages (this one for sure) to log the IP address along with the error every time so that log monitors like fail2ban could keep an eye on things and automatically impose some IP blocking and unblocking by policy.

Allow tags with empty string value

Currently strfry rejects event with tag having empty string value (e.g. ["d", ""]) by this line:

strfry/src/events.cpp

Line 47 in e5ec135

if (tagVal.size() == 0) throw herr("tag val empty");

But:

NIP-33 has example of empty d tags.
Some events with empty tags have already been published. You can see NIP-23 posts (kind:30023) and NIP-58 badges (kind:30009) having tags like ["d", ""], ["thumb", ""] at wss://nostr-pub.wellorder.net.
- Send ["REQ", "subId", { "#d": [""] }] with Nostr Playground to see some cases.

EDIT: If this is allowed, it would be even better if it could also be searched by query { "#d": [""] } which is currently rejected by error "filter item too small".

strfry distribution channels

I'm working on a piece of software to replace Mastodon, and it will use strfry as its database, storing even things like config as Nostr events. I want users to be able to snap install strfry or flatpak install strfry. Any thoughts or feelings? Nerds prefer flatpak, but Snap is easier and comes preinstalled on Ubuntu, so I lean towards that.

Other possibilities:

apt install - not sure what hoops you have to jump through to make it into Debian or Ubuntu proper.
precompiled binary built by GitHub CI served from GitHub - it wouldn't hurt. maybe in addition to a snap/flatpak.

REQ plugin

I haven't fully thought this through yet, but I sometimes wish I could hook into client requests. Like, to be able to get ["REQ", ...] and potentially change the filter or response, or trigger side-effects. Maybe AUTH could even be implemented in a req plugin.

I still don't know if this is even a good idea yet, but I figured I'd drop it in case it strikes anyone else who might be having similar ideas.

./strfry stream unable to get non standard event kind

./strfry stream wss://relay.example.com. The event kind is 802 {kind: 802}. I'm not able to stream the events.

Large database performance

I noticed that as my database grew, the time between:

CONFIG: successfully installed

and

Filter matched XXX records

seemed to increase exponentially. When I had about 5 million events, it took a few minutes and now that I have 15 million events it takes hours (even though it doesnt max out the CPU). This makes it impossible to sync anymore.

By looking at the code my guess is that its stuck at this loop:

while (1) {
            bool complete = query.process(txn, [&](const auto &sub, uint64_t levId, std::string_view eventPayload){
                auto ev = lookupEventByLevId(txn, levId);
                ne.addItem(ev.flat_nested()->created_at(), sv(ev.flat_nested()->id()).substr(0, ne.idSize));

                numEvents++;
            });

            if (complete) break;
        }

At first sight its just adding events to a list, so it should not take exponentially more time? Is there anything that can be done to optimize this part?

Support NIP-26 event delegation

This is probably going to be necessary for long term viability of the protocol: https://github.com/nostr-protocol/nips/blob/master/26.md#relay--client-querying-support

It basically allows apps to generate new pubkeys like access tokens for a main pubkey.

But relays also have to return those events when clients query for the main pubkey.

Question: How can I most efficiently stream from many relays?

I want to stream from many - probably more than 30 - relays and apply a complex policy. The stream command only supports a single relay at a time. How do others achieve this? Do they run multiple strfry stream processes?

Compilation Errors

I'm trying to compile & run strfry and when running make I receive an error message of:

ubuntu@x11r0n:~/strfry$ make -j4
perl golpe/gen-fbs.pl
perl golpe/gen-config.pl
golpe/external/rasgueadb/rasgueadb-generate golpe.yaml build
perl golpe/gen-main.cpp.pl
error: /home/ubuntu/strfry/fbs/nostr-index.fbs:4: 16: error: expecting: ] instead got: :
flatc failure building fbs/nostr-index.fbs at golpe/gen-fbs.pl line 22, <$fh> line 1.
make: *** [golpe/rules.mk:44: build/golpe.h] Error 1
make: *** Waiting for unfinished jobs....
error: /home/ubuntu/strfry/fbs/nostr-index.fbs:4: 16: error: expecting: ] instead got: :
flatc failure at golpe/external/rasgueadb/rasgueadb-generate line 146, <$fh> line 1.
make: *** [golpe/rules.mk:54: build/defaultDb.h] Error 1

The steps I took to reproduce are:

sudo apt install -y git build-essential libyaml-perl libtemplate-perl libregexp-grammars-perl libssl-dev zlib1g-dev liblmdb-dev libflatbuffers-dev libsecp256k1-dev libzstd-dev
git clone https://github.com/hoytech/strfry.git
cd strfry/
git checkout tags/0.9.3
git submodule update --init
make setup-golpe
make -j4

This machine is running Ubuntu 20.04.6 LTS.

Any assistance appreciated!

Compilation error: could not convert ‘cfg().ConfigValues::relayloggingdumpInAll’ from ‘const string’ to bool

$ uname -a
Linux relay-001 5.15.0-52-generic #58-Ubuntu SMP Thu Oct 13 08:03:55 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

$ make
perl golpe/gen-config.pl
golpe/external/rasgueadb/rasgueadb-generate golpe.yaml build
g++ -std=c++20 -O3 -g -Wall -fPIC  -DDOCOPT_HEADER_ONLY -Iinclude -Ibuild -Isrc -Igolpe/external -Igolpe/external/config/include -Igolpe/external/json/include -Igolpe/external/PEGTL/include -Igolpe/external/hoytech-cpp -Igolpe/external/docopt.cpp -Igolpe/external/loguru -Igolpe/external/quadrable/include -MMD -MP -MT golpe/logging.o -MF golpe/logging.d -c golpe/logging.cpp -o golpe/logging.o
g++ -std=c++20 -O3 -g -Wall -fPIC  -DDOCOPT_HEADER_ONLY -Iinclude -Ibuild -Isrc -Igolpe/external -Igolpe/external/config/include -Igolpe/external/json/include -Igolpe/external/PEGTL/include -Igolpe/external/hoytech-cpp -Igolpe/external/docopt.cpp -Igolpe/external/loguru -Igolpe/external/quadrable/include -MMD -MP -MT build/main.o -MF build/main.d -c build/main.cpp -o build/main.o
g++ -std=c++20 -O0 -g -Wall -fPIC  -DDOCOPT_HEADER_ONLY -Iinclude -Ibuild -Isrc -Igolpe/external -Igolpe/external/config/include -Igolpe/external/json/include -Igolpe/external/PEGTL/include -Igolpe/external/hoytech-cpp -Igolpe/external/docopt.cpp -Igolpe/external/loguru -Igolpe/external/quadrable/include -MMD -MP -MT build/config.o -MF build/config.d -c build/config.cpp -o build/config.o
g++ -std=c++20 -O3 -g -Wall -fPIC  -DDOCOPT_HEADER_ONLY -Iinclude -Ibuild -Isrc -Igolpe/external -Igolpe/external/config/include -Igolpe/external/json/include -Igolpe/external/PEGTL/include -Igolpe/external/hoytech-cpp -Igolpe/external/docopt.cpp -Igolpe/external/loguru -Igolpe/external/quadrable/include -MMD -MP -MT src/RelayCron.o -MF src/RelayCron.d -c src/RelayCron.cpp -o src/RelayCron.o
g++ -std=c++20 -O3 -g -Wall -fPIC  -DDOCOPT_HEADER_ONLY -Iinclude -Ibuild -Isrc -Igolpe/external -Igolpe/external/config/include -Igolpe/external/json/include -Igolpe/external/PEGTL/include -Igolpe/external/hoytech-cpp -Igolpe/external/docopt.cpp -Igolpe/external/loguru -Igolpe/external/quadrable/include -MMD -MP -MT src/RelayIngester.o -MF src/RelayIngester.d -c src/RelayIngester.cpp -o src/RelayIngester.o
src/RelayIngester.cpp: In member function ‘void RelayServer::runIngester(ThreadPool<MsgIngester>::Thread&)’:
src/RelayIngester.cpp:20:35: error: could not convert ‘cfg().ConfigValues::relay__logging__dumpInAll’ from ‘const string’ {aka ‘const std::__cxx11::basic_string<char>’} to ‘bool’
   20 |                         if (cfg().relay__logging__dumpInAll) LI << "[" << msg->connId << "] dumpInAll: " << msg->payload;
      |                             ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~
      |                                   |
      |                                   const string {aka const std::__cxx11::basic_string<char>}
src/RelayIngester.cpp:29:39: error: could not convert ‘cfg().ConfigValues::relay__logging__dumpInEvents’ from ‘const string’ {aka ‘const std::__cxx11::basic_string<char>’} to ‘bool’
   29 |                             if (cfg().relay__logging__dumpInEvents) LI << "[" << msg->connId << "] dumpInEvent: " << msg->payload;
      |                                 ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                       |
      |                                       const string {aka const std::__cxx11::basic_string<char>}
src/RelayIngester.cpp:38:39: error: could not convert ‘cfg().ConfigValues::relay__logging__dumpInReqs’ from ‘const string’ {aka ‘const std::__cxx11::basic_string<char>’} to ‘bool’
   38 |                             if (cfg().relay__logging__dumpInReqs) LI << "[" << msg->connId << "] dumpInReq: " << msg->payload;
      |                                 ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                       |
      |                                       const string {aka const std::__cxx11::basic_string<char>}
src/RelayIngester.cpp:46:39: error: could not convert ‘cfg().ConfigValues::relay__logging__dumpInReqs’ from ‘const string’ {aka ‘const std::__cxx11::basic_string<char>’} to ‘bool’
   46 |                             if (cfg().relay__logging__dumpInReqs) LI << "[" << msg->connId << "] dumpInReq: " << msg->payload;
      |                                 ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                       |
      |                                       const string {aka const std::__cxx11::basic_string<char>}
make: *** [golpe/rules.mk:26: src/RelayIngester.o] Error 1

(I've reported a similar error before; if the solution here is to update / change versions lmk, happy to just switch to e.g. whatever version of Debian strfry is being developed on. Cheers!)

Yay spammers arrived 🎉

Need to think seriously about spam.

Basic rate limiting but also enabling bursts of events/msgs. So longer time window, bigger limit can achieve this..

Instead of 3/sec maybe 20/min.

If nobody is following the guy, he should get more penalties, or more rate limiting. If the pubkey is new, more rate limiting. It quickly turns to a statistics problem which bloat the relayer but these are some of the easiest things that can be implemented I think.

Another suggestion is incremental PoW. The relay requires more and more PoW from an IPv4 when it finds it is spamming. IPv6 is harder to control I think because it is cheaper. I don't know if there is a NIP for this. When a relay rejects the spam, client tries harder to find more PoW and resubmits it..

Similar messages can also be slowed down even though they come from different IP. The current spammer sends 'similar' good morning messages constantly. I don't think it is using different IP though, this is more for future proofing.

Track and limit by proximity to group of users

I want to run a relay financed by a tiny percentage of its users and strongly believe in the following being a way to align incentives for all clients and relay operators:

Implement authentication. The relay does only process REQests and EVENTs from clients linked to pubkeys
Measure resource use per pubkey: milliseconds spent on queries, query count, events sent, event kBs sent, etc.
Define group of primary users (how this works is independent of this issue but in my case it might be people I follow or people that pay $x/month).
Secondary users are follows of primary users etc.
Define limits depending on follows distance to primary users (0, 1, 2, 3, 4, 5, >5 for example)

With this in place, the relay can reject events from "distance >5" with links, longer than 50 chars, other than kind-1 or kind-4, ... It can delete events if no follows were achieved within 7 days, ...

On the other hand, the relay can treat "distance 1" and "distance 2" users almost as primary users and manually deal with primary users that actively follow spam accounts.

With users coming from Twitter for example quickly gaining follows from their Twitter followers, their on-boarding would only be mildly affected by this spam mitigation while the ">5" limits could be designed to gain followers without spamming.

With admission to "primary users" group carrying a cost, spam moderation would be paid for but for personal relays, the relay operator can choose to be the primary user with his follows being privileged and sponsored by him, too.

I was tempted to lay the above out in reply to #9 but it's a feature request independent of other spam mitigations that I would offer a $500 bounty for its implementation.

Probabilistic filters to reduce memory footprint of ReqWorker for monitoring

Clients with big filters can demand a lot of RAM on the relay. This can be reduced significantly by using probabilistic filters. While these filters are not great to query a database, they are great to check individual events against.

I had implemented this idea using Cuckoo filters here. While the REQ is received as normal JSON filter object, this is being converted into a probabilistic filter after EOSE if for example filtering for more than 10 pubkeys. The probabilistic filter is set to a false positive rate of 1/10.000.000 with the client is filtering out those false positives anyway.

systemd service unit example

The readme mentions "coming soon" for the service unit, so I just wanted to share mine which works while substantially restricting system access on Ubuntu 22.04:

[Unit]
Description=Nostr relay

[Service]
User=strfry
Group=strfry
WorkingDirectory=/opt/strfry
ExecStart=/opt/bin/strfry --config=strfry.conf relay
Restart=on-failure
RestartSec=5
ProtectHome=yes
NoNewPrivileges=yes
ProtectSystem=full
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

Creating a restricted strfry user:

useradd -mb /opt -k /dev/null -s $(which nologin) strfry

For clarity, here's my paths under /opt:

├── bin
│   └── strfry
└── strfry
    ├── strfry.conf
    └── strfry-db

ProtectSystem=full requires at least systemd version 232, otherwise you should do this instead:

ProtectSystem=strict
ProtectControlGroups=true
ProtectKernelModules=true
ProtectKernelTunables=yes

Docker support

Please add a docker file and a sample docker-compose.

Export fails

After running 'stream's for hours I tried to do an export:

$ ./strfry export | wc

strfry error: couldn't find leaf node in quadrable, corrupted DB?
2023-01-11 15:40:33.138 ( 0.064s) [main thread ]INFO| atexit

3391 22061 1768370

Detect which strfry instance is accessing the policy plugin

Finally found a potential use-case for extending the policy plugin input message: figuring out which instance of strfry originated the message (when multiple strfry policies use the same plugin on a shared machine).

Inspired by this thread: https://gitlab.com/soapbox-pub/strfry-policies/-/issues/4#note_1426085367

cc @Giszmo

btw, not sure it's even a good idea yet or the best solution to that problem, just want to brainstorm. strfry doesn't even know its own relay URL anyway, right? just the port it's running on.

If a Japanese character is set as the subscriber ID and a REQ message is sent, relay will return "NOTICE".

Describe the bug
"If a Japanese character is set as the subscriber ID and a REQ message is sent from the client to the relay, it will return ["NOTICE","ERROR: bad req: invalid character in subscription id"]."

To Reproduce
For example, sending ["REQ", "日本語のサブスクライバーID", {"kinds": 1, "limit": 10}] from the client to the relay.

Expected behavior
It is possible to use Japanese characters(and other language too) as the subscriber ID.

Additional context
For <subscription_id>, the following rules are defined in NIP-01.
<subscription_id>”<subscription_id> is an arbitrary, non-empty string of max length 64 chars, that should be used to represent a subscription.”
The specification for characters is not provided.

Compilation error: "src/events.h:4:10: fatal error: secp256k1_schnorrsig.h: No such file or directory"

When I run the make -j4 command, this is the full output I get, even after doing a git pull with the latest update you pushed.

g++ -std=c++2a -O3 -g -Wall -fPIC  -DDOCOPT_HEADER_ONLY -Iinclude -Ibuild -Isrc -Igolpe/external -Igolpe/external/config/include -Igolpe/external/json/include -Igolpe/external/PEGTL/include -Igolpe/external/hoytech-cpp -Igolpe/external/docopt.cpp -Igolpe/external/loguru -Igolpe/external/quadrable/include -MMD -MP -MT src/RelayWebsocket.o -MF src/RelayWebsocket.d -c src/RelayWebsocket.cpp -o src/RelayWebsocket.o

g++ -std=c++2a -O3 -g -Wall -fPIC  -DDOCOPT_HEADER_ONLY -Iinclude -Ibuild -Isrc -Igolpe/external -Igolpe/external/config/include -Igolpe/external/json/include -Igolpe/external/PEGTL/include -Igolpe/external/hoytech-cpp -Igolpe/external/docopt.cpp -Igolpe/external/loguru -Igolpe/external/quadrable/include -MMD -MP -MT src/RelayIngester.o -MF src/RelayIngester.d -c src/RelayIngester.cpp -o src/RelayIngester.o

g++ -std=c++2a -O3 -g -Wall -fPIC  -DDOCOPT_HEADER_ONLY -Iinclude -Ibuild -Isrc -Igolpe/external -Igolpe/external/config/include -Igolpe/external/json/include -Igolpe/external/PEGTL/include -Igolpe/external/hoytech-cpp -Igolpe/external/docopt.cpp -Igolpe/external/loguru -Igolpe/external/quadrable/include -MMD -MP -MT src/cmd_sync.o -MF src/cmd_sync.d -c src/cmd_sync.cpp -o src/cmd_sync.o

g++ -std=c++2a -O3 -g -Wall -fPIC  -DDOCOPT_HEADER_ONLY -Iinclude -Ibuild -Isrc -Igolpe/external -Igolpe/external/config/include -Igolpe/external/json/include -Igolpe/external/PEGTL/include -Igolpe/external/hoytech-cpp -Igolpe/external/docopt.cpp -Igolpe/external/loguru -Igolpe/external/quadrable/include -MMD -MP -MT src/RelayReqWorker.o -MF src/RelayReqWorker.d -c src/RelayReqWorker.cpp -o src/RelayReqWorker.o

In file included from src/WriterPipeline.h:7,
                 from src/cmd_sync.cpp:9:
src/events.h:4:10: fatal error: secp256k1_schnorrsig.h: No such file or directory
    4 | #include <secp256k1_schnorrsig.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [golpe/rules.mk:26: src/cmd_sync.o] Error 1
make: *** Waiting for unfinished jobs....
In file included from src/RelayServer.h:19,
                 from src/RelayIngester.cpp:1:
src/events.h:4:10: fatal error: secp256k1_schnorrsig.h: No such file or directory
    4 | #include <secp256k1_schnorrsig.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [golpe/rules.mk:26: src/RelayIngester.o] Error 1
In file included from src/RelayServer.h:19,
                 from src/RelayWebsocket.cpp:1:
src/events.h:4:10: fatal error: secp256k1_schnorrsig.h: No such file or directory
    4 | #include <secp256k1_schnorrsig.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [golpe/rules.mk:26: src/RelayWebsocket.o] Error 1
In file included from src/RelayServer.h:19,
                 from src/RelayReqWorker.cpp:1:
src/events.h:4:10: fatal error: secp256k1_schnorrsig.h: No such file or directory
    4 | #include <secp256k1_schnorrsig.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [golpe/rules.mk:26: src/RelayReqWorker.o] Error 1

Build error on latest master

It seems that after pulling the latest master, quadrable.h can not be found.

Any dependency I missed? My previous build worked; I just wanted to update it.

Full output:

root@birb:/opt/strfry# git pull
remote: Enumerating objects: 153, done.
remote: Counting objects: 100% (153/153), done.
remote: Compressing objects: 100% (102/102), done.
remote: Total 153 (delta 96), reused 108 (delta 51), pack-reused 0
Receiving objects: 100% (153/153), 58.00 KiB | 322.00 KiB/s, done.
Resolving deltas: 100% (96/96), completed with 7 local objects.
From https://github.com/hoytech/strfry
   fb21e10..798522a  web        -> origin/web
Fetching submodule golpe
From https://github.com/hoytech/golpe
   99fa9be..c0367d6  master     -> origin/master
Fetching submodule golpe/external/uWebSockets
From https://github.com/hoytech/uWebSockets
   8670f28..1e0fda7  master     -> origin/master
Already up to date.
root@birb:/opt/strfry# git submodule update
root@birb:/opt/strfry# git submodule ^C
root@birb:/opt/strfry# git submodule update --recursive --remote
warning: unable to rmdir 'external/quadrable': Directory not empty
Submodule path 'golpe': checked out 'c0367d6554bad33cecbf46763d0a1934891ff737'
Submodule path 'golpe/external/PEGTL': checked out 'eac50a85e3fd1ee3623cfa150eed457aa61f7b9e'
Submodule path 'golpe/external/config': checked out '5e726d1442beb225789ed0889e8aa4fbc75bea7a'
Submodule path 'golpe/external/docopt.cpp': checked out '400e6dd8e59196c914dcc2c56caf7dae7efa5eb3'
Submodule path 'golpe/external/hoytech-cpp': checked out '121eb4252d38e3a2c3359aee3329d1d4cdf4f512'
Submodule path 'golpe/external/json': checked out '330129305f15fbfba5e0716db25e245f0b4d8b0f'
Submodule path 'golpe/external/loguru': checked out '4adaa185883e3c04da25913579c451d3c32cfac1'
Submodule path 'golpe/external/parallel-hashmap': checked out '79cbd2dafd5aab3829064d1b48b71137623d8ff2'
Submodule path 'golpe/external/rasgueadb': checked out 'cb5631f0fa05622282b6c127b5817df1315254d2'
Submodule path 'golpe/external/uWebSockets': checked out '1e0fda756ad2b64a3e71428ad18cb75f1832781b'
root@birb:/opt/strfry# make setup-golpe
cd golpe && git submodule update --init
Submodule 'external/templar' (https://github.com/hoytech/templar.git) registered for path 'external/templar'
Cloning into '/opt/strfry/golpe/external/templar'...
Submodule path 'external/PEGTL': checked out '9afe8a71920b9dadf309a503d734143e1ff78b3e'
Submodule path 'external/config': checked out 'ab8c38a2d00e58dd004fd71da7f0e70749993fc1'
Submodule path 'external/docopt.cpp': checked out '6f5de76970be94a6f1e4556d1716593100e285d2'
Submodule path 'external/json': checked out 'd73d01389660084a8dbedd44eb674da57f26aba6'
Submodule path 'external/loguru': checked out '644f60dca77de3b0f718a03d370c8ebdf5f97968'
Submodule path 'external/parallel-hashmap': checked out '87ece91c6e4c457c5faac179dae6e11e2cd39b16'
Submodule path 'external/templar': checked out '13961f0c0bff435f045cf62864f2ef4c6f2730cc'
root@birb:/opt/strfry# make -j2
perl golpe/gen-fbs.pl
perl golpe/gen-config.pl
perl golpe/gen-golpe.h.pl
perl golpe/gen-main.cpp.pl
g++ -std=c++20 -O3 -g -Wall -fPIC  -DDOCOPT_HEADER_ONLY -Iinclude -Ibuild -Isrc -Igolpe/external -Igolpe/external/lmdbxx/include -Igolpe/external/config/include -Igolpe/external/json/include -Igolpe/external/PEGTL/include -Igolpe/external/hoytech-cpp -Igolpe/external/docopt.cpp -Igolpe/external/loguru -Igolpe/external/parallel-hashmap -MMD -MP -MT golpe/logging.o -MF golpe/logging.d -c golpe/logging.cpp -o golpe/logging.o
g++ -std=c++20 -O3 -g -Wall -fPIC  -DDOCOPT_HEADER_ONLY -Iinclude -Ibuild -Isrc -Igolpe/external -Igolpe/external/lmdbxx/include -Igolpe/external/config/include -Igolpe/external/json/include -Igolpe/external/PEGTL/include -Igolpe/external/hoytech-cpp -Igolpe/external/docopt.cpp -Igolpe/external/loguru -Igolpe/external/parallel-hashmap -MMD -MP -MT build/main.o -MF build/main.d -c build/main.cpp -o build/main.o
In file included from build/golpe.h:64,
                 from golpe/logging.cpp:1:
src/global.h:9:10: fatal error: quadrable.h: No such file or directory
    9 | #include <quadrable.h>
      |          ^~~~~~~~~~~~~
compilation terminated.
In file included from build/golpe.h:64,
                 from build/main.cpp:13:
src/global.h:9:10: fatal error: quadrable.h: No such file or directory
    9 | #include <quadrable.h>
      |          ^~~~~~~~~~~~~
compilation terminated.
make: *** [golpe/rules.mk:31: golpe/logging.o] Error 1
make: *** Waiting for unfinished jobs....
make: *** [golpe/rules.mk:31: build/main.o] Error 1

add the #nostr tag to this repo

Would it be possible to add the #nostr tag to this repo I'll show up in this nice list here: https://github.com/topics/nostr

Delete DMs

Sorry, I'm not sure that this is related on strfry. I hope to delete my broken DM. But AFAICS, strfry does accept kind 5 for DM (kind 4). Could you please support deletion for DM?

["NOTICE","ERROR: bad req: std::get: wrong index for variant"]

A few days ago I used strfry export to dump events, set up a new strfry server, and then used a combination of strfry import and strfry sync to try to get (most of?) the old events back over.

Everything was working until yesterday when things started to go bad. REQs would hang until I restarted strfry.

I was finally able to get a specific error for a REQ:

➜ echo '["REQ", "neinoienoei", {"authors": "79c2cae114ea28a981e7559b4fe7854a473521a8d22a66bbab9fa248eb820ff6"}]' | nostcat -s wss://relay.mostr.pub
["NOTICE","ERROR: bad req: std::get: wrong index for variant"]

I might just nuke this database.

My strfry.service (path to strfry.conf issue)

I read the home page (https://github.com/hoytech/strfry) and the deploy page for installation (https://github.com/hoytech/strfry/blob/master/docs/DEPLOYMENT.md). For me, the inferred methodology, is slightly different between the two pages. So I made a user and installed under a user (which seems to be the indication from the home page) whereas on the deployment page you install as no login.

Anyway, I am running Ubuntu 22.04 and for the life of me I could not get the relay to run as a service only as logging in and executing ./strfry relay from the install directory.

After 4-5 hours of banging my head against the wall I found the problem it was the path to the config file, this is how I got it work work. The user is "strfry" and the packages are installed is his home directory.

sudo nano /etc/systemd/system/strfry.service

[Unit]
Description=strfry relay service

[Service]
ExecStart=/home/strfry/strfry/strfry --config=/home/strfry/strfry/strfry.conf relay
LimitNOFILE=1000000
LimitNPROC=1000000

[Install]
WantedBy=multi-user.target

Request: NIP-50 Support

For reference:

https://github.com/nostr-protocol/nips/blob/master/50.md

Parser for x-forwarded-for does not work with IPv6 or multiproxy

I have setup haproxy (ssl term) ---> Nginx --> strfry
So x-forwarded-for header contains all proxies in the way, but parser is not happy from it, do not know if it is because of multiple IP or because of client uses IPv6

Websocket ]WARN| Couldn't parse IP from header x-forwarded-for: 2a00:abcd:401a:abcd:ca43:f3a7:c615:aaaa, 192.168.0.1
Websocket ]WARN| Couldn't parse IP from header x-forwarded-for: ffff:123.83.123.203, 192.168.0.1

Can I log the reason for "Request ID too long"?

This is more out of curiosity, really. But I saw this today when updating strfry to latest master.

[Ingester 1      ]INFO| sending error to [10]: bad req: subscription id too long

I'd love to know what the ID was.

Thanks!

Document all the secret strfry commands

I am motivated to build a CLI wrapper around the ./strfry command in my strfry-policies repo so I can control strfry programatically. But only a few of the commands are actually documented in the README. It would be good to get a list of everything that exists.

Compilation error: too few arguments to function ‘int secp256k1_schnorrsig_verify

Running into this when trying to make HEAD of master branch`. (First pre-coffee thought: do I have the right secp256k1 C++ lib, or should I be downloading and installing https://github.com/bitcoin-core/secp256k1 instead?)

# uname -a
Linux relay-001 5.15.0-52-generic #58-Ubuntu SMP Thu Oct 13 08:03:55 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

g++ -std=c++20 -O3 -g -Wall -fPIC  -DDOCOPT_HEADER_ONLY -Iinclude -Ibuild -Isrc -Igolpe/external -Igolpe/external/config/include -Igolpe/external/json/include -Igolpe/external/PEGTL/include -Igolpe/external/hoytech-cpp -Igolpe/external/docopt.cpp -Igolpe/external/loguru -Igolpe/external/quadrable/include -MMD -MP -MT src/events.o -MF src/events.d -c src/events.cpp -o src/events.o
src/events.cpp: In function ‘std::string nostrHash(const value&)’:
src/events.cpp:66:16: warning: ‘int SHA256_Init(SHA256_CTX*)’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
   66 |     SHA256_Init(&sha256);
      |     ~~~~~~~~~~~^~~~~~~~~
In file included from src/events.h:3,
                 from src/events.cpp:1:
/usr/include/openssl/sha.h:73:27: note: declared here
   73 | OSSL_DEPRECATEDIN_3_0 int SHA256_Init(SHA256_CTX *c);
      |                           ^~~~~~~~~~~
src/events.cpp:67:18: warning: ‘int SHA256_Update(SHA256_CTX*, const void*, size_t)’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
   67 |     SHA256_Update(&sha256, encoded.data(), encoded.size());
      |     ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from src/events.h:3,
                 from src/events.cpp:1:
/usr/include/openssl/sha.h:74:27: note: declared here
   74 | OSSL_DEPRECATEDIN_3_0 int SHA256_Update(SHA256_CTX *c,
      |                           ^~~~~~~~~~~~~
src/events.cpp:68:17: warning: ‘int SHA256_Final(unsigned char*, SHA256_CTX*)’ is deprecated: Since OpenSSL 3.0 [-Wdeprecated-declarations]
   68 |     SHA256_Final(hash, &sha256);
      |     ~~~~~~~~~~~~^~~~~~~~~~~~~~~
In file included from src/events.h:3,
                 from src/events.cpp:1:
/usr/include/openssl/sha.h:76:27: note: declared here
   76 | OSSL_DEPRECATEDIN_3_0 int SHA256_Final(unsigned char *md, SHA256_CTX *c);
      |                           ^~~~~~~~~~~~
src/events.cpp: In function ‘bool verifySig(secp256k1_context*, std::string_view, std::string_view, std::string_view)’:
src/events.cpp:79:102: error: invalid conversion from ‘secp256k1_xonly_pubkey*’ to ‘size_t’ {aka ‘long unsigned int’} [-fpermissive]
   79 |     return secp256k1_schnorrsig_verify(ctx, (const uint8_t*)sig.data(), (const uint8_t*)hash.data(), &pubkeyParsed);
      |                                                                                                      ^~~~~~~~~~~~~
      |                                                                                                      |
      |                                                                                                      secp256k1_xonly_pubkey*
src/events.cpp:79:39: error: too few arguments to function ‘int secp256k1_schnorrsig_verify(const secp256k1_context*, const unsigned char*, const unsigned char*, size_t, const secp256k1_xonly_pubkey*)’
   79 |     return secp256k1_schnorrsig_verify(ctx, (const uint8_t*)sig.data(), (const uint8_t*)hash.data(), &pubkeyParsed);
      |            ~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from src/events.h:4,
                 from src/events.cpp:1:
/usr/include/secp256k1_schnorrsig.h:158:48: note: declared here
  158 | SECP256K1_API SECP256K1_WARN_UNUSED_RESULT int secp256k1_schnorrsig_verify(
      |                                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~
make: *** [golpe/rules.mk:26: src/events.o] Error 1

strfry stream segfaults

I have a flaky internet connection and the process ended like this after hundreds of reconnect attempts:

2023-06-11 10:59:12.972 (52195.498s) [main thread     ]INFO| Attempting to connect to wss://relay.damus.io
2023-06-11 10:59:12.972 (52195.498s) [main thread     ]INFO| Websocket connection error
2023-06-11 10:59:17.972 (52200.498s) [main thread     ]INFO| Attempting to connect to wss://relay.damus.io
2023-06-11 10:59:17.973 (52200.498s) [main thread     ]INFO| Websocket connection error
2023-06-11 10:59:22.973 (52205.498s) [main thread     ]INFO| Attempting to connect to wss://relay.damus.io
Segmentation fault

Compression question

Not an issue, just a question.

In the docs it says there is a sliding window compression which applies across messages?

I was a bit confused by this, as I thought each nostr message is its own websocket message, and AFAICT websocket has only per-message compression? I was under the impression that the compression gains occur when a single websocket message is very large (and potentially fragmented into many frames). A large yesstr message, perhaps.

I am quite new to this whole compression thing, and would like to know if it really is possible to achieve cross-message compression. As you say, there is a lot of redundancy across nostr messages.

subscribe with prefixed filter ids (nip-01)

I'd like to subscribe to event ids starting with "0" as described in nip-01, but it seems no relays support that yet. Reason: there are some clients that support nip-13 POW and add nonce tag, but filtering by nonce on the client still means that all events are subscribed too which results in lots of bandwidth usage. Therefore I'd like to only subscribe to events starting with '0' or '00'

"ids": <a list of event ids or prefixes>
The ids and authors lists contain lowercase hexadecimal strings, which may either be an exact 64-character match, or a prefix of the event value. A prefix match is when the filter string is an exact string prefix of the event value. The use of prefixes allows for more compact filters where a large number of values are queried, and can provide some privacy for clients that may not want to disclose the exact authors or events they are searching for.

https://github.com/nostr-protocol/nips/blob/master/01.md

example: filter: { ids: ['0'] }

hoytech / strfry Goto Github PK

strfry's Issues

Recommend Projects

Recommend Topics

Recommend Org