manual import of hashes fails about bitmagnet HOT 8 CLOSED

Thanks, let me look into this and get back to you. I haven't tried the importer in a little while and it definitely needs more work anyway....

from bitmagnet.

This should be fixed in the latest release, could you give it another try?

from bitmagnet.

Perfect, thanks for the quick fix. I had the same issue and works with the latest update.

❯ sqlite3 -json -batch rarbg_db.sqlite "$(cat rarbg-import.sql)" \
  | jq -r --indent 0 '.[] | . * { source: "rarbg" } | . + if .imdb != null then { externalIds: { imdb: .imdb } } else {} end | del(.imdb) | del(..|nulls)' \
  | curl --verbose -H "Content-Type: application/json" --data-binary @- http://localhost:3333/import

*   Trying 127.0.0.1:3333...
* Connected to localhost (127.0.0.1) port 3333 (#0)
> POST /import HTTP/1.1
> Host: localhost:3333
> User-Agent: curl/7.88.1
> Accept: */*
> Content-Type: application/json
> Content-Length: 269567440
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue
< HTTP/1.1 200 OK
< Vary: Origin
< Date: Thu, 05 Oct 2023 16:46:48 GMT
< Content-Type: text/plain; charset=utf-8
< Connection: close
< Transfer-Encoding: chunked
< 
1000 items imported
2000 items imported
3000 items imported
4000 items imported
5000 items imported
6000 items imported
7000 items imported
8000 items imported
....
497000 items imported
498000 items imported
499000 items imported
500000 items imported
....

from bitmagnet.

Good to hear, thanks! Closing this now....

from bitmagnet.

Hey, sorry to resurrect this issue but I have a similar problem.
I followed the example as well but I get this:

ERROR   httpserver/httpserver.go:52     error adding item       {"err": "encoding/hex: invalid byte: U+006D 'm'"}

I tried with multiple lines and with a single one.
Am I doing something wrong?

I am stupid, sorry.
I edited the sqlite when I got it as I added the "magnet" part of the infohashes, not remembering until I tried doing a single one without it.

from bitmagnet.

Hi @bonny1992 this is working for me using 0.1.0. If you could redact any copyrighted material and post a line of JSON here I'll see if anything looks obviously wrong - please do not post any complete info hashes, just remove 1 character is fine.

from bitmagnet.

Hi @bonny1992 this is working for me using 0.1.0. If you could redact any copyrighted material and post a line of JSON here I'll see if anything looks obviously wrong - please do not post any complete info hashes, just remove 1 character is fine.

No, I'm sorry, you're right. It's working fine.
I've edited my message (didn't expect you to reply this fast!), but I had the magnet:... part inside the infohash column, so it was failing because bitmagnet wasn't expecting a string that long (and not hexadecimal).

BTW, sorry to derail this issue, but is there any place where I can find what are the expected fields for an import? There's a very old csv I'd like to import (from an Italian tracker that closed some years ago) and for sure that doesn't have any imdb id :)

from bitmagnet.

is there any place where I can find what are the expected fields for an import?

@bonny1992 I still need to add a proper schema for the import. You can see the data structure here: https://github.com/bitmagnet-io/bitmagnet/blob/main/internal/importer/importer.go#L44 - at a minimum you'll need source, infoHash, name, size, publishedAt - the rest are optional.

from bitmagnet.