Comments (12)
I can't reproduce, and I don't have any *BSD testing setup. But examining dotlock init code, there were several dotlock timeout variables that were being assigned to non-existent constants. So possibly this is related to that.
Can you try compiling with latest code and see if that fixes things? Specifcally commit 14971c9.
Edit: err, I am blind because those constants were defined. So that wouldn't be why this is broken. However, it's worth a shot to see if the commit helps anyway, since it does change dotlock init.
Otherwise, it's possible that dotlock is broken on *BSD.
from dovecot-fts-flatcurve.
Similar crashes:
2021-12-12T12:52:31.178Z symphytum dovecot: indexer-worker(sthen)<27729><ZLo7Bo/wtWHajwAA89wJGQ:jEVNCo/wtWFRbAAA89wJGQ>: Debug: fts-flatcurve: Xapian library version: 1.4.18
2021-12-12T12:52:31.178Z symphytum dovecot: indexer-worker(sthen)<27729><ZLo7Bo/wtWHajwAA89wJGQ:jEVNCo/wtWFRbAAA89wJGQ>: Panic: file file-dotlock.c: line 570 (dotlock_create): assertion failed: (lock_info.fd != -1)
2021-12-12T12:52:31.240Z symphytum dovecot: indexer-worker(sthen)<27729><ZLo7Bo/wtWHajwAA89wJGQ:jEVNCo/wtWFRbAAA89wJGQ>: Fatal: master: service(indexer-worker): child 27729 killed with signal 6 (core dumped)
2021-12-12T12:52:31.262Z symphytum dovecot: indexer-worker(sthen)<35110><ZLo7Bo/wtWHajwAA89wJGQ:bWdkD4/wtWEmiQAA89wJGQ>: Debug: fts-flatcurve: Xapian library version: 1.4.18
2021-12-12T12:52:31.262Z symphytum dovecot: indexer-worker(sthen)<35110><ZLo7Bo/wtWHajwAA89wJGQ:bWdkD4/wtWEmiQAA89wJGQ>: Panic: file file-dotlock.c: line 570 (dotlock_create): assertion failed: (lock_info.fd != -1)
2021-12-12T12:52:31.298Z symphytum dovecot: indexer-worker(sthen)<35110><ZLo7Bo/wtWHajwAA89wJGQ:bWdkD4/wtWEmiQAA89wJGQ>: Fatal: master: service(indexer-worker): child 35110 killed with signal 6 (core dumped)
#0 thrkill () at /tmp/-:3
#1 0x000005f9ea7cd3de in _libc_abort () at /usr/src/lib/libc/stdlib/abort.c:51
#2 0x000005fa18042c86 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:459
#3 0x000005fa180410a4 in fatal_handler_real (ctx=0x7f7ffffda860, format=<optimized out>, args=<optimized out>)
at failures.c:471
#4 0x000005fa18042041 in i_internal_fatal_handler (ctx=0x0,
format=0x6 <error: Cannot access memory at address 0x6>, args=0x0) at failures.c:872
#5 0x000005fa180412e2 in i_panic (format=0x6 <error: Cannot access memory at address 0x6>) at failures.c:524
#6 0x000005fa180461f0 in dotlock_create (dotlock=<optimized out>,
flags=(DOTLOCK_CREATE_FLAG_NONBLOCK | DOTLOCK_CREATE_FLAG_CHECKONLY | unknown: 32636),
write_pid=<optimized out>, lock_path_r=<optimized out>) at file-dotlock.c:570
#7 0x000005fa18045011 in file_dotlock_create_real (dotlock=0x5fa42b48580,
flags=(DOTLOCK_CREATE_FLAG_NONBLOCK | DOTLOCK_CREATE_FLAG_CHECKONLY | unknown: 32636)) at file-dotlock.c:635
#8 file_dotlock_create (set=0x7f7ffffdab68,
path=0x5fa42b22060 "/home/sthen/mdbox/mailboxes/INBOX/dbox-Mails/fts-flatcurve/flatcurve-dotlock",
flags=(DOTLOCK_CREATE_FLAG_NONBLOCK | DOTLOCK_CREATE_FLAG_CHECKONLY | unknown: 32636), dotlock_r=0x5fa42b3f288)
at file-dotlock.c:679
#9 0x000005fa6161d455 in fts_flatcurve_xapian_lock (backend=0x5fa42b54500)
at /usr/obj/ports/dovecot-fts-flatcurve-0.1.1pre20211212/dovecot-fts-flatcurve-55883dd3b606272f1954e99a90ff34c17eb2571f/src/fts-backend-flatcurve-xapian.cpp:527
#10 0x000005fa6161f996 in fts_flatcurve_xapian_db_populate (backend=0x5fa42b54500,
opts=(FLATCURVE_XAPIAN_DB_NOCREATE_CURRENT | FLATCURVE_XAPIAN_DB_IGNORE_EMPTY))
at /usr/obj/ports/dovecot-fts-flatcurve-0.1.1pre20211212/dovecot-fts-flatcurve-55883dd3b606272f1954e99a90ff34c17eb2571f/src/fts-backend-flatcurve-xapian.cpp:620
#11 0x000005fa6161aa80 in fts_flatcurve_xapian_read_db (backend=0x5fa42b54500,
opts=(FLATCURVE_XAPIAN_DB_NOCREATE_CURRENT | FLATCURVE_XAPIAN_DB_IGNORE_EMPTY))
at /usr/obj/ports/dovecot-fts-flatcurve-0.1.1pre20211212/dovecot-fts-flatcurve-55883dd3b606272f1954e99a90ff34c17eb2571f/src/fts-backend-flatcurve-xapian.cpp:688
#12 0x000005fa6161bb70 in fts_flatcurve_xapian_get_last_uid (backend=0x5fa42b54500, last_uid_r=0x7f7ffffdae6c)
at /usr/obj/ports/dovecot-fts-flatcurve-0.1.1pre20211212/dovecot-fts-flatcurve-55883dd3b606272f1954e99a90ff34c17eb2571f/src/fts-backend-flatcurve-xapian.cpp:1129
#13 0x000005fa61617ff3 in fts_backend_flatcurve_get_last_uid (_backend=0x5fa42b54500, box=0x5fa42b56448,
last_uid_r=0x7f7ffffdae6c)
at /usr/obj/ports/dovecot-fts-flatcurve-0.1.1pre20211212/dovecot-fts-flatcurve-55883dd3b606272f1954e99a90ff34c17eb2571f/src/fts-backend-flatcurve.c:151
#14 0x000005fa357efdc4 in fts_backend_get_last_uid (backend=0x0, box=0x6, last_uid_r=0x7f7ffffdae6c) at fts-api.c:106
#15 0x000005fa357fa322 in fts_mailbox_get_last_cached_seq (box=0x5fa42b56448, seq_r=0x7f7ffffdaea4)
at fts-storage.c:84
#16 0x000005fa357f80f6 in fts_mailbox_get_status (box=0x5fa42b56448,
--Type <RET> for more, q to quit, c to continue without paging--
items=(STATUS_MESSAGES | STATUS_LAST_CACHED_SEQ), status_r=0x7f7ffffdb060) at fts-storage.c:109
#17 0x000005f9df3b66c7 in mailbox_get_status (box=0x5fa42b56448, items=(STATUS_MESSAGES | STATUS_LAST_CACHED_SEQ),
status_r=0x7f7ffffdb060) at mail-storage.c:2169
#18 0x000005f7a69dde1e in index_mailbox_precache (conn=0x5fa42b37a00, box=0x5fa42b56448) at master-connection.c:92
#19 index_mailbox (conn=0x5fa42b37a00, user=<optimized out>, mailbox=<optimized out>,
max_recent_msgs=<optimized out>, what=<optimized out>) at master-connection.c:239
#20 master_connection_input_args (_conn=0x5fa42b37a00, args=<optimized out>) at master-connection.c:283
#21 0x000005fa18035ff2 in connection_input_default (conn=0x5fa42b37a00) at connection.c:95
#22 0x000005fa1805e6c1 in io_loop_call_io (io=0x5fa42b2a300) at ioloop.c:737
#23 0x000005fa180617fe in io_loop_handler_run_internal (ioloop=0x5fa42b5c600) at ioloop-kqueue.c:164
#24 0x000005fa1805ed31 in io_loop_handler_run (ioloop=0x5fa42b5c600) at ioloop.c:789
#25 0x000005fa1805eb38 in io_loop_run (ioloop=0x5fa42b5c600) at ioloop.c:762
#26 0x000005fa17fb5a95 in master_service_run (service=0x5fa42b37e00, callback=0x6) at master-service.c:863
#27 0x000005f7a69dd828 in main (argc=1, argv=0x7f7ffffdb458) at indexer-worker.c:76
Program terminated with signal SIGSEGV, Segmentation fault.
#0 fts_flatcurve_xapian_write_db_get (backend=0x823d6251100, xdb=0x0, wopts=FLATCURVE_XAPIAN_WDB_CREATE)
at /usr/obj/ports/dovecot-fts-flatcurve-0.1.1pre20211212/dovecot-fts-flatcurve-55883dd3b606272f1954e99a90ff34c17eb2571f/src/fts-backend-flatcurve-xapian.cpp:366
366 if (xdb->dbw != NULL)
(gdb) bt
#0 fts_flatcurve_xapian_write_db_get (backend=0x823d6251100, xdb=0x0, wopts=FLATCURVE_XAPIAN_WDB_CREATE)
at /usr/obj/ports/dovecot-fts-flatcurve-0.1.1pre20211212/dovecot-fts-flatcurve-55883dd3b606272f1954e99a90ff34c17eb2571f/src/fts-backend-flatcurve-xapian.cpp:366
#1 0x0000082437116785 in fts_flatcurve_xapian_write_db_current (backend=0x823d6251100,
opts=<error reading variable: Cannot access memory at address 0x7>)
at /usr/obj/ports/dovecot-fts-flatcurve-0.1.1pre20211212/dovecot-fts-flatcurve-55883dd3b606272f1954e99a90ff34c17eb2571f/src/fts-backend-flatcurve-xapian.cpp:660
#2 fts_flatcurve_xapian_init_msg (ctx=0x823d627f2c8)
at /usr/obj/ports/dovecot-fts-flatcurve-0.1.1pre20211212/dovecot-fts-flatcurve-55883dd3b606272f1954e99a90ff34c17eb2571f/src/fts-backend-flatcurve-xapian.cpp:1197
#3 0x000008243711348d in fts_backend_flatcurve_update_set_build_key (_ctx=0x823d627f2c8, key=0x7f7ffffcb9c0)
at /usr/obj/ports/dovecot-fts-flatcurve-0.1.1pre20211212/dovecot-fts-flatcurve-55883dd3b606272f1954e99a90ff34c17eb2571f/src/fts-backend-flatcurve.c:254
#4 0x00000824426e7171 in fts_backend_update_set_build_key (ctx=0x823d627f2c8, key=0x7f7ffffcb9c0) at fts-api.c:178
#5 0x00000824426e842a in fts_build_mail_header (ctx=<optimized out>, block=<optimized out>) at fts-build-mail.c:160
#6 fts_build_mail_real (update_ctx=0x823d627f2c8, mail=<optimized out>, retriable_err_msg_r=<optimized out>,
may_need_retry_r=<optimized out>) at fts-build-mail.c:576
#7 fts_build_mail (update_ctx=0x823d627f2c8, mail=<optimized out>) at fts-build-mail.c:625
--Type <RET> for more, q to quit, c to continue without paging--
#8 0x00000824426eed23 in fts_mail_index (_mail=0x823d625a048) at fts-storage.c:540
#9 fts_mail_precache (_mail=0x823d625a048) at fts-storage.c:561
#10 0x00000823e18b5cb5 in mail_precache (mail=0x823d625a048) at mail.c:463
#11 0x0000082146906205 in index_mailbox_precache (conn=0x823d6270a00, box=<optimized out>) at master-connection.c:119
#12 index_mailbox (conn=0x823d6270a00, user=<optimized out>, mailbox=<optimized out>,
max_recent_msgs=<optimized out>, what=<optimized out>) at master-connection.c:239
#13 master_connection_input_args (_conn=0x823d6270a00, args=<optimized out>) at master-connection.c:283
#14 0x000008237ee78ff2 in connection_input_default (conn=0x823d6270a00) at connection.c:95
#15 0x000008237eea16c1 in io_loop_call_io (io=0x823d624e300) at ioloop.c:737
#16 0x000008237eea47fe in io_loop_handler_run_internal (ioloop=0x823d6267300) at ioloop-kqueue.c:164
#17 0x000008237eea1d31 in io_loop_handler_run (ioloop=0x823d6267300) at ioloop.c:789
#18 0x000008237eea1b38 in io_loop_run (ioloop=0x823d6267300) at ioloop.c:762
#19 0x000008237edf8a95 in master_service_run (service=0x823d626fa00, callback=0x0) at master-service.c:863
#20 0x0000082146905828 in main (argc=1, argv=0x7f7ffffcc048) at indexer-worker.c:76
from dovecot-fts-flatcurve.
One possible change (though I'm not 100% sure), I don't seem able to trigger the dotlock crash after expunging any more
from dovecot-fts-flatcurve.
I'm going to see i I can setup a local container with openbsd and run the test suite and see if I can get the assertion.
from dovecot-fts-flatcurve.
Unfortunately, it is just too much work to get testing up and running on non-Linux at this time.
If you could run the test suite (https://github.com/slusarz/dovecot-fts-flatcurve/blob/master/.github/actions/dovecot-fts-flatcurve-test/fts-flatcurve-test.sh) locally, that would be useful. Or, better still, tracing why the assert is being called (i.e. why is lock_info.fd != -1).
from dovecot-fts-flatcurve.
from dovecot-fts-flatcurve.
Declaring an enum should initialize it to zero. But maybe you can explicit set the enum to zero and see if that fixes things?
Change to:
enum dotlock_create_flags flags = 0;
from dovecot-fts-flatcurve.
from dovecot-fts-flatcurve.
Ugh, sorry, forgot that C++ isn't smart enough to cast enums itself that aren't explicitly defined.
You should change that line to:
enum dotlock_create_flags flags = (enum dotlock_create_flags) 0;
from dovecot-fts-flatcurve.
from dovecot-fts-flatcurve.
Thanks for your efforts in this! I added e3af260, which is a cleaned-up version of your patch. All tests are passing, so if you could verify that it works for you I think we will be able to close this ticket as resolved.
from dovecot-fts-flatcurve.
Yes, that's looking good, thanks for your help Michael.
from dovecot-fts-flatcurve.
Related Issues (20)
- Build failing HOT 1
- Searching for phrases with IMAP SEARCH vs. doveadm search, v0.2.0 vs. v0.3.0 HOT 3
- Feature Request: Change Flatcurve Index Location via config parameter HOT 1
- Separate directory for indexing? HOT 7
- Search by some email addresses is not working HOT 3
- Virtual search doesn't consider unindexed messages HOT 2
- backtrace during indexing HOT 2
- Inconsistent results when searching in attachment HOT 4
- Filename too long errors during FTS optimize HOT 5
- Panic when running a search HOT 2
- Inaccurate results while searching for a phrase in subject HOT 2
- Excessively long string sent to xapian to index HOT 1
- Using fts-flatcure with Dovecot 2.3.13 / Debian Bookworm HOT 2
- Example config anywhere? HOT 2
- Error: fts: Failed to initialize backend 'flatcurve': Unknown backend HOT 2
- imap segfault with fts flatcurve HOT 11
- remove use of .la files HOT 7
- fts_filter_normalizer_icu: libicu support not built in HOT 6
- Assertion crash for search in virtual mailbox
- Segfault with Dovecot 1:2.3.21+dfsg1-2 from Debian HOT 17
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dovecot-fts-flatcurve.