ocsigen / lwt Goto Github PK
View Code? Open in Web Editor NEWOCaml promises and concurrent I/O
Home Page: https://ocsigen.org/lwt
License: MIT License
OCaml promises and concurrent I/O
Home Page: https://ocsigen.org/lwt
License: MIT License
It should apparently be done (required by OpenSSL).
src/unix/lwt_unix_unix.c makes the assumption that poll(&pollfd,1,0)
will not block and will always return immediately. Indeed, this is what the man page for poll
states.
Unfortunately, poll waits for a response from the filesystem which, if unresponsive, will never arrive. In this case, poll will block the process.
I do not see a means to read an fd without first polling for readability with Lwt_unix
. Is there a way? I also do not see any way to get an 'a Lwt_unix.job
for use with Lwt_unix.run_job
. I would really like to have a way to attempt a read (in another thread) without first polling in the main thread.
I relocated every opam files from the ocsigen-dev repository to each individual project (so that we can track them using git and use jenkins to update the repository). I noticed lwt was not in the repository, Do I add it ?
If so, do I use the opam file from the main opam repo ?
I configured lwt to install beneath the current directory:
$ ocaml setup.ml -configure --enable-unix --destdir (pwd)/install
Which includes the output:
Prepend a path when installing package: .............. /home/tim/dev/ocaml/lwt/install
And the -build
action runs fine. But then the -install
action fails, with:
$ ocaml setup.ml -install
ocamlfind: Package lwt is already installed
- (file /home/tim/.opam/4.01.0/lib/lwt/META already exists)
Please excuse if this is a rookie misunderstanding - I have not compiled and installed my own ocaml packages before, so maybe it's not even possible to install to a nonstandard location. I hope that's not the case - I don't want to install this version globally, as that would potentially break anything expecting the real opam package.
[vb@haramix ~/code/lwt]% ./configure --enable-ssl
./setup-dev.exe -configure --enable-ssl
W: Field XMETAType is set but matching plugin is not enabled.
W: Field XMETAType is set but matching plugin is not enabled.
W: Field XMETAType is set but matching plugin is not enabled.
W: No exported module defined for library lwt-simple-top
W: No exported module defined for library lwt-syntax-options
W: Field XMETAType is set but matching plugin is not enabled.
W: Field XMETAType is set but matching plugin is not enabled.
W: Field XMETAType is set but matching plugin is not enabled.
W: No exported module defined for library lwt-simple-top
W: No exported module defined for library lwt-syntax-options
E: Cannot find buildable internal library 'lwt-unix' when checking build depends
E: Failure("1 configuration error")
make: *** [configure] Erreur 1
zsh: exit 2 ./configure --enable-ssl
Currently the type of Lwt_unix.read
has string
instead of Bytes.t
, and doesn't match the type of Unix.read
when compiling with -safe-string
.
Newer ocamlfind versions provide a backward compatible Bytes module, so you should be able to make this change and still support OCaml 3.12.1.
ocaml -safe-string
OCaml version 4.02.0
# #use "topfind";;
- : unit = ()
Findlib has been successfully loaded. Additional directives:
#require "package";; to load a package
#list;; to list the available packages
#camlp4o;; to load camlp4 (standard syntax)
#camlp4r;; to load camlp4 (revised syntax)
#predicates "p,q,...";; to set these predicates
Topfind.reset();; to force that packages will be reloaded
#thread;; to enable threads
- : unit = ()
# #require "lwt.unix";;
/home/edwin/.opam/4.02.0/lib/lwt: added to search path
/home/edwin/.opam/4.02.0/lib/lwt/lwt.cma: loaded
/home/edwin/.opam/4.02.0/lib/lwt/lwt-log.cma: loaded
/home/edwin/.opam/4.02.0/lib/ocaml/unix.cma: loaded
/home/edwin/.opam/4.02.0/lib/ocaml/bigarray.cma: loaded
/home/edwin/.opam/4.02.0/lib/lwt/lwt-unix.cma: loaded
# Lwt_unix.read;;
- : Lwt_unix.file_descr -> string -> int -> int -> int Lwt.t = <fun>
# Unix.read;;
- : Unix.file_descr -> bytes -> int -> int -> int = <fun>
This test program runs an echo subprocess in a loop. It should go on forever, but with the Lwt_glib.install line it stops randomly for me (e.g. after 699, 17 and 111 iterations when I just tried it three times.)
(* ocamlfind ocamlc -o test -package lwt.unix,lwt,lwt.glib -linkpkg test.ml *)
let () =
Lwt_glib.install ();
let x = ref 0 in
while true do
let a = Lwt_main.run @@ Lwt_process.pread ("", [| "echo"; "-n"; string_of_int !x|]) in
incr x;
print_endline a
done
Happens with Lwt 2.4.3 and Git master (9c7f485).
Strace shows it doing this endlessly:
poll([{fd=4, events=POLLIN}, {fd=5, events=POLLIN}, {fd=3, events=POLLIN}], 3, 4294966296) = 1 ([{fd=5, revents=POLLHUP}])
poll([{fd=4, events=POLLIN}, {fd=5, events=POLLIN}, {fd=3, events=POLLIN}], 3, 4294966296) = 1 ([{fd=5, revents=POLLHUP}])
...
Tested on x86_64 Arch Linux with OCaml 4.0.1 from OPAM.
I'm trying to read the output from GPG. It works on Linux but not on Windows. Here's a simple test case:
let () =
let out = Lwt_main.run (Lwt_process.pread ("", [| "gpg"; "--version" |])) in
print_endline ("Got: " ^ out)
On Linux:
Got: gpg (GnuPG) 2.0.21
...
On Windows (Windows 7, OCaml 4.00.1, LWT 2.4.3, MinGW 32-bit):
Fatal error: exception Unix.Unix_error(31, "bytes_read", "")
http://ocsigen.org/lwt/install ends with 2.4.3
ocaml setup.ml -build fails with mesage 👍
File "src/unix/lwt_unix.mli", line 301, characters 5-233:
Error: This variant or record definition does not match that of type
Unix.open_flag
The field O_CLOEXEC is only present in the original definition.
Exit code 2 while executing this command:
findlib: [WARNING] Interface topdirs.cmi occurs in several directories: /Users/avsm/.opam/4.02.1/lib/ocaml, /Users/avsm/.opam/4.02.1/lib/ocaml/compiler-libs
/Users/avsm/.opam/4.02.1/bin/ocamlfind ocamlc -a -o ppx/ppx.cma
/Users/avsm/.opam/4.02.1/bin/ocamlfind ocamlopt -a -o ppx/ppx.cmxa
+ /Users/avsm/.opam/4.02.1/bin/ocamlfind ocamlopt -a -o ppx/ppx.cmxa
ar: no archive members specified
usage: ar -d [-TLsv] archive file ...
ar -m [-TLsv] archive file ...
ar -m [-abiTLsv] position archive file ...
ar -p [-TLsv] archive [file ...]
ar -q [-cTLsv] archive file ...
ar -r [-cuTLsv] archive file ...
ar -r [-abciuTLsv] position archive file ...
ar -t [-TLsv] archive [file ...]
ar -x [-ouTLsv] archive [file ...]
File "_none_", line 1:
Error: Error while creating the library ppx/ppx.a
Command exited with code 2.
Empty ar
not allowed on OSX
related : ocsigen/ocsigenserver#2
I run LWT on a NetBSD system. I recently submitted a patch [/pull/79] allowing the configuration to properly recognize that libev
is present on a system using PKGSRC. Now that I'm able to properly install and use it, it appears that KQUEUEs (NetBSD's most efficient handle multiplexer) aren't being used. I also don't see any way for me to insist that LWT use KQUEUEs with libev
.
When I installed libev
, its config detection found kqueue.h
, et al. LWT's config now finds libev
. But when I run my program, I see (via top
) that it's blocking via select
, not kqueue
like it should be.
write_bytes
and read_bytes
allocate temporary string buffers in order to perform the IO using SSL.read
and SSL.write
, which work with strings (and not Lwt_bytes.t
).
A preliminary fix would be to allocate one string buffer per channel (same size as the Lwt_bytes one) and reuse it.
In the long-term, it'll be nice to have a Ssl.write_bigarray
able to work with Lwt_bytes.t
directly.
It would also get rid of not one but two(!) copies (Ssl mallocs a buffer & copies the data there before releasing the runtime).
Lwt_daemon.daemonize () should redirect stdout and stderr to syslog.
This does not work. I investigated and found an issue in the code (see function redirect in lwt_daemon.ml), but this did not seem sufficien to solve the issue.
It would be nice to have a few words describing the meaning of the various kinds of redirections in Lwt_process
. As I understand it Keep` means sharing stdX with the parent,
Dev_null and ``Close
are self-explanatory (but there may be subtleties worth mentioning). I have however no clear explanation of the difference between FD_move` and
FD_copy`.
Also, when calling Lwt_process.exec
with an FD_move` redirection, the file descriptor should be closed while it is not the case with
FD_copy`. I think this is also worth mentionning.
This confusion is most probably a consequence of my own ignorance of Unix programming, but some more explanations would not hurt I think. Maybe pointers to relevant external documents is enough (like the unix ocaml book)?
I am not sure to totally understand why Lwt_bytes.map_file
doesn't take a Lwt_unix.file_descr
as argument.
That API is not as easy to use as it should (see for instance https://github.com/samoht/ocaml-git/blob/master/lib/gitUnix.ml#L53)
$ uname -a
Darwin _.local 13.1.0 Darwin Kernel Version 13.1.0: Thu Jan 16 19:40:37 PST 2014; root:xnu-2422.90.20~2/RELEASE_X86_64 x86_64
===== ERROR while installing lwt.2.4.5 =====
#1 error generated.
'opam install lwt' failed.
It seems there's no sys/eventfd.h in Mac OS X (according to google research)
I needed to find elements stored in Lwt_sequence.t. I have added the required functions:
https://github.com/nojb/lwt/compare/ocsigen:master...master
Is there a reason not to include them in Lwt?
Thanks!
lwt.syntax extension considers "finally" as a keyword and rejects it used as a label name:
File "blah.ml", line 157, characters 18-27:
`finally' is a keyword, it cannot be used as label name
Is it possible to remove this restriction? There are some functions which have finally as label names like Core.Exn.protect : f:(unit -> 'a) -> finally:(unit -> unit) -> 'a
, and they cannot be used directly with lwt.syntax.
Comparing Lwt_ssl to Ssl (https://github.com/savonet/ocaml-ssl/blob/master/src/ssl.mli), it seems to me that Lwt_ssl misses some functions (for example: open_connextion_, input__ and output_*, …).
Is it possible to complete the binding ?
Since ppx is now properly merged and 4.02.0 is released, it would be great if downstream code could use ppx syntax.
Using lwt.syntax
, the following fails:
lwt () = let i = ref 0 in Lwt_io.printlf "Hello %d" !i >> incr i; Lwt_io.printlf "Hello %d" !i
The error is (with incr i
as the offending piece):
Error: This expression has type unit but an expression was expected of type 'a Lwt.t
@whitequark reports an interesting bug on avsm/ocaml-cohttp#57
If we ignore SIGPIPE, then writes to closed sockets will continue for some time, leading to memory leaks unless Lwt_unix.getsockopt_error
is called after the write to ensure that it succeeded.
See gist for repro case: https://gist.github.com/whitequark/5b4f7b026e96394042b1
Fixing this using fds is fairly easy, but cohttp uses Lwt_io channels, and we cannot retrieve the fd from it. It would probably be best to fix this within Lwt_io itself though, by checking for socket errors after flushing a write.
Hi!
Nowadays I (as well as several other people I would guess) am using opam for installing basically every ocaml library I use and among them is lwt.
However, when installing lwt opam will by default only install core (and unix most probably), but not the sublibraries which depend on some optional dependencies (unless these are present, ofc).
This creates (imho) two problems :
What I would like to see is lwt distributed as several packages : lwt, lwt-ssl, lwt-text, lwt-react, ...
What do you think?
Here is the log according to opam (but I also get this error if I compile "by hand").
===== ERROR while installing lwt.2.4.5 =====
# opam-version 1.1.2 (007c9356a3129fced7c3d7016216839013dbfb90)
# os cygwin
# command make build
# path /home/vagrant/.opam/system/build/lwt.2.4.5
# compiler system (4.01.0)
# exit-code 2
# env-file /home/vagrant/.opam/system/build/lwt.2.4.5/lwt-1788-ffb3fd.env
# stdout-file /home/vagrant/.opam/system/build/lwt.2.4.5/lwt-1788-ffb3fd.out
# stderr-file /home/vagrant/.opam/system/build/lwt.2.4.5/lwt-1788-ffb3fd.err
### stdout ###
# ...[truncated]
# ^
# src/unix/lwt_unix_stubs.c:1136:3: warning: implicit declaration of function ‘sigaltstack’ [-Wimplicit-function-declaration]
# sigaltstack(&new_stack, &old_stack);
# ^
# src/unix/lwt_unix_stubs.c:1142:21: error: ‘SA_ONSTACK’ undeclared (first use in this function)
# new_sa.sa_flags = SA_ONSTACK;
# ^
# src/unix/lwt_unix_stubs.c:1142:21: note: each undeclared identifier is reported only once for each function it appears in
# Command exited with code 2.
# Makefile:27: recipe for target 'build' failed
### stderr ###
# ocamlfind: Package `wikidoc' not found
# E: Failure("Command ''/usr/bin/ocamlbuild' syntax/optcomp.cma syntax/optcomp.cmxa syntax/optcomp.a src/core/lwt.cma src/core/lwt.cmxa src/core/lwt.a src/logger/lwt-log.cma src/logger/lwt-log.cmxa src/logger/lwt-log.a src/unix/liblwt-unix_stubs.a src/unix/dlllwt-unix_stubs.so src/unix/lwt-unix.cma src/unix/lwt-unix.cmxa src/unix/lwt-unix.a src/simple_top/lwt-simple-top.cma src/simple_top/lwt-simple...[truncated]
# make: *** [build] Error 1
'opam install lwt' failed.
From d77e4b67c67f47d2bd3b680a04604c9552d0f14d Mon Sep 17 00:00:00 2001
From: ygrek <[email protected]>
Date: Wed, 12 Feb 2014 22:20:38 +0800
Subject: [PATCH] fix potential deadlock in lwt_unix_recv_notifications
caml_alloc_tuple may provoke a thread switch and an unfortunate
send_notification call will deadlock on notification_mutex
still being held by recv_notifications
---
src/unix/lwt_unix_stubs.c | 19 +++++++++++++++++--
1 file changed, 17 insertions(+), 2 deletions(-)
diff --git a/src/unix/lwt_unix_stubs.c b/src/unix/lwt_unix_stubs.c
index 168b492..00863ba 100644
--- a/src/unix/lwt_unix_stubs.c
+++ b/src/unix/lwt_unix_stubs.c
@@ -587,7 +587,7 @@ value lwt_unix_send_notification_stub(value id)
value lwt_unix_recv_notifications()
{
- int ret, i;
+ int ret, i, current_index;
value result;
#if !defined(LWT_ON_WINDOWS)
sigset_t new_mask;
@@ -616,8 +616,23 @@ value lwt_unix_recv_notifications()
unix_error(error, "recv_notifications", Nothing);
}
#endif
+
+ do {
+ /*
+ release the mutex while calling caml_alloc,
+ which may call gc and switch the thread,
+ resulting in a classical deadlock,
+ when thread in question tries another send
+ */
+ current_index = notification_index;
+ lwt_unix_mutex_unlock(¬ification_mutex);
+ result = caml_alloc_tuple(current_index);
+ lwt_unix_mutex_lock(¬ification_mutex);
+ /* check that no new notifications appeared meanwhile (rare) */
+ }
+ while (current_index != notification_index);
+
/* Read all pending notifications. */
- result = caml_alloc_tuple(notification_index);
for (i = 0; i < notification_index; i++)
Field(result, i) = Val_long(notifications[i]);
/* Reset the index. */
--
1.7.10.4
I pinned lwt
to master
to try to get ppx
working on os x (2.4.6 does not work on os x) and opam install
failed because of some undefined symbols in lwt-unix
: _lwt_unix_get_affinity
, _lwt_unix_set_affinity
, _lwt_unix_get_cpu
.
See https://gist.github.com/nojb/730a46fa77b1e3c46f77 for details.
I've been experimenting with js_of_ocaml for weeks now and all of a sudden when I did an opam upgrade
lwt fails to compile. It's defaulting to 2.4.4 but 2.4.3 fails with the same error. The only thing I can think of that might have changed is I'm on OS X 10.9 and there was an Xcode update recently (with a corresponding command line tools update).
Here's the details from opam:
E: Failure("Command 'ocaml discover.ml -ocamlc /Users/spyder/.opam/4.01.0/bin/ocamlc.opt -ext-obj .o -exec-name a.out -use-libev false -os-type Unix -use-glib false -ccomp-type cc -use-pthread true -use-unix true -android-target false -libev_default false' terminated with error code 1")
not checking for pkg-config
not checking for libev
testing for pthread: ........................... unavailable
not checking for glib
The following recquired C libraries are missing: pthread.
Please install them and retry. If they are installed in a non-standard location
or need special flags, set the environment variables <LIB>_CLFAGS and <LIB>_LIBS
accordingly and retry.
For example, if libev is installed in /opt/local, you can type:
export LIBEV_CLFAGS=-I/opt/local/include
export LIBEV_LIBS=-L/opt/local/lib
To compile without libev support, use ./configure --disable-libev ...
As near as I can tell, pthread does exist.
It is released accorded to the website but has no corresponding tag in the git repo. This is confusing.
Also, it would be nice to release a 2.4.4 soon, given the amount of changes that have been contributed in the repo since 2.4.3.
It would be great if Lwt did not depend on camlp4.
The core of Lwt doesn't use its syntax extension itself, and pa_optcomp is easily replaced with cppo (I am willing to do this work). However, some of non-core modules use the syntax quite a bit. Would you be willing to accept patches that convert the syntax into core function calls? Alternatively, it's possible to alter the buildsystem so that release builds include preprocessed code.
not checking for pkg-config
not checking for libev
testing for pthread: ........................... unavailable
not checking for glib
2.4.5 works fine, I suspect b8f92b5 breaks the discovery of pthread library on FreeBSD. I think http://www.freebsd.org/doc/en/books/porters-handbook/dads-pthread.html is related to this.
This program usually segfaults for me on Windows (Windows 7, OCaml 4.01.0, Lwt 2.4.5):
let () =
Lwt_main.run (
let gpg = "C:\\wodi32\\bin\\echo.exe" in
let child = new Lwt_process.process_full (gpg, [| gpg |]) in
ignore child#close;
Lwt.return ()
)
(you can set gpg
to any existing executable; I first saw the bug with gpg but it happens with echo too)
_tags
is:
true: package(lwt, lwt.unix)
true: warn(A)
Output:
$ ocamlbuild -use-ocamlfind test.native
$ ./test.native
Segmentation fault
I'm currently testing using WODI, but I saw the same problem on a plain Cygwin system.
Building lwt-2.4.5 fails when "--enable-react" is passed to configure
I believe this is caused by commit bec7372
Beginning with lwt here. I compile the following code with ocamlfind ocamlc -syntax camlp4o -package lwt.syntax -package lwt.unix -linkpkg -o try try.ml
:
open Lwt
lwt () =
let open Lwt_unix in
let f = socket PF_INET SOCK_STREAM 0 in
try_lwt
connect f (ADDR_INET ((Unix.inet_addr_of_string "127.0.0.1"), 8000))
with
exn as e -> print_endline (Printexc.to_string e); return ()
The compiler reports the following although the match case is used:
File "try.ml", line 9, characters 4-65:
Warning 11: this match case is unused.
No warning is given when I either:
as e
and its use from my example)try
insteadMy compiler, version 4.02.1, was compiled by opam. lwt's version is 2.4.6.
Lwt_preemptive.detach
stops working when calling Lwt_daemon.demonize
With this piece of code, "Detached after daemon"
never gets to the log-file:
let say fmt = Printf.ksprintf (Printf.eprintf "%s\n%!") fmt
open Lwt
let () =
Lwt_main.run (
Lwt_preemptive.detach (fun () ->
say "Detached before daemon !") ()
>>= fun () ->
Lwt_log.file ~mode:`Append ~perm:0o600 ~file_name:"loooooggg-file" ()
>>= fun logger ->
Lwt_daemon.daemonize ~stdout:(`Log logger) ~stderr:(`Log logger) ~directory:(Sys.getcwd ()) ();
say "Daemonized !";
Lwt_preemptive.detach (fun () ->
say "Detached after daemon!") ()
>>= fun () ->
say "After detach";
return ()
)
EDIT: the second Lwt_preemptive.detach
hangs by the way (does not crash or anything).
Lwt_react.E.next
's waiter is woken up with Lwt.wakeup
within the current React
update step, so a second call to Lwt_react.E.next
returns the same occurrence
as the first one, unless there's a context switch between them.
I propose this fix, but would appreciate an extra pair of eyes on it:
diff --git a/src/react/lwt_react.ml b/src/react/lwt_react.ml
index 2cb3920..f74f847 100644
--- a/src/react/lwt_react.ml
+++ b/src/react/lwt_react.ml
@@ -28,7 +28,7 @@ module E = struct
let next ev =
let waiter, wakener = Lwt.task () in
- let ev = map (fun x -> Lwt.wakeup wakener x) (once ev) in
+ let ev = map (fun x -> Lwt.wakeup_later wakener x) (once ev) in
Lwt.on_cancel waiter (fun () -> stop ev);
waiter
This code exemplifies the issue:
(* ocamlfind ocamlopt -package lwt.react,lwt.unix,lwt.syntax -o treact treact.ml -syntax camlp4o -linkpkg *)
open Printf
let next ev =
let waiter, wakener = Lwt.task () in
let ev = React.E.map (fun x -> Lwt.wakeup_later wakener x) (React.E.once ev) in
Lwt.on_cancel waiter (fun () -> React.E.stop ev);
waiter
let print s =
let rec loop e =
lwt x = Lwt_react.E.next e in
printf "%d\n%!" x;
(* Doing [loop e] directly DOES NOT WORK because as per the synchrony
* hypothesis the whole loop is "instantaneous" and the next call to
* Lwt_react.E.next would happen within the same step. We have to do one
* of the 2 following: *)
(* loop (React.E.drop_once e) in *)
(* NOTE that drop_once cannot be used if a context switch might happen
between the 2 calls to Lwt_react.E.next! *)
(* COMMENT THE FOLLOWING OUT to observe the unexpected behavior *)
Lwt_unix.yield () >>
loop e in
let e = React.S.changes s in
let e' = React.E.map (fun x -> printf "DDDD %d\n%!" x) e in (* this one is fine *)
Lwt_react.E.keep e';
loop e
let print2 s =
let rec loop e =
lwt x = next e in
printf "X%d\n%!" x;
loop e
in
loop (React.S.changes s)
let () =
Lwt_unix.run begin
let s, p = React.S.create 0 in
ignore (print s);
ignore (print2 s);
for_lwt i = 0 to 20 do
p i;
Lwt_unix.sleep 0.1
done
end
I'm trying to install LWT (using OPAM) on a NetBSD system. NetBSD uses PKGSRC as it's 3rd-party package system, which is how I installed libev
. For whatever reason, the libev
package maintainers put the headers and libraries one subdirectory further than one would expect. I tried to reflect this in configuring LWT, but discover.ml
doesn't seem to pay attention (or doesn't handle them correctly:)
export LIBEV_CFLAGS=-I/usr/pkg/include/ev
export LIBEV_LIBS=-L/usr/pkg/lib/ev
opam install lwt
OPAM says the install failed and the error output indicates LWT's discover.ml
couldn't find the headers and library (but they are in the directories I specified.)
This segfaults for me on Windows, using the current master:
let () =
Lwt_io.open_connection (Unix.ADDR_INET (Unix.inet_addr_loopback, 80)) |> ignore;
Lwt_io.read_char Lwt_io.stdin |> Lwt_main.run |> ignore
Dr. Memory reports:
Dr. Memory version 1.7.0 build 5 built on Apr 4 2014 23:38:05
Dr. Memory results for pid 2204: "test.native"
Application cmdline: "./_build/test.native"
Recorded 104 suppression(s) from default C:\Program Files (x86)\Dr. Memory\bin\suppress-default.txt
Error #1: UNINITIALIZED READ: reading 0x014efc8d-0x014efc90 3 byte(s) within 0x014efc8c-0x014efc98
#0 system call NtDeviceIoControlFile AFD_ACCEPT_DATA
#1 MSWSOCK.dll!NSPStartup +0x106d (0x7319aa8b <MSWSOCK.dll+0xaa8b>)
#2 WS2_32.dll!WSAAccept +0x81 (0x76156958 <WS2_32.dll+0x6958>)
#3 WS2_32.dll!accept +0x16 (0x761568cd <WS2_32.dll+0x68cd>)
#4 lwt_unix_socketpair [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/crt/crtexe.c:212]
#5 lwt_unix_init_notification [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/crt/crtexe.c:212]
#6 camlLwt_unix__entry [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/crt/crtexe.c:212]
#7 .text [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/crt/crtexe.c:212]
#8 caml_start_program [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/libsrc/wspiapi/WspiapiClone.c:28]
#9 caml_main [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/libsrc/wspiapi/WspiapiClone.c:28]
#10 main [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/misc/invalid_parameter_handler.c:9]
Note: @0:00:02.375 in thread 2492
Error #2: UNINITIALIZED READ: reading 0x01bce800-0x01bce804 4 byte(s)
#0 socket_poll_add [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/libsrc/wspiapi/WspiapiClone.c:28]
#1 select_data_dispatch [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/libsrc/wspiapi/WspiapiClone.c:28]
#2 unix_select [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/libsrc/wspiapi/WspiapiClone.c:28]
#3 camlLwt_engine__fun_2185 [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/crt/crtexe.c:212]
#4 camlLwt_engine__fun_2089 [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/crt/crtexe.c:212]
#5 camlLwt_engine__fun_2089 [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/crt/crtexe.c:212]
#6 camlLwt_main__run_1012 [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/crt/crtexe.c:212]
#7 camlTest__entry [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/crt/crtexe.c:212]
#8 .text [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/crt/crtexe.c:212]
#9 caml_start_program [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/libsrc/wspiapi/WspiapiClone.c:28]
#10 caml_main [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/libsrc/wspiapi/WspiapiClone.c:28]
#11 main [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/misc/invalid_parameter_handler.c:9]
Note: @0:00:03.094 in thread 2492
Note: instruction: cmp 0x08(%ecx) %ebx
Error #3: UNADDRESSABLE ACCESS: reading 0x01bcdf18-0x01bcdf1c 4 byte(s)
#0 socket_poll_add [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/libsrc/wspiapi/WspiapiClone.c:28]
#1 select_data_dispatch [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/libsrc/wspiapi/WspiapiClone.c:28]
#2 unix_select [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/libsrc/wspiapi/WspiapiClone.c:28]
#3 camlLwt_engine__fun_2185 [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/crt/crtexe.c:212]
#4 camlLwt_engine__fun_2089 [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/crt/crtexe.c:212]
#5 camlLwt_engine__fun_2089 [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/crt/crtexe.c:212]
#6 camlLwt_main__run_1012 [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/crt/crtexe.c:212]
#7 camlTest__entry [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/crt/crtexe.c:212]
#8 .text [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/crt/crtexe.c:212]
#9 caml_start_program [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/libsrc/wspiapi/WspiapiClone.c:28]
#10 caml_main [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/libsrc/wspiapi/WspiapiClone.c:28]
#11 main [/usr/src/debug/mingw64-i686-runtime-3.1.0-1/misc/invalid_parameter_handler.c:9]
Note: @0:00:03.141 in thread 2492
Note: next higher malloc: 0x01bcdf20-0x01bced08
Note: prev lower malloc: 0x01bcd118-0x01bcdf00
Note: instruction: cmp 0x08(%ecx) %ebx
In particular, pthreads can be used as on any other *nix since API level 9... that's 2010 or so. sched_getcpu and sched_getsetaffinity are also present, though you need to invoke the syscall directly.
let mem_usage v =
let open Objsize in
let x = objsize v in
show_info ~map:string_of_int x
let () =
let (stream,_) = Lwt_stream.create () in
let rec loop () =
lwt _ = Lwt.pick [
(lwt () = Lwt_unix.sleep 0.1 in Lwt.return ());
(lwt _ = Lwt_stream.get stream in Lwt.return ());
] in
loop ()
in
let tasks = Array.to_list (Array.init 10_000 (fun _ -> loop ())) in
let nr = ref 0 in
let rec memory () =
lwt () = Lwt_unix.sleep 0.05 in
Gc.compact ();
let st = Gc.stat () in
Printf.printf "%d) heap live %d, tasks %s\n%!" !nr st.Gc.live_words (mem_usage tasks);
incr nr;
memory ()
in
Lwt_main.run (Lwt.join (memory () :: tasks))
mem_usage
requires objsize, you can remove it's usage and just observe heap growth.
Build with ocamlbuild -use-ocamlfind -tags 'syntax_camlp4o, package(lwt.syntax)' -pkgs lwt.unix,objsize lwt_leak.native
See discussion in https://sympa.inria.fr/sympa/arc/ocsigen/2014-04/msg00005.html
In discover.ml
, if the configuration script can't find the libev library, it displays a message informing the user to define environment variables. Unfortunately, LIBEV_CFLAGS is spelled "LIBEV_CLFAGS" in the help text, so the user is misinformed as to what variables should work.
Steps to reproduce:
LWT_LOG=debug ocaml
#require "lwt.log";;
Lwt_log.debug "foo";;
This happens because Lwt_log_core is initialized before Lwt_log, and the level is alread set for main at the moment when the environment variable is being examined. Lwt_log_core.load_rules should call Section.recompute_levels ()
.
Also, this tooks me about two hours to debug. :/
Hi,
I have an example of code that runs perfectly fine on Linux and is broken on Windows. A simplified version of the code is the following one.
let worker invocation =
let open Lwt in
let worker = Lwt_process.open_process_none invocation in
Lwt_unix.waitpid [(* Unix.WUNTRACED *)] worker#pid >>=
begin
fun (pid,status) ->
return ()
end
let main cores jobs =
let pool = Lwt_pool.create cores Lwt.return in
let f job () = worker job in
let e = Lwt_list.iter_p (fun job -> Lwt_pool.use pool (f job)) jobs in
Lwt_main.run e
On Linux, jobs are properly created and there are never more than cores
jobs running at once. On Windows, all the jobs are launched at once and then, I get a Bad file descriptor (waitpid) error. What could be going wrong?
I installed both pthread and libev via system packages (Fedora 20), so they ought to be in reasonable locations. But I still get "The following recquired C libraries are missing: pthread, libev." when I run ocaml setup.ml -configure --enable-unix
Digging into discover.ml, I modified it to print the location of log_file and not to delete it. Aside: it seems rather awkward to debug this sort of build failure, as everything gets cleaned up immediately even in the case of failure. Unless I've missed a -debug argument or something.
Anyway, the log says:
/tmp/lwt_stub35ffd5.c: In function ‘lwt_test’:
/tmp/lwt_stub35ffd5.c:7:3: warning: null argument where non-null required (argument 1) [-Wnonnull]
pthread_create(0, 0, 0, 0);
^
/tmp/lwt_stub35ffd5.c:7:3: warning: null argument where non-null required (argument 3) [-Wnonnull]
/usr/bin/ld: cannot find -lcurses
collect2: error: ld returned 1 exit status
File "/tmp/lwt_caml396824.ml", line 1:
Error: Error while building custom runtime system
So I've installed ncurses-devel, and that fixed it. To be honest I'm not sure what is requiring -lcurses since the word "curses" doesn't appear in the lwt repo, but I thought you'd like to know.
The program below never terminates, despite the exit 0
. What happens is that exit 0
attempts to flush the channel while Lwt_io.flush oc
is still in progress.
let (>>=) = Lwt.bind
let flags = [Unix.O_WRONLY; Unix.O_CREAT; Unix.O_APPEND; Unix.O_NONBLOCK]
let _ =
Lwt_main.run
(Lwt_unix.openfile "/tmp/out.txt" flags 0o666 >>= fun fd ->
let oc = Lwt_io.of_fd ~mode:Lwt_io.output fd in
ignore (Lwt_io.write oc "BUG" >>= fun () -> Lwt_io.flush oc);
exit 0)
I have a NetBSD/amd64 6.1.4 system. From OPAM 1.1.1, I've installed LWT 2.4.5 and cohttp 0.11.2 (and all their assorted dependencies.) When I run the following test program, I get a segmentation fault and a core file.
open Cohttp_lwt_unix.Client
let url = Uri.of_string "http://www.netbsd.org"
let main () =
lwt (resp, _) = get url in
Lwt_log.notice_f "Response -> status:%s"
(Cohttp.Code.string_of_status resp.Response.status)
let () =
Lwt_main.run @@ (main () >> Lwt_unix.sleep 5.0)
The program was built with:
$ ocamlfind ocamlopt -g -syntax camlp4o -package unix,lwt.syntax,uri,cohttp.lwt -c test.ml
$ ocamlfind ocamlopt -g -syntax camlp4o -package unix,lwt.syntax,uri,cohttp.lwt -linkpkg -o a.out test.cmx
There are no compiler or linker warnings, so I assume that all modules are found and linked together (the -linkpkg
option helps guarantee this.) I've recently completed a project that used LWT and had no problems, so that library appears to be installed and working properly. This new project was going to use cohttp, so I built this test module to familiarize myself with the API. Unfortunately, my first attempt caused a segmentation fault.
I don't think I've done anything wrong in my test module (and if I misused the API, it shouldn't SEGFAULT.) So I thought I'd report it.
The GDB stack trace shows:
(gdb) bt
#0 0x00007f7ff669f98a in ?? () from /usr/lib/libc.so.12
#1 0x00007f7ff66a1dfe in free () from /usr/lib/libc.so.12
#2 0x000000000053c939 in result_getprotobyname ()
#3 0x000000000053f058 in lwt_unix_self_result ()
#4 0x000000000047aec4 in camlLwt_unix__self_result_1168 ()
at src/unix/lwt_unix.ml:240
#5 0x0000000000479012 in camlLwt_unix__fun_2363 () at src/unix/lwt_unix.ml:206
#6 0x00000000005006cc in camlArray__iter_1054 () at array.ml:72
#7 0x00000000004aa6bd in camlLwt_sequence__loop_1066 ()
at src/core/lwt_sequence.ml:149
#8 0x00000000005028e1 in camlList__iter_1061 () at list.ml:73
#9 0x0000000000476442 in camlLwt_engine__fun_2089 ()
at src/unix/lwt_engine.ml:342
#10 0x0000000000478098 in camlLwt_main__run_1012 () at src/unix/lwt_main.ml:41
#11 0x000000000046b0a9 in camlTest__entry () at test.ml:11
#12 0x0000000000467649 in caml_startup__code_begin ()
#13 0x0000000000558a0e in caml_start_program ()
#14 0x00007f7ff6208eef in __register_frame_info_bases ()
from /usr/lib/libgcc_s.so.1
#15 0x000000000083c360 in ?? ()
#16 0x00007f7fffffd400 in ?? ()
#17 0x00000000005478e4 in main ()
(gdb)
From the stack trace, it doesn't appear to be crashing in the 'cohttp' module. It seems like getprotobyname
is failing in LWT (cohttp probably calls getprotobyname
when parsing the URL -- although the stack trace doesn't show any functions in cohttp.) My previous LWT project didn't use that function, so I tried calling it from the shell and had no problem.
I can't rebuild the app as byte-code because LWT is installed 'native' when using OPAM. Because of this, I can't run it in the OCaml debugger.
Does anyone have any suggestions as to how I can find what's causing this problem?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.