I think it would be worthwhile to make the lisp kernel buildable with Microsoft's toolchain (either or both of the command-line tools or with Visual Studio).

My understanding is that this would let us to Microsoft's debugging tools (such as windbg), which may be better (in some sense) than using gdb.

One challenge is going to be dealing with the various assembly language files. There's an assembler called yasm (http://yasm.tortall.net) which should be able to support the AT&T style syntax that we use.

https://ccl.clozure.com
Clozure Common Lisp Download Wiki Report Bug http://trac.clozure.com/ccl/newticket
...
Reporting bugs
To report a bug or request a feature, please create a ticket http://trac.clozure.com/ccl/newticket

I have been working on a project called cl-wamp for a while and just recently discovered that these weird connection issues I was having were due to ccl (they stopped when I switched to sbcl).

I'm not sure what's causing it, but I know that commenting out lines 127 and 128 seems to fix it sometimes.

I think it's a really weird heisenbug, but I could also just be an idiot.

I'm using 1.11-r16635 (LinuxX8664).

To see this error:

? (setq *print-pretty* t)
? (format t "~:@_FAILURE DETAILS:~:@_~4:I================")
FAILURE DETAILS:================ NIL

As I read the CLHS, section 22.3, the ~:@_ directive is equivalent to a mandatory newline (i.e. ~%), but that’s not the case here.

http://www.lispworks.com/documentation/HyperSpec/Body/22_cea.htm

(copied from https://trac.clozure.com/ccl/ticket/632)

More than a few users don't like the way CCL escapes certain characters in namestrings. Perhaps it is time to remove this behavior.

Gary Byers wrote:

Section 19.1.1 of the spec says that pretty much anything that an implementation does with respect to namestrings that aren't logical pathname namestrings is conformant, and that portable programs can't assume much of anything about the syntax of such namestrings (or about how they're translated to PATHNAME objects and back again.)

When translating a namestring into a pathname, most implementations probably follow a convention where a dot character in a namestring separates the substring used to provide the pathname's PATHNAME-NAME from the substring used to provide its PATHNAME-TYPE. When there's more than one dot involved or when dots appear at the beginning or end of the namestring, things are less clear: does ".emacs" describe a pathname whose name is NIL and type is "emacs", a pathname whose name is ".emacs" and whose type is NIL, or something else? Which parts of "a.b.c" are parsed as a pathname name and which parts are parsed as a pathname type?

When generating a NAMESTRING from a pathname (as happens when printing a pathname), CCL tries to avoid some potential ambiguity by quoting/escaping characters (dots, usually) that're constituents of some component and which might otherwised be used to separate components; this quoting/escaping involves prefixing the character with a backslash on Unix platforms, so:

"a\\.b.c" has name "a.b", type "c"
"a.b\\.c" has name "a", type "b.c"

Unfortunately:

a) the rules that CCL uses for introducing/removing escaping can still lead to ambiguity and prevent round-trip translations
b) people don't expect to see it and report the escaping as a bug.

(b) isn't valid, but it still takes time to write messages like this to explain that this is just "implementation-dependent behavior". It'd be easier to justify that particular behavior if it led to less ambiguity and more reliable round-tripping, but it doesn't.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Several users have expressed interest in package-local nicknames.

There's a mailing list thread about it at https://lists.clozure.com/pipermail/openmcl-devel/2013-February/thread.html

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

In 64-bit CCL foreign types :int32 and :uint32 are somehow defined as 64-bit. At least FreebsdX8664 is affected.

(ccl::foreign-size (:unsigned 32)) => 32
(ccl::foreign-size (:signed 32)) => 32

Should be the same, but isn't:
(ccl::foreign-size :uint32) => 64
(ccl::foreign-size :int32) => 64

Workaround is trivial - redifinition to correct values.

SBCL has a --script command line option used to load and evaluate a file, ignoring the shebang if there is one, skipping the loading of the init file, disabling the debugger, and limiting verbosity. This make it really simple to write scripts in Common Lisp.

It would be nice if CCL were to have the same option with the same semantic.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

It might be worthwhile to make an Xcode project for the lisp kernel. This would provide easier access to the static analyzer.

This would be in addition to (i.e., it would not replace) the current makefile-based build.

This may require some minor restructuring. I'm thinking in particular of the platform-specific include file (e.g., platform-darwinx8664.h) that's included via the -include option to the C compiler driver program.

Issue #59 (build lisp kernel with MS tools) might have a similar requirement.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

A strong visual identity for the Clozure CL project would be very valuable.

This could include a logo, a project style implemented in CSS, and so forth.

I can hardly expect someone to contribute something like this for free, but I'm making the issue as a reminder anyway.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Port CCL to the 64-bit ARM.

Historically, a port to a new architecture has taken about 3 or 4 wizard-months.

The 64-bit ARM can be configured in such a way that the high 8 bits of address will be ignored by the address-translation hardware. This requires OS support, but the last time we looked, Linux kernel version 3.12.0 seemed to have adequate support (i.e., the tags were preserved in signal context, in addition to setting the MMU bit to enable ignoring the high 8 bits of addresses).

This makes it worthwhile to consider using high tags. I have collected some notes about this and other issues that I could probably transcribe somewhere when/if we ever get the resources to do this port.

As of Mac OS X 10.6 Snow Leopard (released in 2009), it should be possible for a Cocoa application to build its menu bar and other UI entirely in code. Interface Builder in Xcode 4.0 also removed much of the support for setting up connections in Cocoa nib files for bridged and unsupported languages.

Therefore it would probably be in the CCL IDE’s best interest to switch entirely to code for setting up its interface, rather than continue to use nib files that can’t easily be edited.

Jeff Johnson recently published an example of doing this on modern macOS in Swift: https://github.com/lapcat/NiblessMenu

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Step is a no-op at the moment.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

I hope this finds you all well in this time of transition.

Our compute cluster software needs a heuristic to determine when a process is using more RAM than it's been allocated. Since resident size is unreliable as a measure of memory a process is actually using once processes start swapping, the cluster uses virtual memory size as that heuristic. Unfortunately, this means that the compute cluster thinks CCL is using way more memory than it's actually using. We've mitigated this confusion with two mechanics: (1) using the -R flag to reduce heap size and (2) using a script Gary wrote for us to unmalloc what I think is a lot of the static cons space.

Could CCL supply a flag or compile-time option to its users that allows us to allocate significantly less static cons space? Another option, which I think would be more difficult to implement but still might be reasonable, would be to dynamically grow the static cons space as more space is needed (but never allowing the allocated static cons space to shrink). I've included the script GB wrote below the signature.

Thanks,
David

(in-package "CCL")

(defun find-area (code)
  (let* ((p (%null-ptr)))
    (%setf-macptr-to-object p (%get-kernel-global all-areas))
    (if (eql (%get-object p target::area.code) code)
      p
      (do* ((q (%get-ptr p target::area.succ) (%get-ptr q target::area.succ)))
           ((eql p q))
        (when (eql (%get-object q target::area.code) code)
          (return q))))))

(defun dynamic-heap-start-address ()
  (%int-to-ptr (ash (%get-kernel-global heap-start) target::fixnumshift)))

(defun purified-end-address ()
  (let* ((a (find-area area-managed-static)))
    (if a
      (%get-ptr a target::area.high)
      (error "can't find the end of purified space"))))

(defun room-for-n-static-conses (n)
  (* (logandc2 (+ 32767 n) 32767) target::cons.size))

(defun target-dynamic-heap-start-address (nstatic-conses)
  (%inc-ptr
   (dynamic-heap-start-address)
   (- (room-for-n-static-conses (- nstatic-conses (reserved-static-conses))))))

(defun unmap-reserved-static-space (nstatic-conses)
  (let* ((start (purified-end-address))
         (end (target-dynamic-heap-start-address nstatic-conses))
         (nbytes (- (%ptr-to-int end) (%ptr-to-int start))))
    (#_munmap start nbytes)))

; for pairing with a heap-reserve of 142GB
; (ccl::unmap-reserved-static-space (expt 2 17))

; for pairing with a heap-reserve of 158GB, -Z 1024000000 --heap-reserve 158G
 (ccl::unmap-reserved-static-space (expt 2 16))

; for pairing with a heap-reserve of 190GB
; (ccl::unmap-reserved-static-space (expt 2 15)) 

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

From https://trac.clozure.com/ccl/ticket/1260

SLSIA:

Welcome to Clozure Common Lisp Version 1.10-r16304M  (DarwinX8664)!
? (find-symbol "ARGS")
NIL
NIL
? (arglist 'setf)
(&REST ARGS)
:DECLARATION
? (find-symbol "ARGS")
ARGS
:INTERNAL

The problem occurs only when calling CCL::ARGLIST on macros, not functions.

I know it's not kosher to complain about the behavior of unexported functions, but swank uses this and it causes real problems. For example, trying to (use-package :cl-who) in a swank session fails because the symbol STR gets interned in cl-user during the swank startup process.

First of all I want to note that ccl:arglist is indeed external to the ccl package, and seems to be the only way to get lambda lists of functions/macros/..., which is a common operation in introspection tools (SLIME, documentation generators, ...). I would argue this to be a popular feature, and hence a problematic bug.

I’ve investigated a bit, and the interning happens because for macros the lambda list is saved as a string (returned by %arglist-internal) which is then parsed by ccl::%arglist using read in the current package.

In a way this is not a bug since its not defined which package should intern the symbols in question.

It would be a valuable feature for socket streams to be asynchronous .

I think this feature may reference Java NIO, libevent, libev, libuv.
It should support IOCP on windows, epoll on linux, kqueue on freebsd.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

The documentation for advise ([http://ccl.clozure.com/docs/ccl.html#advising]) is buggy. Instead of arglist both examples should say (car arglist).

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Allegro and Lispworks provide a function logical-pathname-translations-database-pathnames allowing users to, for example:

(pushnew "~/myhosts" (excl:logical-pathname-translations-database-pathnames))

so that users can put their translations outside of the sometimes protected or otherwise unavailable CCL: directory. Users might consider adding a file such as "~/myhosts" prior to a call to dump so that they can easily provide translations to the restarted image during startup (before the first prompt).

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

I am running ccl on an Ubuntu 64 VM and I am having trouble getting lx86cl to run. This is a clean install of ubuntu with nothing installed on it but what comes with it by default, build-essential and subversion. I do not think the fact that I'm working on a virtual machine is causing the issue.

I followed the instructions to install ccl from https://ccl.clozure.com/install.html and the associated webpages.
I used the command svn co http://svn.clozure.com/publicsvn/openmcl/release/1.11/linuxx86/ccl
and this created the directory in /home/username/ccl.

I attempted to run ccl directly from the terminal using the command: ./lx86cl64 and this started the 64 bit ccl just fine.
However using the command ./lx86cl returned bash: ./lx86cl: No such file or directory (same goes for trying to set up the ccl script in /usr/local/bin . ccl64 works fine, but not ccl).
Which is strange because when I move to the ~/ccl directory the lx86cl binary is included in the folder.

What I haven't done:

• Attempted to try this on a 32 bit version of unbuntu
• Compiled from the source

In the past I just shrugged and moved on using ccl64 only, but now I am at a point where I need to test to make sure my project works in the 32bit version. So, I'm trying to make sure each works before I set up emacs, slime and quicklisp on this clean VM.

Any thoughts on getting the 32 bit version of CCL to work for this setup? Is this a true bug? Am I alone out here with this issue? I will reply with my experiments in other VMs later.

I'm not very experienced with linux so please let me know if I'm missing something obvious!

;; $ ccl64 --eval '(compile-file "bug.lisp")'
;;
;;; Compiler warnings for "/tmp/bug.lisp" :
;;;   In an anonymous lambda form at position 109: In the call to foo with arguments (1 2),
;;;     2 arguments were provided, but at most 0 are accepted
;;;     by the current global definition of foo
;; Welcome to Clozure Common Lisp Version 1.11-r16812  (Darwinx8664)!

(eval-when (:compile-toplevel :load-toplevel :execute)
  (setf (fdefinition 'foo) (constantly nil)))

(print (foo 1 2))

The warning is incorrect because the closure returned by CONSTANTLY is required to accept "any number of arguments."

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

There is a test that is basically this:

(defun junk ()
 (let ((s (make-array '(10) :element-type 'character
		       :initial-contents "0123456789" :fill-pointer 3)))
   (values (equal s "012") (equal "012" s))))

On current master, this function incorrectly returns (values nil nil) when compiled using the x8632 compiler.

I compared the disassembly, and it appears to me that something goes wrong at the places I marked with ***.

In the first snippet (from a ccl-1.11 dx86cl), we load the subtag of the array's data vector into arg_z, and then branch and push it. In the second segment (from a trunk version), we do the same kind of work, but extract the subtag into temp1. We then branch and push arg_z anyway, which doesn't contain what we just computed.

;;; (= x-type target::subtag-vectorH)
L203
   (movl (@ -12 (% ebp)) (% arg_z))        ;   [205]
   (cmpl ($ #x3C8) (% arg_z))              ;   [208] are we subtag-vectorh?
   (jne L238)                              ;   [214]

;;; (%svref x target::arrayH.flags-cell)
   (movl (@ -4 (% ebp)) (% temp1))         ;   [216] we are, so get flags word
   (movl (@ 14 (% temp1)) (% temp1))       ;   [219] from array header

;;; (ldb target::arrayH.flags-cell-subtag-byte (the fixnum (%svref x target::arrayH.flags-cell)))
   (movl (% temp1) (% imm0))               ;   [222] extract subtag of data vector
   (sarl ($ 8) (% imm0))                   ;   [224] from flags word (byte 8 8)
   (andb ($ #xFC) (% imm0.b))              ;   [227]
   (movl (% imm0) (% arg_z))               ;   [230] ***
   (andl ($ #x3FC) (% arg_z))              ;   [232] ***
   (jmp L241)                              ;   [238]

;;; (if (= x-type target::subtag-vectorH) (ldb target::arrayH.flags-cell-subtag-byte (the fixnum (%svref
L238
   (movl (@ -12 (% ebp)) (% arg_z))        ;   [240]

;;; (let* ((x-simple (if (= x-type target::subtag-vectorH) (ldb target::arrayH.flags-cell-subtag-byte (t
L241
   (pushl (% arg_z))                       ;   [243] *** push subtag

And from current master:

;;; (= x-type target::subtag-vectorH)
L209
   (movl (@ -12 (% ebp)) (% arg_z))        ;   [211]
   (cmpl ($ #x3C8) (% arg_z))              ;   [214] are we subtag-vectorh?
   (jne L244)                              ;   [220]

;;; (%svref x target::arrayH.flags-cell)
   (movl (@ -4 (% ebp)) (% temp1))         ;   [222] we are, so grab flags word
   (movl (@ 14 (% temp1)) (% temp1))       ;   [225] from array header

;;; (ldb target::arrayH.flags-cell-subtag-byte (the fixnum (%svref x target::arrayH.flags-cell)))
   (movl (% temp1) (% imm0))               ;   [228]
   (sarl ($ 8) (% imm0))                   ;   [230]
   (andb ($ #xFC) (% imm0.b))              ;   [233]
   (movl (% imm0) (% temp1))               ;   [236] ***
   (andl ($ #x3FC) (% temp1))              ;   [238] ***
   (jmp L247)                              ;   [244]

;;; (if (= x-type target::subtag-vectorH) (ldb target::arrayH.flags-cell-subtag-byte (the fixnum (%svref
L244
   (movl (@ -12 (% ebp)) (% arg_z))        ;   [246]

;;; (let* ((x-simple (if (= x-type target::subtag-vectorH) (ldb target::arrayH.flags-cell-subtag-byte (t
L247
   (pushl (% arg_z))                       ;   [249] ***

A simpler test case:

(defparameter *a* (make-array 10 :element-type 'character
			      :initial-contents "0123456789" :fill-pointer 3))

(defun foo (x)
  (let* ((x-type (typecode x))
	 (x-simple (if (= x-type target::subtag-vectorH)
		     (ldb target::arrayH.flags-cell-subtag-byte
			  (the fixnum (%svref x target::arrayH.flags-cell)))
		     x-type)))
    (values x-type x-simple)))

? (foo *a*)
242
242
;; should be 242 (subtag-vectorH), 191 (subtag-simple-base-string)

The following example:

;; 
(defun randu (n) (loop :repeat n :collect (random 1.0)))

;; 
(defun mean (x) (/ (loop for i in x sum i) (length x)))

;;
(defun moment (x p) (/ (loop for i in x sum (expt i p)) (length x)))

;; 
(defun var (x) (- (moment x 2) (expt (mean x) 2)))

On a netbook with Intel(R) Atom(TM) CPU N455 @ 1.66GHz, CCL took 4.5 seconds to run (var (randu 1e6)) while SBCL only took 0.8 seconds (in both case the source code file is compiled then the .fasl or .lx86fsl files are loaded).

I am wondering if this is due to I didn't use CCL properly, any comments are appreciated.

It’s inconvenient to have to merge a binary distribution into a cloned repository in order to build it. It should be possible to build CCL without having binaries in the same directory, as long as ccl or ccl64 is in the PATH.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

When using make-vector-output-stream, the default external-format is :DEFAULT. For with-output-to-vector, it's NIL.

In recent FreeBSD versions, the MAP_NORESERVE option for mmap(2) seems to have been removed.

According to https://reviews.freebsd.org/D848, it had never been implemented.

This causes lisp kernel compilation to fail with errors such as these:

cc -include ../platform-freebsdx8664.h -c ../memory.c -DFREEBSD -D_REENTRANT -DX86 -DX8664 -D_GNU_SOURCE -DHAVE_TLS -DSVN_REVISION=16810 -g  -Wno-format -m64 -o memory.o
../memory.c:91:34: error: use of undeclared identifier 'MAP_NORESERVE'
               MAP_PRIVATE | MAP_ANON | MAP_NORESERVE,
                                        ^
../memory.c:101:79: error: use of undeclared identifier 'MAP_NORESERVE'
    if(mmap(start, totalsize, PROT_NONE, MAP_PRIVATE | MAP_ANON | MAP_FIXED | MAP_NORESERVE...
                                                                              ^
../memory.c:909:53: error: use of undeclared identifier 'MAP_NORESERVE'
  p = mmap(NULL,size,PROT_NONE,MAP_PRIVATE|MAP_ANON|MAP_NORESERVE,-1,0);
                                                    ^

Copied from Trac Issue#1416 per request from rme. In addition, see earlier Trac Issue#1297. Links are at the bottom of this Issue.

Environment:
CCL version 1.11-r16635 64bit under Windows 10 64bit

Possible issue:
The Format function's F directive's k parameter fails to move the decimal point of the floating point argument to the right when preceded by a tilde and 2 consecutive commas. This fails in Clozure, but works in SBCL v1.3.15 64bit and in Clisp 2.49 when tested.

Example1: The Clozure prompt is "?". Note the issue is that 0.1 is returned rather than 10.0

? (format t "10 Percent ~,,2f ~%" .10)
10 Percent 0.1
NIL

Example2: The SBCL prompt is an asterisk "*". Clisp produces the same result.

* (format t "10 Percent ~,,2f ~%" .10)
10 Percent 10.0
NIL

Other scenarios seem to work fine...

Example3: Clozure produces the same result as SBCL Example4 if a 1 is entered for the d parameter (decimal digits) between the two commas.

? (format t "10 Percent ~,1,2f ~%" .10)
10 Percent 10.0
NIL

Example4: SBCL produces the same result as Clozure Example3 when entering a 1 between the two commas.

* (format t "10 Percent ~,1,2f ~%" .10)
10 Percent 10.0
NIL

Example5: Clozure "?" & SBCL "*" produce the same results when entering a 0 between the two commas.

? (format t "10 Percent ~,0,2f ~%" .10)
10 Percent 10.
NIL

* (format t "10 Percent ~,0,2f ~%" .10)
10 Percent 10.
NIL

Reference:

CLHS: 22.3.3.1 Tilde F: Fixed-Format Floating-Point.

"The full form is ~w,d,k,overflowchar,padcharF. The parameter w is the width of the field to be printed; d is the number of digits to print after the decimal point; k is a scale factor that defaults to zero."

http://www.lispworks.com/documentation/HyperSpec/Body/22_cca.htm

Original Trac Issue Tickets:
https://trac.clozure.com/ccl/ticket/1297
https://trac.clozure.com/ccl/ticket/1416

The function determine_executable_name() in the lisp kernel is supposed to return the full path to the running lisp kernel executable.

This function uses various system-specific ways to do this, depending on what the platform the lisp kernel is being compiled for. Unfortunately, the bit of code for FreeBSD doesn't do the job. It essentially just calls realpath(3) on the value of argv[0].

To see why this is inadequate:

$ ./fx86cl64
? (save-application "magic" :prepend-kernel t)
$ cp magic ~/bin    # or some other place in your path
$ cd /
$ magic
Couldn't load lisp heap image from magic.image: No such file or directory

This problem was discovered in dimitri/pgloader#551

It would be a valuable feature for socket streams to support SSL.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

When building kernel, wx86cl64 complains missing libwinpthread-1.dll. 32bit wx86cl is fine. There is no explicit reference to this in windows Makefile. Do a search found that this is required by libgcc, which is required by mingw-w64 x86_64 build. We can make it statically linked. I'll send a pull request for this soon.

(copied from http://trac.clozure.com/ccl/ticket/1380)

1b8a369 causes trouble on my Raspberry Pi 3.

It reliably fails when doing (rebuild-ccl :clean t) with the following strange error.

;Compiling "/home/rme/ccl/compiler/nxenv.lisp"...
> Error: The value -1069547520 is not of the expected type INTEGER.
> While executing: ARM::ENCODE-ARM-IMMEDIATE, in process listener(1).
> Type :POP to abort, :R for a list of available restarts.
> Type :? for other options.
1 > b
*(76B87E28) : 0 (ENCODE-ARM-IMMEDIATE ???) 40
*(76B87E28) : 1 (FUNCALL #'#<ARM2-FIXNUM> #<VINSN-LIST #x1457779E> #<LREG 8 GPR {5}/LISP> 2 -267386880) 580
 (76B87E28) : 2 (ARM2-FORM #<VINSN-LIST #x1457779E> #<LREG 8 GPR {5}/LISP> 2 #<ACODE fixnum (-267386880)>) 788
 (76B87E38) : 3 (FUNCALL #'#<ARM2-IF> #<VINSN-LIST #x1457779E> #<LREG 8 GPR {5}/LISP> 2 #<ACODE call (#<# # #> # NIL)> #<ACODE fixnum (-267386880)> #<ACODE fixnum (-268435456)>) 2100

Linux kupo 4.4.50-v7+ #970 SMP Mon Feb 20 19:18:29 GMT 2017 armv7l GNU/Linux

The CCL IDE doesn’t recognize the lsp file extension, making it difficult to load and run the ANSI compliance suite via the “File ▸ Load File…” menu command.

The top-level .gitignore file is a terrible mess. It should be cleaned up.

Currently, there is one common x86 backend.

The 32-bit and 64-bit x86 really differ quite a lot more than PPC32 and PPC64. On the other hand, they're not completely different, the way 32-bit and 64-bit ARM are. So, while there's still x86 stuff that can be shared (the assembler, at least), I believe it would be better for them to go their separate ways, and not incidentally, keep out of each other's way.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Because the Solaris implementation of ccl:run-external-process uses the forkall system call instead of fork, there are problems (due to multiple processes' running threads having references to file handles) when using run-program. This can be seen by doing this on Solaris:-

(ql:quickload :aserve)
(net.aserve:start :port 8080 :server :new)
(ccl:run-program "/bin/ls" '("/"))

(There is actually an error in the error reporting of this, but there is a legitimate error).

Forkall is used in Solaris due to this change: http://trac.clozure.com/ccl/changeset/11176 - since Solaris doesn't clone the stacks of threads other than the one calling fork. This works fine if there is only one thread in the process which forks.

This change (using forkall) was needed because of a previous change in CCL whereby argv is set up in a separate thread from the one which calls fork. Setting up argv in the thread which calls fork fixes the issue.

(defun run-external-process (proc in-fd out-fd error-fd argv &optional env)
    (let* ((signaled nil))
      (unwind-protect
           ;; !!! Don't use forkall in Solaris - it causes problems when aserve is running
           (let* ((child-pid (#_fork))) ; (#-solaris-target #_fork #+solaris-target #_forkall)))
             (declare (fixnum child-pid))
             (cond ((zerop child-pid)
                    ;; Running in the child; do an exec
                    (setq signaled t)
                    (dolist (pair env)
                      (setenv (string (car pair)) (cdr pair)))
                    (without-interrupts
                      (exec-with-io-redirection
                       in-fd out-fd error-fd argv)))
                   ((> child-pid 0)
                    ;; Running in the parent: success
                    (setf (external-process-pid proc) child-pid)
                    (add-external-process proc)
                    (signal-semaphore (external-process-signal proc))
                    (setq signaled t)
                    (monitor-external-process proc))
                   (t
                    ;; Fork failed
                    (setf (external-process-%status proc) :error
                          (external-process-%exit-code proc) (%get-errno))
                    (signal-semaphore (external-process-signal proc))
                    (setq signaled t))))
        (unless signaled
          (setf (external-process-%status proc) :error
                (external-process-%exit-code proc) -1)
          (signal-semaphore (external-process-signal proc))))))


  (defun run-program (program args &key
                                     (wait t) pty
                                     input if-input-does-not-exist
                                     output (if-output-exists :error)
                                     (error :output) (if-error-exists :error)
                                     status-hook (element-type 'character)
                                     env
                                     (sharing :private)
                                     (external-format `(:character-encoding ,*terminal-character-encoding-name*))
                                     (silently-ignore-catastrophic-failures
                                      *silently-ignore-catastrophic-failure-in-run-program*))
    "Invoke an external program as an OS subprocess of lisp."
    (declare (ignore pty))
    (unless (every #'(lambda (a) (typep a 'simple-string)) args)
      (error "Program args must all be simple strings : ~s" args))
    (setq external-format (normalize-external-format t external-format))
    (dolist (pair env)
      (destructuring-bind (var . val) pair
        (check-type var (or string symbol character))
        (check-type val string)))
    (push (native-untranslated-namestring program) args)
    (let* ((token (list 0))
           (in-fd nil)
           (in-stream nil)
           (out-fd nil)
           (out-stream nil)
           (error-fd nil)
           (error-stream nil)
           (close-in-parent nil)
           (close-on-error nil)
           (proc
            (make-external-process
             :pid nil
             :args args
             :%status :running
             :input nil
             :output nil
             :error nil
             :token token
             :status-hook status-hook
             :external-format (setq external-format (normalize-external-format t external-format)))))
      (unwind-protect
           (progn
             (multiple-value-setq (in-fd in-stream close-in-parent close-on-error)
               (get-descriptor-for input proc  nil nil :direction :input
                                   :if-does-not-exist if-input-does-not-exist
                                   :element-type element-type
                                   :sharing sharing
                                   :external-format external-format))
             (multiple-value-setq (out-fd out-stream close-in-parent close-on-error)
               (get-descriptor-for output proc close-in-parent close-on-error
                                   :direction :output
                                   :if-exists if-output-exists
                                   :element-type element-type
                                   :sharing sharing
                                   :external-format external-format))
             (multiple-value-setq (error-fd error-stream close-in-parent close-on-error)
               (if (eq error :output)
                   (values out-fd out-stream close-in-parent close-on-error)
                   (get-descriptor-for error proc close-in-parent close-on-error
                                       :direction :output
                                       :if-exists if-error-exists
                                       :sharing sharing
                                       :element-type element-type
                                       :external-format external-format)))
             (setf (external-process-input proc) in-stream
                   (external-process-output proc) out-stream
                   (external-process-error proc) error-stream)
          
             (process-run-function
              (list :name
                    (format nil "Monitor thread for external process ~a" args)
                    :stack-size (ash 128 10)
                    :vstack-size (ash 128 10)
                    :tstack-size (ash 128 10))
              (lambda (proc in-fd out-fd error-fd env)
                (call-with-string-vector
                 #'(lambda (argv)
                     (run-external-process proc in-fd out-fd error-fd argv env))
                 args
                 (external-format-character-encoding external-format)))
              proc in-fd out-fd error-fd env)

             (wait-on-semaphore (external-process-signal proc)))

        (dolist (fd close-in-parent) (fd-close fd))
        (unless (external-process-pid proc)
          (dolist (fd close-on-error) (fd-close fd)))
        (when (and wait (external-process-pid proc))
          (with-interrupts-enabled
            (wait-on-semaphore (external-process-completed proc)))))
      (unless (external-process-pid proc)
        ;; something is wrong
        (if (eq (external-process-%status proc) :error)
            ;; Fork failed
            (unless silently-ignore-catastrophic-failures
              (cerror "Pretend the program ran and failed" 'external-process-creation-failure :proc proc))
            ;; Currently can't happen.
            (error "Bug: fork failed but status field not set?")))
      proc))

Under some rare and unknown circumstances, calling (static-cons ...) from ACL2 running on top of CCL resulted in the following error:

Error: Current process #<TTY-LISTENER listener(1) [Active] #x302000452AED> does not own lock #<RECURSIVE-LOCK "Kernel exception-lock" [ptr @ #x16DFC40] #x30200001146D>

Here is the value of (lisp-implementation-version):

? (lisp-implementation-version)
"Version 1.12-dev-r16752  (LinuxX8664)"

Here is an abbreviated backtrace, which I believe contains all the frames that are running in CCL itself, rather than user (i.e. ACL2) code. The full backtrace (with a few frames here and there removed because of proprietary information in function arguments, sorry) can be seen here (click "raw" on that page to get the raw text file). This backtrace is produced via ACL2 so I think the formatting is a bit different from what you'd usually get from CCL.

 (7FCED7FF09A0) : 0 (PRINT-CALL-HISTORY :CONTEXT NIL :PROCESS NIL :ORIGIN NIL :DETAILED-P NIL :COUNT 10000 :START-FRAME-NUMBER 0 :STREAM #<SYNONYM-STREAM to *TERMINAL-IO* #x3020033EC66D> :PRINT-LEVEL 2 :PRINT-LENGTH 5 :SHOW-INTERNAL-FRAMES NIL :FORMAT :TRADITIONAL) 869
 (7FCED7FF0B08) : 1 (CALL-CHECK-REGS CCL:PRINT-CALL-HISTORY :DETAILED-P NIL :COUNT 10000) 229
 (7FCED7FF0B40) : 2 (CHEAP-EVAL (CCL:PRINT-CALL-HISTORY :DETAILED-P NIL :COUNT *CCL-PRINT-CALL-HISTORY-COUNT*)) 101
 (7FCED7FF0B78) : 3 (PRINT-CALL-HISTORY) 229
 (7FCED7FF0B88) : 4 (OUR-ABORT #<CCL::NOT-LOCK-OWNER #x3021D590335D> OUR-ABORT) 189
 (7FCED7FF0BB8) : 5 (BREAK-LOOP-HANDLE-ERROR #<CCL::NOT-LOCK-OWNER #x3021D590335D> 17565795475856) 701
 (7FCED7FF0C58) : 6 (%ERROR #<CCL::NOT-LOCK-OWNER #x3021D590335D> (:LOCK #<RECURSIVE-LOCK "Kernel exception-lock" [ptr @ #x16DFC40] #x30200001146D>) 17565795475856) 365
 (7FCED7FF0C80) : 7 (%UNLOCK-RECURSIVE-LOCK-OBJECT #<RECURSIVE-LOCK "Kernel exception-lock" [ptr @ #x16DFC40] #x30200001146D>) 405
 (7FCED7FF0CB0) : 8 (STATIC-CONS 102 ((SV::CONCAT 104 # #))) 181
              [...]
 (7FCED7FFEA18) : 490 (LP) 7709
 (7FCED7FFEA80) : 491 (CALL-CHECK-REGS LP) 229
 (7FCED7FFEAB8) : 492 (TOPLEVEL-EVAL (LP) NIL) 789
 (7FCED7FFEB30) : 493 (READ-LOOP :INPUT-STREAM #<SYNONYM-STREAM to *TERMINAL-IO* #x3020033EC7CD> :OUTPUT-STREAM #<SYNONYM-STREAM to *TERMINAL-IO* #x3020033EC66D> :BREAK-LEVEL 0 :PROMPT-FUNCTION #<Compiled-function (:INTERNAL CCL::READ-LOOP) (Non-Global)  #x300000590B8F>) 2509
 (7FCED7FFED78) : 494 (RUN-READ-LOOP :BREAK-LEVEL 0) 157
 (7FCED7FFEDA0) : 495 (TOPLEVEL-LOOP) 93
 (7FCED7FFEDB0) : 496 (FUNCALL #'#<(:INTERNAL (CCL:TOPLEVEL-FUNCTION (CCL::LISP-DEVELOPMENT-SYSTEM T)))>) 109
 (7FCED7FFEDD0) : 497 (FUNCALL #'#<(:INTERNAL CCL::MAKE-MCL-LISTENER-PROCESS)>) 661
 (7FCED7FFEE68) : 498 (RUN-PROCESS-INITIAL-FORM #<TTY-LISTENER listener(1) [Active] #x302000452AED> (#<CCL:COMPILED-LEXICAL-CLOSURE # #x3020033EB2DF>)) 813
 (7FCED7FFEEF0) : 499 (FUNCALL #'#<(:INTERNAL (CCL::%PROCESS-PRESET-INTERNAL (CCL:PROCESS)))> #<TTY-LISTENER listener(1) [Active] #x302000452AED> (#<CCL:COMPILED-LEXICAL-CLOSURE # #x3020033EB2DF>)) 581
 (7FCED7FFEF98) : 500 (FUNCALL #'#<(:INTERNAL CCL::THREAD-MAKE-STARTUP-FUNCTION)>) 277
***********************************************
************ ABORTING from raw Lisp ***********
Error:  Current process #<TTY-LISTENER listener(1) [Active] #x302000452AED> does not own lock #<RECURSIVE-LOCK "Kernel exception-lock" [ptr @ #x16DFC40] #x30200001146D>
NOTE: See above for backtrace.
***********************************************

I'm afraid I'm at a loss for how to reproduce this error as it occurred after more than a week of runtime in an ACL2 program. 63 other almost identical runs of the program executing simultaneously did not result in this error, so you could even say that this appeared after "more than a year" of CPU time... Hopefully the problem is obvious at a glance to someone sufficient understanding of CCL, but if not, please let me know if you have any ideas about how to isolate it. Thanks!

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

The CCL Objective-C bridge and Cocoa wrapper are great and useful, but it's not clear to me that they should be part of the main codebase.

I'd expect the main codebase to be the language, its basic low-level FFI, and their tests, with other things layered atop via ASDF.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

I think get-internal-real-time should return a monotonic/non-decreasing value. Even though the function is not mentioned in the manual (also an issue), its docstring says

Return the real time in the internal time format. (See INTERNAL-TIME-UNITS-PER-SECOND.) This is useful for finding elapsed time.

which a non-monotonic clock is not. Proposed solution: have get-internal-real-time derive its return value from clock_gettime(3) instead of gettimeofday(2).

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

http://trac.clozure.com/ccl/ticket/1345
http://trac.clozure.com/ccl/ticket/1393
http://trac.clozure.com/ccl/ticket/1408
http://trac.clozure.com/ccl/ticket/1409

possibly related:
http://trac.clozure.com/ccl/ticket/1142

For some reason, (quit) sometimes hangs on Windows. It seems to happen with both 32- and 64-bit versions of CCL, and on both 32- and 64-bit versions of Windows.

http://trac.clozure.com/ccl/ticket/1345 has the most details.

I've noticed that I can avoid the problem quite reliably when I change the compatibility setting of the wx86cl64.exe binary to Windows7. (This setting can be changed in the properties dialogue for the file wx86cl64.exe)

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Given

(format nil "~,,2f%" (coerce (/ 10 1576) 'float))

ccl result: "0.0063451775%"
sbcl result: "0.63451775%"

The interface databases that CCL uses are generated by a program called ffigen4. It is a set of patches to gcc-4.0.0 (see http://svn.clozure.com/publicsvn/ffigen4/)

These patches should be brought up-to-date. Alternatively, it might be an option to replace ffigen4 with some other tool. https://github.com/rpav/c2ffi might be suitable.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Typo at https://github.com/Clozure/ccl/blob/master/lisp-kernel/pmcl-kernel.c#L1932

"WINDOXS" should be "WINDOWS"

The fix needs to be compiled and tested on Windows.

From openmcl-devel (https://lists.clozure.com/pipermail/openmcl-devel/2017-April/011640.html)

(lisp-implementation-version)
"Version 1.11-r16635  (WindowsX8632)"

(ql:quickload '3d-vectors)

(let ((vec (3d-vectors::vec 5 0 0)))
  (psetf (3d-vectors::%vx3 vec) 5.0
         (3d-vectors::%vy3 vec) 5.0))

Signal error:
The value 5.0 is not of the expected type STRUCTURE.
[Condition of type TYPE-ERROR]

The sender notes: "The problem seems to be related to psetf. If you remove one of the set pairs, it works fine. If you use setf, it also works fine. 64 bit versions do not have such an issue, LinuxARM32 also passes."

A follow-up message provides a simpler test case

(defstruct vec
  (x 0.0 :type single-float)
  (y 0.0 :type single-float))

(let ((vec (make-vec)))
  (psetf (vec-x vec) 5.0
         (vec-y vec) 5.0))

The follow-up's author notes that the specific trigger combination seems to be psetf with typed structure slots.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

I primarily use the IDE when working with CCL, but it surprised me to learn it was part of the main codebase. It seems like it should be separate from CCL, as well as separate from the ObjC bridge and Cocoa wrapper, and maintainable independently.

An extracted IDE could be useful also as an example of how to build a native application using CCL, without integrating your code into the CCL codebase.

The spec does not indicate how to undefine a logical host. One way would be to:

(setf (logical-pathname-translations "HOST") nil)

A discussion on undefining logical hosts occurred in 2014 on Google groups.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

It has a bug when on android arm cffi call core dump .

(copied from http://trac.clozure.com/ccl/ticket/1406)

CL-USER> (let ((a (make-array 4 :element-type '(complex double-float)))
               (b (make-array 4 :element-type '(complex double-float))))
           (dotimes (i 4)
             (setf (aref b i) (complex (* 1.0d0 i) (* 2.0d0 i))))
           (format t "A: ~A~%" a)
           (format t "B: ~A~%" b)
           (replace a b)
           (format t "A <- B: ~A~%" a)
           nil)
A: #(#C(0.0D0 0.0D0) #C(0.0D0 0.0D0) #C(0.0D0 0.0D0) #C(0.0D0 0.0D0))
B: #(#C(0.0D0 0.0D0) #C(1.0D0 2.0D0) #C(2.0D0 4.0D0) #C(3.0D0 6.0D0))
A <- B: #(#C(0.0D0 0.0D0) #C(1.0D0 2.0D0) #C(2.0D0 4.0D0) #C(3.0D0 0.0D0))

The imagpart of the last element isn't making it though. Not sure if this test case exhausts all of the scenarios of this bug.

In windows 7 x64 bit system, I executed the following commands in CCL:
? (find-definition-sources 'defun 'function)
(((#<FUNCTION-DEFINITION-TYPE FUNCTION #x210028A3AD> . DEFUN) #<SOURCE-NOTE "ccl:lib;macros.lisp":29553-30891 NIL>))

? (ensure-source-note-text (nth 1 (car (find-definition-sources 'defun 'function))))
......

? (source-note-text (nth 1 (car (find-definition-sources 'defun 'function))))
"(if global-name
((declare (global-function-name ,global-name)))) ,@decls ,body)) (info (if (and inline-spec (binding-free-environment-p env) (nx-declared-inline-p inline-spec env) (not (and (symbolp inline-spec) (gethash inline-spec *NX1-ALPHATIZERS*)))) (cons doc lambda-expression) doc))) (progn
(%defun (nfunction ,spec ,lambda-expression) ',info)
',spec))))

(defmacro %defvar-init (var initform doc)
`(unless (%defvar ',var ,doc)
(set ',var ,initform)))

(defmacro defvar (&environment env var &optional (value () value-p) doc)
"Define a global variable at top level. Declare the variable
SPECIAL and, optionally, initialize it. If the variable already has a
value, the old value is not clobbered. The third argument is an optional
documentation string for the variable."
(if (and doc (not (stringp doc))) (report-bad-arg doc 'string))
(if (and (compile-file-environment-p env) (not fasl-save-doc-strings))
(setq doc nil))
(progn (eval-when (:compile-toplevel) (note-variable-info ',var ,value-p ,env)) ,(if value-p (%defvar-ini"
1338

Actually, function's source text position in file "ccl:lib;macros.lisp" is as follows:
29523 - 30955

I think, the reason of this problem maybe that source location parameter in function record-source-file is not correct.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

The compiler doesn't seem to deduce that the result of (+ a b) in this case is a fixnum:

(defun test-1 (a b)
  (declare (fixnum a b))
  (let ((sum (+ a b)))
    (DECLARE (FIXNUM sum))
    sum))

the disassembly shows a call to .SPFIX-OVERFLOW.

Using THE like below compiles entirely in line:

(defun test-2 (a b)
  (declare (fixnum a b))
  (let ((sum (THE FIXNUM (+ a b))))
    sum))

Unfortunately, I believe we have some code, eg in bignum arithmetic, that uses the former idiom rather than the latter...

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

(defun wrong-file-length (pathname)
  (with-open-file (stream pathname :direction :output)
    (write-string "hello" stream)
    (finish-output stream)
    (file-position stream :start)
    (file-length stream)))

The dirty flag gets cleared by fd-stream-force-output, which is a problem because the ioblock's fileeof slot won't get updated (in %ioblock-output-file-position) unless it's dirty.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

To be a good UNIX citizen, a built binary distribution of CCL should be structured for extraction into either /opt or /usr/local by default.

For example, the contents of the Darwin 64-bit Intel distribution should probably look something like this:

• bin/ccl64
• lib/ccl/dx86cl64
• lib/ccl/dx86cl64.image
• share/doc/ccl/release-notes.txt
• share/doc/ccl/manual.pdf
• share/doc/ccl/examples/
• share/man/man1/ccl64.1

Or something along those lines.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.