vseloved / rutils Goto Github PK

View Code? Open in Web Editor NEW

246.0 15.0 36.0 513 KB

Radical Utilities for Common Lisp

License: Other

Common Lisp 100.00%

rutils's Introduction

RUTILS is a syntactic utilities package for Common Lisp

(that aims to radically simplify the Lisp programmer's life both in the REPL and in large system development)

RUTILS stands for RADICAL-UTILITIES :)

See the announcement.

Read the tutorial for usage details or the quickdocs.

Short history

Originally, RUTILS stood for REASONABLE-UTILITIES, but now I switched to calling them RADICAL-UTILITIES, because they make some uncommon and not generally acceptable choices in terms of style of CL development. Namely:

Introducing several literal notations for anonymous functions, hash-tables, heredoc-strings etc.
Adding a lot of abbreviations of long symbols from the CL package
Having several versions of the same utilities with different names

But, what's more important, they aim to radically simplify Lisp programming in the small (with a special accent on simplifying REPL-driven development).

See the original rationale for REASONABLE-UTILITIES.

The current version is 5.0.0.

Major Features

"Modern" readtable with support for literal syntax for hash-tables, lambdas and heredoc-strings
Common macros like WITH-GENSYMS, ONCE-ONLY, EVAL-ALWAYS etc. and other symbol manipulation utilities
Basic anaphoric macros
A simple pair data structure and accompanying utilities
List manipulation utilities
Hash-table manipulation utilities
Hash-set manipulation utilities
Sequence manipulation utilities (including SPLIT-SEQUENCE, or rather, simply SPLIT)
Array manipulation utilities
String manipulation utilities
Tree manipulation utilities
generic access to elements of any data structure: GENERIC-ELT/?
ITER macro with keywords support
generic BIND
Clojure-style threading macros -> and ->>

Explicitly not supported

The following broad topics are not supported by design:

Concurrency.

The reason is not that I consider this not useful or general-purpose enough, but rather, that it's a whole new paradigm, and the scope of RUTILS is too small to comfortably accommodate it.
Functional paradigm.

As in the above, it's as well a whole other paradigm. It has a limited, but reasonable support in CL. Other features should be unified in it's own package, and maybe RUTILS can serve as a model for such package or even accommodate it in the future.
Advanced collections are as well a separate area in CS, so it requires a lot of effort to maintain and develop a comprehensive package in it.

Look at FSET and CL-CONTAINERS for such things.
MOP. MOP abstraction layer is a CDR and it is as well an essential part of CL. It is implemented and well supported by CLOSER-MOP.

Technical notes

Dependencies

Exported features

The following symbols are added to *FEATURES*:

:split-sequence
:iter

Organizational notes

See LICENSE for usage permissions. See AUTHORS for credits.

rutils's People

Stargazers

Watchers

rutils's Issues

5 tests currently failed in the latest commit `79cb029`

On the master branch (commit 79cb029), 5 tests currently failed. The failures may have been ignored due to another mis(?)behavior of the in-house test suite rutils use.

Here are the test procedures.

CL-USER> (ql:quickload :rutils-test)
CL-USER> (should-test:test :package (find-package :rutils.test))
Test WITH-OUT-FILE/READ-FILE:   OK
T
CL-USER> (should-test:test :package (find-package :rtl))
Test HASH-TABLE-FROM-ALIST:   OK
Test MAPINDEX:   OK
Test SUBSTR:   OK
Test MAPTIMES:   OK
Test BUTLAST2:   OK
Test ->:   OK
Test DOWHILE-LET:   OK
Test XOR#:   OK
Test HASH-TABLE-FROM-PLIST:   OK
Test NOT-LESS:   OK
Test ABBR:   OK
Test LENGTH=:   OK
Test MAPTREE:   OK
Test COPY-HASH-TABLE:   OK
Test DOLEAVES:   OK
Test FMT:   OK
Test PLIST-TO-ALIST:   OK
Test SMART-SLOT-VALUE: 
(SMART-SLOT-VALUE (MAKE-FOO BAR T) 'BAR) FAIL
error: #<SB-PCL::MISSING-SLOT BAR {7018A6CD33}>
  FAILED
Test MAPPEND:   OK
Test NAMED-LAMBDA:   OK
Test ENDS-WITH:   OK
Test DYADIC:   OK
Test RANGE:   OK
Test LISTCASE:   OK
Test MERGE-HASH-TABLES:   OK
Test DECASE:   OK
Test ONCE-ONLY:   OK
Test DOPLIST:   OK
Test PCASE:   OK
Test NOT-MORE:   OK
Test IF-IT:   OK
Test PCCASE:   OK
Test SINGLE:   OK
Test DCCASE:   OK
Test HASH-TABLE-VALS:   OK
Test TAKE:   OK
Test STRCAT:   OK
Test CSWITCH:   OK
Test MAPKV:   OK
Test TREE-SIZE: 
(TREE-SIZE '(FOO)) FAIL
expect: 1
actual: 2
(TREE-SIZE '(FOO BAR)) FAIL
expect: 2
actual: 3
(TREE-SIZE '(FOO (BAR BAZ FOO))) FAIL
expect: 4
actual: 6
(TREE-SIZE '(FOO (BAR BAZ FOO) BAR)) FAIL
expect: 5
actual: 7
  FAILED
Test INTERLEAVE:   OK
Test MAPLEAVES: 
(MAPLEAVES #'1+ '(1 (2 3) (4 5))) FAIL
expect: (1 (2 4) (4 6))
actual: (2 (3 4) (5 6))
  FAILED
Test HASH-TABLE-TO-PLIST:   OK
Test ->>:   OK
Test UNION#:   OK
Test SHUFFLE:   OK
Test TREE-DEPTH:   OK
Test ROTATE:   OK
Test MAPCANINDEX:   OK
Test HASH-SET:   OK
Test TRYADIC:   OK
Test LAST1:   OK
Test MORE:   OK
Test WITH-OUTPUT-TO-LIST:   OK
Test VEC:   OK
Test ESWITCH:   OK
Test XOR:   OK
Test NSHUFFLE:   OK
Test INTER#:   OK
Test ADD#:   OK
Test HASH-TABLE-KEYS:   OK
Test DOWHILE-IT:   OK
Test COND-LET:   OK
Test BIND: 
(BIND ((A 1) (B C (VALUES 2 3)) ((D E &REST F) '(4 5 13 14 15 16))
       (((G G) (H H)) ?
        (LET ((G395 (MAKE-HASH-TABLE TEST 'EQL)))
          (SETF (GETHASH G G395) 6)
          (SETF (GETHASH H G395) 7)
          G395))
       ((I J) @ (MAKE-DUMMY-STRUCT-FOR-BIND I 8 J 9))
       (((K K) (L L)) @ (MAKE 'DUMMY-OBJ-FOR-BIND K 10 L 11))
       (#(_ M) (FUNCALL (LAMBDA () #(0 12)))))
  (LIST* A B C D E G H I J K L M F)) FAIL
error: #<SB-PCL::MISSING-SLOT J {7018AF9713}>
  FAILED
Test SUM:   OK
Test DOTABLE:   OK
Test HASH-TABLE-TO-ALIST:   OK
Test WHEN-IT:   OK
Test PECASE:   OK
Test EQUAL-LENGTHS:   OK
Test WITH-KEYS:   OK
Test BLANKP:   OK
Test AND-IT:   OK
Test DCONS:   OK
Test GROUP:   OK
Test INTERPOSE:   OK
Test SET-EQUAL:   OK
Test SWITCH:   OK
Test MULTIPLE-VALUE-PROG2:   OK
Test ENSURE-LIST:   OK
Test DLISTP:   OK
Test FLATTEN:   OK
Test LESS:   OK
Test SPLIT-STRING:   OK
Test DELETE-FROM-PLIST:   OK
Test SLICE:   OK
Test LAST-CHAR:   OK
Test AND-LET:   OK
Test IF-LET:   OK
Test DCASE:   OK
Test PARTITION-WITH:   OK
Test PERMUTATIONS:   OK
Test REMOVE-FROM-PLIST:   OK
Test STARTS-WITH:   OK
Test ALIST-TO-PLIST:   OK
Test PAIRS->HT:   OK
Test ?:   OK
Test COPY:   OK
Test PLISTP:   OK
Test DOINDEX:   OK
Test ZIP-WITH:   OK
Test STRJOIN:   OK
Test DOTREE: 
(LET (REZ)
  (DOTREE (EL '(1 (2 3) (4 5)) REZ)
    (PUSH (FIRST (MKLIST EL)) REZ))) FAIL
expect: (5 4 3 2 1)
actual: (5 4 4 3 2 2 1 1)
  FAILED
Test SPLIT-SEQUENCE:   OK
Test ALISTP:   OK
Test KEEP:   OK
Test COND-IT:   OK
Test ZIP*-WITH:   OK
Test DOLINES:   OK
Test DIFF#:   OK
Test ZIP:   OK
Test SPLIT-SEQUENCE-IF:   OK
Test ASSOC1:   OK
Test RE-SETF:   OK
Test PRINT-HASH-TABLE:   OK
Test 2ND:   OK
Test HT->PAIRS:   OK
Test ZIP*:   OK
Test LAST-ELT:   OK
Test WHEN-LET:   OK
Test PRODUCT:   OK
Test SPLIT-SEQUENCE-IF-NOT:   OK
NIL
#<HASH-TABLE :TEST EQL :COUNT 3 {7018A69BF3}>
#<HASH-TABLE :TEST EQL :COUNT 2 {7018A69C93}>

Clojure macros

Hello.

That must be useful to add «threading» macros -> and ->>.
What do you think about it?

"optimize (debug 0)" policy defeats slime-autodoc and slime-describe-function under SBCL

core/packages.lisp establishes compilation policy of (debug 0). It makes SBCL forget function arglist: http://sbcl.org/manual/index.html#Debugger-Policy-Control. Consequently, SLIME is unable to display arglist in the echo area and in the buffer of slime-describe-function.

It seems best not to modify default debug policy.

Doesn't build - bad MAPCAR use

I get this today:

;     (MAPCAR
;      (RUTILS.READTABLE::TRIVIAL-POSITIONAL-LAMBDA
;       (RUTILSX.BIND::SMART-SLOT-VALUE RUTILS.READTABLE:% RUTILSX.BIND::OBJ)))
; 
; caught WARNING:
;   The function was called with one argument, but wants at least two.

Help: := is a macro, not a function. [Condition of type UNDEFINED-FUNCTION]

Hello,

I've been using the smart slot-value accessors, so far with great success, but out of a sudden I'm getting this error.

I only use (named-readtables:in-readtable rutils:rutils-readtable) in my file and then can use @obj.slot.

I'm getting this error (image form due to color highlights):

And text form (given @vindarel's request):

:= is a macro, not a function.
   [Condition of type UNDEFINED-FUNCTION]

Restarts:
 0: [CONTINUE] Retry calling :=.
 1: [USE-VALUE] Call specified function.
 2: [RETURN-VALUE] Return specified values.
 3: [RETURN-NOTHING] Return zero values.
 4: [RETRY] Retry SLIME REPL evaluation request.
 5: [*ABORT] Return to SLIME's top level.
 --more--

Backtrace:
  0: ("undefined function" NIL (#<HASH-TABLE :TEST EQUAL :COUNT 0 {10061C4E13}>))
      [No Locals]
  1: ((:METHOD RUTILS.GENERIC::SMART-SET-SLOT-VALUE (T T T)) #<LOX.RESOLVER:RESOLVER {10061C4993}> LOX.RESOLVER::SCOPES (#<HASH-TABLE :TEST EQUAL :COUNT 0 {10061C4E13}>)) [fast-method]
      Locals:
        RUTILS.GENERIC::OBJ = #<LOX.RESOLVER:RESOLVER {10061C4993}>
        RUTILS.GENERIC::SLOT = LOX.RESOLVER::SCOPES
        RUTILS.GENERIC::VAL = (#<HASH-TABLE :TEST EQUAL :COUNT 0 {10061C4E13}>)
  2: ((SB-PCL::DEFAULT-ONLY RUTILS.GENERIC::SMART-SET-SLOT-VALUE) #<LOX.RESOLVER:RESOLVER {10061C4993}> LOX.RESOLVER::SCOPES (#<HASH-TABLE :TEST EQUAL :COUNT 0 {10061C4E13}>))
      Locals:
        SB-PCL::.ARG0. = #<LOX.RESOLVER:RESOLVER {10061C4993}>
        SB-PCL::.ARG1. = LOX.RESOLVER::SCOPES
        SB-PCL::.ARG2. = (#<HASH-TABLE :TEST EQUAL :COUNT 0 {10061C4E13}>)
        SB-PCL::EMF = #S(SB-PCL::FAST-METHOD-CALL :FUNCTION #<FUNCTION (SB-PCL::FAST-METHOD RUTILS.GENERIC::SMART-SET-SLOT-VALUE (T T T))> :PV NIL :NEXT-METHOD-CALL NIL :ARG-INFO (3))
  3: (LOX.RESOLVER::BEGIN-SCOPE #<LOX.RESOLVER:RESOLVER {10061C4993}>)
  4: ((:METHOD LOX.RESOLVER:RESOLVE (LOX.RESOLVER:RESOLVER LOX.SYNTAX.STMT:STMT-BLOCK)) #<LOX.RESOLVER:RESOLVER {10061C4993}> #<LOX.SYNTAX.STMT:STMT-BLOCK {10061C3A53}>) [fast-method]
  5: ((:METHOD LOX.RESOLVER:RESOLVE (LOX.RESOLVER:RESOLVER LIST)) #<LOX.RESOLVER:RESOLVER {10061C4993}> (#<LOX.SYNTAX.STMT:STMT-BLOCK {10061C3A53}>)) [fast-method]
  6: (LOX:RUN "{var a=3; print a;}")
  7: (SB-INT:SIMPLE-EVAL-IN-LEXENV (LOX:RUN "{var a=3; print a;}") #<NULL-LEXENV>)
  8: (EVAL (LOX:RUN "{var a=3; print a;}"))
 --more--

Unfortunately I can't find where this is even after enabling all debugging during compilation in my project.

Can you give any tips as what's going on? Thanks

Does not build on CLISP due to multiple-value-bind abbrev

On CLISP, multiple-value-bind is a special operator (as permitted by 3.1.2.1.2.2). As a result, rutils fails to build with the error "Can't ABBREViate a special-operator MULTIPLE-VALUE-BIND"

rutils compiled in presense of cl-ppcre fails to load without cl-ppcre

If we (ql:quickload :rutils) having cl-ppcre loaded, the rutils library is compiled and fasl files are generated.

When later, in another lisp session without cl-ppcre, we again (ql:quickload :rutils), we got an error message similar like this:

Package ((CL-PPCRE . #<CL-PPCRE package>)) referenced in compiled file
  C:/Users/anton/projects/cl-test-grid/work-dir/agent4/test-runs/20131011163611.g-ecl/fasl/ql/dists/quicklisp/software/rutils-20130720-git/core/syntax.fasc
but has not been created

This issue affects several libraries depending on rutils. During the cl-test-grid testing of quicklisp 2013-10-03 we see rutils, cl-redis and cl-secure-read failing because of this. The failure logs may be found in this report:
http://common-lisp.net/project/cl-test-grid/ql/quicklisp-2013-10-03-diff2.html - search for results marked by #1243540 and click the red link.

The reason is the following code at the bottom of core/syntax.lisp:

#+:cl-ppcre
(defmethod bind-dispatch ((arg string) &rest args)
  (assert (cdr args))
  `(ppcre:register-groups-bind ,(car args) (,arg ,(cadr args))))

Such conditional compilation sections often lead to difficulties.
I would propose to change solve it somehow differently:

add cl-ppcre as a dependency to rutils, if this method is definitely needed
separate cl-ppcre depending code into another ASDF system, rutils-ppcre,
if this method is needed only for some developers
remove the method if it's not neede

Make it possible to escape newlines in the #/ reader macro

As these should be used in documentation strings it might be really useful escape newlines as they don't look pretty (especially with whitespace) in many editors. Something like ~ at the end like format has would be really nice.

`with` does not support vectors

Looking at the tutorial and the source my impression is that with does not support vector/array destructuring, unlike metabang-bind https://common-lisp.net/project/metabang-bind/user-guide.html#array-bindings

Is this observation correct and are there any plans to implement that?

setf macro issue

rutils/core/abbr.lisp

Line 100 in 0ff2b06

(defsetf ? generic-setf)

I get this:

;   defining setf macro for ? when (SETF ?) was previously treated as a function

Tests fails on ABCL 1.9.2

Caught ASDF/FIND-SYSTEM:LOAD-SYSTEM-DEFINITION-ERROR while processing --eval option "(asdf:operate (quote asdf:test-op) (quote rutils))":
  #<LOAD-SYSTEM-DEFINITION-ERROR {579F4155}>

:left

Hello,

I believe that line 38 of sequence.lisp should read

:for left := (max (or (apply #'position delimiter seq

(the colon in :left should be omitted).

With kind regards ,

Bart

Iterate collect result-type inconsistent

In the version of iterate in contrib, the result type is not quoted in the macroexpansion so that it can "be non-literal" (Line 3069 of contrib/iter.lisp). However, in order to deal with the "maybe quoted" stuff, earlier in return-collection-code one level of quotation is stripped off the :result-type argument (Line 3003). This means that, in order to pass a literal type here, it has to be double quoted.

Example:

(rutilsx:iter
  (:repeat 5)
  (:collect "foo" :result-type ''vector))

As opposed to:

(rutilsx:iter
  (:repeat 5)
  (:collect "foo" :result-type 'vector))

Or:

(rutilsx:iter
  (:repeat 5)
  (:collect "foo" :result-type vector))

Which both result in (coerce #:result vector) and error out.

Reader Macros don't work

I put (:use :rutilsx) in my package definition and the ? operator also works.

But when I try to use the literal syntax for hashtables like this:

  (setf *state* #h(
                   :spaceship-acceleration 0.2
                   :max-speed 16.0
                   :inc-speed-p nil
                   :dec-speed-p nil
                   :rotate-delta 0.0
                   :player-dead-p nil
                   :init-timeout-millis 3000
                   :game-start-time (get-millis)
                   :spaceship nil))

I get

; caught ERROR:
;   READ error during COMPILE-FILE:
;   
;     no dispatch function defined for #\H

Am I using it wrong?

Alias := to psetf

I think that := should be useful as a replacement not only for setf but also for psetf. For this, it must be an alias for psetf. Its appearance already is light enough to prefer

(:= x ...)
(:= y ...)

over

(:= x ...
    y ...)

for sequential assigment, but the latter could be a great way to express parallel assignment.

dotable compilation breaks if body contains (declare (ignore ...))

The issue is really not a show-stopper. I am raising it as fixing it eliminates an unexpected error that may throw off a user. Also, dolist accepts declare forms. So should then dotable

The following code (adapted from the tutorial):

(let ((rez ()))
  (dotable (k v #h(1 :foo 2 :bar) rez)
    (declare (ignore v))
    (when (oddp k)
      (push k rez))))

does not compile because in the macro-expansion a NIL is inserted by the ,(when _ '(declare (ignore _)) form. When _ is nil, when returns nil and the declaration is placed incorrectly:

                (DOLIST (#:G11537 #:TABLE11538)
                  (DESTRUCTURING-BIND
                    (K . V)
                    #:G11537
                    NIL ; generated by the when form
                    (DECLARE (IGNORE V))
                    (WHEN (ODDP K) (PUSH K REZ))))

To avoid generating the NIL, the when form needs to be written like so:

,@(when _ `((declare (ignore _)))

Note the @ and the extra parenthesis around declare.

This whole problem can and should be avoided by using the _ underscore, eliminating the need for the user to use declare.

Again, I raise this, as it raises an error whereas dolist would not issue one.

`bind` fails

In bind-test.lisp, bind fails in that arbitrary symbols don't seem be bind well with structs.
I tested this with (should-test:test :package (find-package :rtl)).

The error message:

When attempting to read the slot's value (slot-value), the slot RUTILS.BIND::J is missing from the object #S(DUMMY-STRUCT-FOR-BIND :I 8 :J 9). It has a slot RUTILS.TEST::J, while RUTILS.BIND::J is requested.

The root cause may be the same as #77

dotable _ weirdness

I haven't looked at the expanded code yet, but when I try to load hash-set.lisp I get warnings of the following sort:

; processing (DEFUN |INTER#| ...)

; file: /home/sly/quicklisp/dists/quicklisp/software/rutils-20201220-git/core/hash-set.lisp
; in: DEFUN |INTER#|
;     (RUTILS.HASH-TABLE:DOTABLE (RUTILS.HASH-SET::ITEM RUTILS.HASH-SET::_
;                                 RUTILS.HASH-SET::SET1)
;       (WHEN (GETHASH RUTILS.HASH-SET::ITEM RUTILS.HASH-SET::SET2)
;         (RUTILS.HASH-SET:|ADD#| RUTILS.HASH-SET::ITEM SET)))
; --> LET BLOCK ETYPECASE LET COND IF PROGN MAPHASH LET* 
; ==>
;   (SB-KERNEL:%COERCE-CALLABLE-TO-FUN
;    (LAMBDA (RUTILS.HASH-SET::ITEM RUTILS.HASH-SET::_)
;      (DECLARE (IGNORE RUTILS.BIND:_))
;      (WHEN (GETHASH RUTILS.HASH-SET::ITEM RUTILS.HASH-SET::SET2)
;        (RUTILS.HASH-SET:|ADD#| RUTILS.HASH-SET::ITEM SET))))
; 
; caught STYLE-WARNING:
;   IGNORE declaration for an unknown variable: _
; 
; caught STYLE-WARNING:
;   The variable _ is defined but never used.

This suggests that the scope of _ is somehow wrong.

Broken today - "The variable BY occurs more than once in the lambda list."

Full log here: http://report.quicklisp.org/2020-06-14/failure-report/rutils.html#rutils

Broken docs on quickdocs

Although the main page:
http://quickdocs.org/rutils Is visible, links that lead to:
https://quickdocs.org/docs/tutorial.md
and
https://quickdocs.org/docs/api
are 404.

Style warning in SBCL (unused variable) using DOPLIST

Code to reproduce:

(defparameter *pandoc-vars*
  '(:mainfont "Liberation Sans"
     :documentclass "report"
     :geometry (:margin "2cm")))

(rtl:doplist (key val *pandoc-vars*)
  (princ (list val key)))

Warning:

; in: RUTILS.LIST:DOPLIST (KEY VAL *PANDOC-VARS*)
;     (DESTRUCTURING-BIND
;         (KPTB::KEY KPTB::VAL &REST #:G822)
;         #:G821
;       (PRINC (LIST KPTB::VAL KPTB::KEY)))
; ==>
;   (LET* ((#:G2
;           (SB-C::CHECK-DS-LIST/&REST #:G821 2 2
;                                      '(KPTB::KEY KPTB::VAL &REST #:G822)))
;          (KPTB::KEY (POP #:G2))
;          (KPTB::VAL (POP #:G2))
;          (#:G822 #:G2))
;     (PRINC (LIST KPTB::VAL KPTB::KEY)))
; 
; caught STYLE-WARNING:
;   The variable #:G822 is defined but never used.
; 
; compilation unit finished
;   caught 1 STYLE-WARNING condition

Does not build on SBCL 1.1.17 - sharpquote issue?

I'm not sure what's up, it's something in once-only choking it up:

http://report.quicklisp.org/rutils/2014-05-04/failtail.txt

Push an element to the end of a list?

Is there some utility to push some element to the end of a list? Makes sense to implement one?

(let ((l (list :foo)))
(push-end :bar l)
l)
=> (:foo :bar)

push-end, push-last, I don't know how to call it.

RUTILS.ITER:ITER, :in-vector or :in-sequence "The value NIL is not of type NUMBER"

The code:

(rutils.iter:iter
	  (:for item :in-vector #(1 2 3 4 5))
	  (format t "Hi ~A~%" item))

(rutils.iter:iter
	  (:for item :in-sequence #(1 2 3 4 5))
	  (format t "Hi ~A~%" item))

causes the following error:

The value
  NIL
is not of type
  NUMBER
   [Condition of type TYPE-ERROR]

Backtrace:

  0: (SB-KERNEL:%NEGATE NIL)
  1: (- NIL)
  2: (SB-INT:SIMPLE-EVAL-IN-LEXENV (- NIL) #<NULL-LEXENV>)
  3: (EVAL (- NIL))
  4: (RUTILS.ITER::RETURN-SEQ-CODE :ELEMENT-VAR ITEM :SEQUENCE #(1 2 3 4 5) :ACCESS-FN AREF :SIZE-FN #<unavailable argument> :ELEMENT-TYPE NIL :SEQUENCE-TYPE #<unavailable argument> :FROM #<unavailable arg..
  5: (RUTILS.ITER::APPLY-CLAUSE-FUNCTION #<unavailable argument> #<unavailable argument>)
  6: (RUTILS.ITER::PROCESS-CLAUSE #<unavailable argument>)
  7: (RUTILS.ITER::WALK-LIST-NCONCING #<unavailable argument> #<FUNCTION RUTILS.ITER::WALK> #<FUNCTION (LAMBDA (RUTILS.ITER::FORM RUTILS.ITER::BODY) :IN "/home/piknik/.local/share/common-lisp/source/rutils..
  8: ((MACRO-FUNCTION RUTILS.ITER:ITER) #<unavailable argument> #<unavailable argument>)
  9: ((FLET SB-IMPL::PERFORM-EXPANSION :IN MACROEXPAND-1) #<FUNCTION (MACRO-FUNCTION RUTILS.ITER:ITER) {52A7C70B}> NIL)
 10: (MACROEXPAND (RUTILS.ITER:ITER (:FOR ITEM :IN-VECTOR #(1 2 3 4 5)) (FORMAT T "Hi ~A~%" ITEM)) #<NULL-LEXENV>)
 11: (SB-INT:SIMPLE-EVAL-IN-LEXENV (RUTILS.ITER:ITER (:FOR ITEM :IN-VECTOR #(1 2 3 4 5)) (FORMAT T "Hi ~A~%" ITEM)) #<NULL-LEXENV>)
 12: (EVAL (RUTILS.ITER:ITER (:FOR ITEM :IN-VECTOR #(1 2 3 4 5)) (FORMAT T "Hi ~A~%" ITEM)))

The following code outputs the expected result:

(rutils.iter:iter
	  (:for item :in '(1 2 3 4 5))
	  (format t "Hi ~A~%" item))
Hi 1
Hi 2
Hi 3
Hi 4
Hi 5

Not sure what I am doing wrong or if it's a bug. Used both the current quicklisp version and the master branch.

Feature request: (setf (rutil.lt X) Y) and similar for rutil.rt

When executing

(setf (rutil:lt (rutil:pair 0 1)) 1)

I get

The function (COMMON-LISP:SETF RUTILS.PAIR:LT) is undefined.

I can use car and cadr as workarounds.

Fails to loop `:in-vector`

Based on the documentation of rutils, it seems that rutils:iter should behave in the same way as iter:iter, except that keywords are actually keywords. While looping across a vector, I realized there's a deviation from that promise.

iter>  (iter (for x in-vector (vector 1 2 3))
       (collect x))
(1 2 3)
iter>  (rutils:iter (:for x :in-vector (vector 1 2 3))
       (:collect x))
; Debugger entered on #<TYPE-ERROR expected-type: NUMBER datum: NIL>

Here is the full debugger info.

The value
  NIL
is not of type
  NUMBER
   [Condition of type TYPE-ERROR]

Restarts:
 0: [RETRY] Retry SLY mREPL evaluation request.
 1: [*ABORT] Return to SLY's top level.
 2: [ABORT] abort thread (#<THREAD "sly-channel-1-mrepl-remote-1" RUNNING {100BEC4633}>)

Backtrace:
 0: (SB-KERNEL:%NEGATE NIL)
 1: (- NIL)
 2: (SB-INT:SIMPLE-EVAL-IN-LEXENV (- NIL) #<NULL-LEXENV>)
 3: (EVAL (- NIL))
 4: (RUTILS.ITER::RETURN-SEQ-CODE :ELEMENT-VAR X :SEQUENCE (VECTOR 1 2 3) :ACCESS-FN AREF :SIZE-FN #<unavailable argument> :ELEMENT-TYPE NIL :SEQUENCE-TYPE #<unavailable argument> :FROM #<unavailable argu..
 5: (RUTILS.ITER::APPLY-CLAUSE-FUNCTION #<unavailable argument> #<unavailable argument>)
 6: (RUTILS.ITER::PROCESS-CLAUSE #<unavailable argument>)
 7: (RUTILS.ITER::WALK-LIST-NCONCING #<unavailable argument> #<FUNCTION RUTILS.ITER::WALK> #<FUNCTION (LAMBDA (RUTILS.ITER::FORM RUTILS.ITER::BODY) :IN "/Users/jin/.roswell/lisp/quicklisp/dists/quicklisp/..
 8: ((MACRO-FUNCTION RUTILS.ITER:ITER) #<unavailable argument> #<unavailable argument>)
 9: ((FLET SB-IMPL::PERFORM-EXPANSION :IN MACROEXPAND-1) #<FUNCTION (MACRO-FUNCTION RUTILS.ITER:ITER) {2312B26B}> NIL)
10: (MACROEXPAND (RUTILS.ITER:ITER (:FOR X :IN-VECTOR (VECTOR 1 2 3)) (:COLLECT X)) #<NULL-LEXENV>)
11: (SB-INT:SIMPLE-EVAL-IN-LEXENV (RUTILS.ITER:ITER (:FOR X :IN-VECTOR (VECTOR 1 2 3)) (:COLLECT X)) #<NULL-LEXENV>)
12: (EVAL (RUTILS.ITER:ITER (:FOR X :IN-VECTOR (VECTOR 1 2 3)) (:COLLECT X)))
13: ((LAMBDA NIL :IN SLYNK-MREPL::MREPL-EVAL-1))
14: (SLYNK::CALL-WITH-RETRY-RESTART "Retry SLY mREPL evaluation request." #<CLOSURE (LAMBDA NIL :IN SLYNK-MREPL::MREPL-EVAL-1) {104761E12B}>)
15: ((LAMBDA NIL :IN SLYNK-MREPL::MREPL-EVAL-1))
16: ((LAMBDA NIL :IN SLYNK::CALL-WITH-LISTENER))
17: (SLYNK::CALL-WITH-BINDINGS ((*PACKAGE* . #<PACKAGE "ITER">) (*DEFAULT-PATHNAME-DEFAULTS* . #P"/Users/jin/") (* . #1=(1 2 3)) (** 1 2 3) (*** 1 2 3) (/ #1#) ...) #<CLOSURE (LAMBDA NIL :IN ..
18: (SLYNK-MREPL::MREPL-EVAL-1 #<SLYNK-MREPL::MREPL mrepl-1-1> " (rutils:iter (:for x :in-vector (vector 1 2 3)) ..)
19: (SLYNK-MREPL::MREPL-EVAL #<SLYNK-MREPL::MREPL mrepl-1-1> " (rutils:iter (:for x :in-vector (vector 1 2 3)) ..)
20: (SLYNK:PROCESS-REQUESTS NIL)
21: ((LAMBDA NIL :IN SLYNK::SPAWN-CHANNEL-THREAD))
22: ((LAMBDA NIL :IN SLYNK::SPAWN-CHANNEL-THREAD))
23: (SLYNK-SBCL::CALL-WITH-BREAK-HOOK #<FUNCTION SLYNK:SLYNK-DEBUGGER-HOOK> #<CLOSURE (LAMBDA NIL :IN SLYNK::SPAWN-CHANNEL-THREAD) {100BEBFE6B}>)
24: ((FLET SLYNK-BACKEND:CALL-WITH-DEBUGGER-HOOK :IN "/Users/jin/.emacs.d/.local/straight/repos/sly/slynk/backend/sbcl.lisp") #<FUNCTION SLYNK:SLYNK-DEBUGGER-HOOK> #<CLOSURE (LAMBDA NIL :IN SLYNK::SPAWN-C..
25: ((LAMBDA NIL :IN SLYNK::CALL-WITH-LISTENER))
26: (SLYNK::CALL-WITH-BINDINGS ((*PACKAGE* . #<PACKAGE "ITER">) (*DEFAULT-PATHNAME-DEFAULTS* . #P"/Users/jin/") (* . #1=(1 2 3)) (** 1 2 3) (*** 1 2 3) (/ #1#) ...) #<CLOSURE (LAMBDA NIL :IN ..
27: ((LAMBDA NIL :IN SLYNK::SPAWN-CHANNEL-THREAD))
28: ((FLET SB-UNIX::BODY :IN SB-THREAD::NEW-LISP-THREAD-TRAMPOLINE))
29: ((FLET "WITHOUT-INTERRUPTS-BODY-4" :IN SB-THREAD::NEW-LISP-THREAD-TRAMPOLINE))
30: ((FLET SB-THREAD::WITH-MUTEX-THUNK :IN SB-THREAD::NEW-LISP-THREAD-TRAMPOLINE))
31: ((FLET "WITHOUT-INTERRUPTS-BODY-1" :IN SB-THREAD::CALL-WITH-MUTEX))
32: (SB-THREAD::CALL-WITH-MUTEX #<CLOSURE (FLET SB-THREAD::WITH-MUTEX-THUNK :IN SB-THREAD::NEW-LISP-THREAD-TRAMPOLINE) {D1F7E7B}> #<SB-THREAD:MUTEX "thread result lock" owner: #<SB-THREAD:THREAD "sly-chan..
33: (SB-THREAD::NEW-LISP-THREAD-TRAMPOLINE #<SB-THREAD:THREAD "sly-channel-1-mrepl-remote-1" RUNNING {100BEC4633}> NIL #<CLOSURE (LAMBDA NIL :IN SLYNK::SPAWN-CHANNEL-THREAD) {100BEC3C1B}> NIL)
34: ("foreign function: call_into_lisp")
35: ("foreign function: new_thread_trampoline")

Does not build - issue with RUTILS.CORE::ARGLIST

http://report.quicklisp.org/2015-09-09/failure-report/rutils.html#rutils has a log. I get an error that RUTILS.CORE::ARGLIST is not a function.

rutilsx system does not build

I get this: Package RUTILSX.BIND does not exist. Line: 68, Column: 39, File-Position: 2282 Stream: #<SB-INT:FORM-TRACKING-STREAM for "file /home/quicklisp/quicklisp-controller/dist/build-cache/rutils/0261dca135efb32cd339cef9a377774e31782b4d/rutils-20190830-git/contrib/generators.lisp" {1006421253}>

Full log here: http://report.quicklisp.org/2019-08-30/failure-report/rutils.html#rutilsx

Following the instructions on writing a POS tagger results in error on text-tokens (CCL::UNDEFINED-FUNCTION-CALL).

On docs/user-guide/examples/eng-pos-tagger.md are given some instructions that fail:

The following code:

NLP> (let ((words-dist #h(equal))
       (map-corpus :ptb-tagged (corpus-file "ptb/TAGGED/POS/WSJ")
                   #`(dolist (sent (text-tokens %))
                       (dolist (tok sent)
                         (unless (in# (token-word tok) words-dist)
                           (:= (get# (token-word tok) words-dist) #h()))
                         (:+ (get# (token-tag tok)
                                   (get# (token-word tok) words-dist)
                                   0))))
                   :ext "POS")
       words-dist)
#<HASH-TABLE :TEST EQUAL :COUNT 51457 {10467E6543}>
NLP> (reduce #'+ (mapcan #'ht-vals (ht-vals *)))
1289201

... apears to be two separate forms: the let and the reduce form.

The let form appears to be unbalanced. It lacks one parenthesis.
If we add a closing parenthesis on (words-dist #h(equal))), there appears two errors:

NLP> (let ((words-dist #h(equal)))
       (map-corpus :ptb-tagged (corpus-file "ptb/TAGGED/POS/WSJ")
                   #`(dolist (sent (text-tokens %))
                       (dolist (tok sent)
                         (unless (in# (token-word tok) words-dist)
                           (:= (get# (token-word tok) words-dist) #h()))
                         (:+ (get# (token-tag tok)
                                   (get# (token-word tok) words-dist)
                                   0))))
                   :ext "POS")
       words-dist)

The first error is that there is no file WSJ under corpora/ptb/TAGGED/POS/

But if we change it to an existing corpora under corpora/, as "onf-wsj":

NLP> (let ((words-dist #h(equal)))
       (map-corpus :ptb-tagged (corpus-file "onf-wsj")
                   #`(dolist (sent (text-tokens %))
                       (dolist (tok sent)
                         (unless (in# (token-word tok) words-dist)
                           (:= (get# (token-word tok) words-dist) #h()))
                         (:+ (get# (token-tag tok)
                                   (get# (token-word tok) words-dist)
                                   0))))
                   :ext "POS")
       words-dist)

Then CCL:UNDEFINED-FUNCTION-CALL is spawned. There is no such function.

Any clues?

I'm using Clozure Common Lisp 1.10. Under SBCL, it made a thread-error by just running the first let. Using Windows 8 64-bit.

Iterate's example of "in" does not work with rutils

Here is the code I'm trying to evaluate:

(iter outer (:for i :below (array-dimension ar 0))
             (iter (:for j :below (array-dimension ar 1))
               (:in outer (:collect (aref ar i j)))))

It produces error:

Iterate, in (IN OUTER #1=(COLLECT (AREF AR I J))):
#1# should be a symbol
   [Condition of type SIMPLE-ERROR]

Restarts:
 0: [RETRY] Retry SLY mREPL evaluation request.
 1: [*ABORT] Return to SLY's top level.
 2: [ABORT] abort thread (#<THREAD "sly-channel-2-mrepl-remote-2" RUNNING {100501E0C3}>)

Backtrace:
 0: (RUTILS.ITER::CLAUSE-ERROR #<unavailable argument> #<more unavailable arguments>) [more]
 1: (RUTILS.ITER::PREPROCESS-CLAUSE (RUTILS.ITER::IN OUTER (:COLLECT (AREF AR I J))))
 2: (RUTILS.ITER::PROCESS-CLAUSE #<unavailable argument>)
 3: (RUTILS.ITER::WALK-LIST-NCONCING #<unavailable argument> #<FUNCTION RUTILS.ITER::WALK> #<FUNCTION (LAMBDA (RUTILS.ITER::FORM RUTILS.ITER::BODY) :IN RUTILS.ITER::WALK-ARGLIST) {22A0FB4B}>)
 4: (RUTILS.ITER::WALK-ARGLIST #<unavailable argument>)
 5: (RUTILS.ITER::RETURN-CODE-MODIFYING-BODY #<FUNCTION RUTILS.ITER::WALK-ARGLIST> ((SETQ #1=#:LIMIT67 (ARRAY-DIMENSION AR 1)) (SETQ J -1) LOOP-TOP-NIL (SETQ J (+ J 1)) (WHEN (>= J #1#) (GO LOOP-END-NIL))..
 6: (RUTILS.ITER::WALK-SPECIAL-FORM #<unavailable argument>)
 7: (RUTILS.ITER::WALK-LIST-NCONCING #<unavailable argument> #<FUNCTION RUTILS.ITER::WALK> #<FUNCTION (LAMBDA (RUTILS.ITER::FORM RUTILS.ITER::BODY) :IN RUTILS.ITER::WALK-ARGLIST) {22A0FB4B}>)
 8: (RUTILS.ITER::WALK-ARGLIST #<unavailable argument>)
 9: (RUTILS.ITER::RETURN-CODE-MODIFYING-BODY #<FUNCTION RUTILS.ITER::WALK-ARGLIST> ((TAGBODY (SETQ #:LIMIT67 #) (SETQ J -1) LOOP-TOP-NIL (SETQ J #) (WHEN # #) ...) NIL) #<CLOSURE (LAMBDA (&OPTIONAL RUTILS..
10: (RUTILS.ITER::WALK-SPECIAL-FORM #<unavailable argument>)
11: (RUTILS.ITER::WALK-LIST-NCONCING #<unavailable argument> #<FUNCTION RUTILS.ITER::WALK> #<FUNCTION (LAMBDA (RUTILS.ITER::FORM RUTILS.ITER::BODY) :IN "/Users/art/projects/lisp/lisp-project-of-the-day/.q..
12: (RUTILS.ITER::WALK-LET LET* #<unavailable argument> #<unavailable &REST argument>)
13: (RUTILS.ITER::WALK-SPECIAL-FORM #<unavailable argument>)
14: (RUTILS.ITER::WALK-LIST-NCONCING #<unavailable argument> #<FUNCTION RUTILS.ITER::WALK> #<FUNCTION (LAMBDA (RUTILS.ITER::FORM RUTILS.ITER::BODY) :IN "/Users/art/projects/lisp/lisp-project-of-the-day/.q..
15: ((MACRO-FUNCTION ITER) #<unavailable argument> #<unavailable argument>)
16: ((FLET SB-IMPL::PERFORM-EXPANSION :IN MACROEXPAND-1) #<FUNCTION (MACRO-FUNCTION ITER) {22A0DC5B}> NIL)
17: (MACROEXPAND (ITER OUTER (:FOR I :BELOW (ARRAY-DIMENSION AR 0)) (ITER (:FOR J :BELOW #) (:IN OUTER #))) #<NULL-LEXENV>)

This is a direct translation of this example from the original documentation:

(iter outer (for i below (array-dimension ar 0))
            (iter (for j below (array-dimension ar 1))
                  (in outer (collect (aref ar i j)))))

rutils won’t load on LispWorks 7.1.1

Similar to #29, when I load rutils from Quicklisp on LispWorks 7.1.1, I get this:

Error: Defining macro := visible from package KEYWORD { *handle-warn-on-redefinition* is :ERROR }
   1 (continue) Define it anyway.
   2 Try loading C:\Documents and Settings\Administrator\Local Settings\Application Data\cache\common-lisp\lw-7.1.1-win-x64\C\Documents and Settings\Administrator\quicklisp\dists\quicklisp\software\rutils-20180831-git\core\abbr.64ofasl again.
   3 Give up loading C:\Documents and Settings\Administrator\Local Settings\Application Data\cache\common-lisp\lw-7.1.1-win-x64\C\Documents and Settings\Administrator\quicklisp\dists\quicklisp\software\rutils-20180831-git\core\abbr.64ofasl.
   4 Try loading another file instead of C:\Documents and Settings\Administrator\Local Settings\Application Data\cache\common-lisp\lw-7.1.1-win-x64\C\Documents and Settings\Administrator\quicklisp\dists\quicklisp\software\rutils-20180831-git\core\abbr.64ofasl.
   5 Recompile abbr and try loading it again
   6 Retry loading FASL for
   #<ASDF/LISP-ACTION:CL-SOURCE-FILE "rutils" "core" "abbr">.
   7 Continue, treating loading FASL for
   #<ASDF/LISP-ACTION:CL-SOURCE-FILE "rutils" "core" "abbr">
     as having been successful.
   8 Retry ASDF operation.
   9 Retry ASDF operation after resetting the configuration.
  10 Retry ASDF operation.
  11 Retry ASDF operation after resetting the configuration.
  12 (abort) Give up on "rutils"
  13 Return to top loop level 0.
 
Type :b for backtrace or :c <option number> to proceed.
Type :bug-form "<subject>" for a bug report template or :? for other options.
 
RUTILS.ABBR 2 : 1 >

Is not defining those macros on LispWorks the only option?

Possible improvement to generic-elt on objects

From your blog post:

Still, one issue remains unsolved in this approach: the preferred Lisp slot-access method is not via slot-value, but with an accessor method that is exported

given that the preferred method is with an accessor, why not allow something like:

(? obj #'accessor-fn)

generic-elt doesn't work for plists

I have a plist like

(defparameter my-plist (list :a 1 :b 2 :c (list :a 4)))

(? my-plist :a) returns an error, because it tries to do

(nthcdr :a (:a 1 :b 2 :c (:a 4)))

Is it viable to make a distinction between regular lists and plists/alists in (generic-elt ((obj list)...))) or would the decrease in performance not be worth it?

Can't compile version from QL on LW7.0

Trying to load from Quicklisp in LispWorks7 REPL. Here is the result:

(ql:quickload "rutils")
To load "rutils":
  Load 1 ASDF system:
    rutils
; Loading "rutils"

Error: Defining macro :* visible from package KEYWORD.
   1 (continue) Define it anyway.
   2 Try loading /Users/alexeyv/.cache/common-lisp/lw-7.0.0-macosx-x86/Users/alexeyv/.quicklisp/dists/quicklisp/software/rutils-20160531-git/core/abbr.xfasl again.
   3 Give up loading /Users/alexeyv/.cache/common-lisp/lw-7.0.0-macosx-x86/Users/alexeyv/.quicklisp/dists/quicklisp/software/rutils-20160531-git/core/abbr.xfasl.
   4 Try loading another file instead of /Users/alexeyv/.cache/common-lisp/lw-7.0.0-macosx-x86/Users/alexeyv/.quicklisp/dists/quicklisp/software/rutils-20160531-git/core/abbr.xfasl.
   5 Recompile abbr and try loading it again
   6 Retry loading FASL for
   #<ASDF/LISP-ACTION:CL-SOURCE-FILE "rutils" "core" "abbr">.
   7 Continue, treating loading FASL for
   #<ASDF/LISP-ACTION:CL-SOURCE-FILE "rutils" "core" "abbr">
     as having been successful.
   8 Retry ASDF operation.
   9 Retry ASDF operation after resetting the configuration.
  10 (abort) Give up on "rutils"
  11 Return to level 0.
  12 Return to top loop level 0.

Type :b for backtrace or :c <option number> to proceed.
Type :bug-form "<subject>" for a bug report template or :? for other options.

remove-empty-subseqs defaults to nil in split*

The tutorial states that the split* functions use a default value of t for remove-empty-subseqs. However, a value of nil is actually used.

I am using the quicklisp version (20191130-git) but looking at core/sequence.lisp, I see that it contains (remove-empty-subseqs nil) in the argument list for the split function, so I assume that the issue is present there as well.

Potential bugs for `RUTILS:ITER`

I ported a lot of tests from the traditional iter package, and there are a few potential bugs RUTILS:ITER has to tackle with. Please note that the traditional iter also suffers from some test failures as well. However, RUTILS:ITER has more.

generic-elt with negative keys (list/vector)

generic-elt on lists & vectors has special logic for handling negative keys:

(when (minusp key) (setf key (- (length obj) key)))

My understanding is that this is for "Python-style" access where we can access the last element of a list with:

my_list[-1]

However the above doesn't seem to work in practice:

RTL-USER> (? '(foo bar) -1)
NIL
RTL-USER> (? (make-array 10) -1)
;; Debugger: Invalid index 11 for (SIMPLE-VECTOR 10), should be a non-negative integer below 10.

Because the key is negative, generic-elt should be adding the key to the length rather than subtracting it (equivalent to adding the abs val of key).

:= is too surprising compared to psetf

With regards to the most recent commit,

f2d1914#diff-e9b8b78804398dc454c884ce19e57889

:= was changed from a simple alias of psetf to a buggy (depending on interpretation) re-implementation of psetf.

In this example,

(:= a b
c d)

If we are to follow the other behaviour of psetf as closely as possible, then order of evaluation of the values must be sequential, b must be evaluated before d.

Furthermore, side effects in places should also occur before those in values. To use a different example,

(let ((i 0))
(:= (elt a (incf i)) (incf i)))

is now equivalent to (setf (elt a 2) 1) when it should be equivalent to (setf (elt a 1) 2).

I believe the intend for := should be to follow psetf and setf as closely as possible, so the above behaviour should be considered buggy. In that case, the best intro to generalised variables I've found is in On Lisp:

http://www.bookshelf.jp/texi/onlisp/onlisp_13.html#SEC89

VOID macro declaimed as inline

The following line in misc.lisp declaims the macro VOID to be inline:

(declaim (inline or2 and2 xor2 void true))

This should probably be removed.

`smart-slot-value` fails

In generic-test.lisp, smart-slot-value fails in that it doesn't disregard the package of the slot name.

  (should be true (smart-slot-value (rtl::make-foo :bar t) 'bar))

The error message:

When attempting to read the slot's value (slot-value), the slot BAR is missing from the object #S(RTL::FOO :BAR T). It has a slot RTL::BAR, while RUTILS.TEST::BAR is requested.

with/bind ignorable variable `_' generates "unused lexical variable" compiler warning

By way of example,

(rutils:bind (((d _ e &rest f) '(4 0 5 12 13 14 15)))
  (list d e f))

generates

;Compiler warnings :
;   In an anonymous lambda form: Unused lexical variable _

This is on

CL-USER> (lisp-implementation-type)
"Clozure Common Lisp"
CL-USER> (lisp-implementation-version)
"Version 1.11-r16635  (WindowsX8664)"

:+ and :- break indentation of + and - in SLIME

Since incf and decf are macros, and macro definitions alter SLIME indentation of all symbols with the same name regardless of their package, loading rutils alters indentation of + and -.

In SBCL, before loading rutils:

(+ (f)
   1
   2)

after loading rutils:

(+ (f)
    1
    2)

This can be fixed either by evaluating (put '+ 'common-lisp-indent-function '(as nil)) (same with '-) in Emacs (and describing this workaround in rutils wherever :+ and :- and defined or documented), or by altering swank:*application-hints-tables* accordingly (as implemented in trivial-indent).