Hi,
On Mac OS 10.6 (64-bit build).  I ran into a problem with the gffi module.  
I've bumped into this 
problem before with macs where the MacPorts libiconv conflict with the system 
version.

Here's how i fixed it.  First, at least on mac, ffi depends on iconv, but it's 
not in specified and the 
dlopen borks on "_iconv" boo.  Probably some crappy static initializer that 
isn't actually used.
Anyways that's easy to fix:

IN gpsee/modules/gff/module.mk 

added:
ifneq (X$(ICONV_LIB_NAME),X)
EXTRA_LDFLAGS           += -l$(ICONV_LIB_NAME)
endif

then
sudo mv  /opt/local/lib/libiconv.dylib /opt/local/lib/libiconv.dylib-old

then did a sudo make build/sudo make install


then I was able to have a 

I think it would help to print out dlerror to help track down these buggies.
gpsee_modules.c

  if (!module->DSOHnd)
  {
printf(">>> load failed %s\n", dlerror());  <--- OR SOMETHING

    return dlerror();
  }


hope this helps someone

The system module's name conflicts with CommonJS.

We need a new name; probably "gpsee"

donny@donny-desktop:~/src/hg/gpsee/tests/binary$ make
./ByteString.js -q > ByteString.test.temp && touch ByteString
ByteString.test && diff ByteString.test ByteString.test.temp
gpsee: Already throwing an exception; not throwing
'gpsee.module.ca.page.binary.ByteString.constructor.arguments: invalid kind
or number of arguments'!
0a1,5
> Exception thrown: Error:
gpsee.module.ca.page.binary.ByteString.constructor.transcode.length: String
length exceeds maximum characters, cannot convert
> the test that failed this way:
> function () {
>     return (new ByteString("hello", "utf-8")).toByteString("utf-8",
"utf-16").length === 12;
> }
make: [all] Error 1 (ignored)
./ByteArray.js -q > ByteArray.test.temp && touch ByteArray.test && diff
ByteArray.test ByteArray.test.temp
gpsee: Already throwing an exception; not throwing
'gpsee.module.ca.page.binary.ByteArray.constructor.arguments: invalid kind
or number of arguments'!
0a1,5
> Exception thrown: Error:
gpsee.module.ca.page.binary.ByteArray.constructor.transcode.length: String
length exceeds maximum characters, cannot convert
> the test that failed this way:
> function () {
>     return (new ByteArray("hello", "utf-8")).toByteString("utf-8",
"utf-16").length === 12;
> }
make: [all] Error 1 (ignored)

We are moving to a #! env commonjs solution.

Requiring /usr/bin/gsr is a little hostile.

This change will also affect the test suite.

Work begun in modules/fs-base.

Howdy,

this in system.jsdoc

/** Load and interpreter a script in the caller's context 
 *  @param      filename        Script filename, either relative or rooted.
 *  @param      thisObj         JS object to bind as global/this during execution. Optional.
 */
function input(filename, thisObj){};


hmmm,  me thinks it's 'include' not 'input' and  the order of arguments in the 
code is (thisObj, 
filename) not (filename, thisObj).

Currently, we have to do this

const _getpwnam = new ffi.CFunction(ffi.pointer, "getpwnam", ffi.pointer);
var pwent = _getpwnam.call("wes");
pwent = ffi.MutableStruct("struct pwent", pwent);
print("Your home directory is " + pwent.pw_dir);

It would be nice to collapse that to

const _getpwnam = new ffi.CFunction(ffi.MutableStruct("struct pwent"),
"getpwnam", ffi.pointer);
print("Your home directory is " + _getpwnam.call("wes").pw_dir);

We can accomplish that like so:

 - All ByteThing casters are of the form Caster(X, Y), where X is a series
of 0 or more specifying arguments and Y is a ByteThing
 - When we call Caster(X) it returns a closure which takes a single argument
 - This closure, when invoked as CasterXClosure(Y) returns the equivalent
of Caster(X,Y)
 - The CFunction constructor will treat a return type-indicator which
happens to be a closure as though it were ffi.pointer
 - When the constructed CFunction is invoked with call, the boxing code
will box up CasterXClosure(Y) instead of Y

We need to tweak GSR to send error reports out to stderr in a more
reasonable way on the FOSS streams.

At the very least, uncaught should always arrive on stderr and warnings
should land there when debugging.

The gsr help list a "    gsr [-z #]", but there is no ref to -z, and using -z 
seems to make gsr 
angry.

$gsr -z 10 -c "xxx + yyy";
gsr: illegal option -- z
Fatal Error in gsr: Invalid option: z



$ gsr

gsr 1.0-pre1 - GPSEE Script Runner for GPSEE 0.2-pre1
Copyright (c) 2007-2009 PageMail, Inc. All Rights Reserved.

As an interpreter: #! gsr {-/*flags*/}
As a command:      gsr [-z #] [-n] <[-c code]|[-f filename]>  <------- HERE
                       {-/*flags*/} {[--] [arg...]}
Command Options:
    -c code     Specifies literal JavaScript code to execute
    -f filename Specifies the filename containing code to run
    -F filename Like -f, but skip shebang if present.
    -h          Display this help
    -n          Engine will load and parse, but not run, the script
    flags       A series of one-character flags which can be used
                in either file interpreter or command mode
    --          Arguments after -- are passed to the script

Valid Flags:
    a - Allow (read-only) access to caller's environment
    C - Disables compiler caching via JSScript XDR serialization
    d - Increase verbosity
    e - Do not limit regexps to n^3 levels of backtracking
    J - Disable nanojit
    S - Disable Strict mode
    R - Load RC file for interpreter (gsr) based on
        script filename.
    U - Disable UTF-8 C string processing
    W - Do not report warnings
    x - Parse <!-- comments --> as E4X tokens

 - Old system module to be rename gpsee
 - New system module per CommonJS

http://wiki.commonjs.org/wiki/System

Features wanted immediately
  - env
  - args
  - print

Features wanted when streams specification complete
  - stdin
  - stdout
  - stderr

For now, making stdin yield ByteStrings should be enough

donny@pacemates:~/gpsee/src/embrace/gpsee-js$ make
cp -f gpsee-js.cpp
/home/donny/gpsee/src/spidermonkey/builds/debug/shell/gpsee-js.cpp
sed -e 's/= js/= gpsee-js/'
</home/donny/gpsee/src/spidermonkey/builds/debug/shell/Makefile
>/home/donny/gpsee/src/spidermonkey/builds/debug/shell/gpsee-js.mk
echo "GARBAGE += gpsee-js.mk gpsee-js.cpp" >>
/home/donny/gpsee/src/spidermonkey/builds/debug/shell/gpsee-js.mk
[ -d "/home/donny/gpsee/src/spidermonkey/builds/debug/shell/.deps" ] ||
mkdir "/home/donny/gpsee/src/spidermonkey/builds/debug/shell/.deps"
cd /home/donny/gpsee/src/spidermonkey/builds/debug/shell && make -f
/home/donny/gpsee/src/spidermonkey/builds/debug/shell/gpsee-js.mk
"CXXFLAGS=-I/usr/local/gpsee/include -I/home/donny/gpsee/tracemonkey/js/src
-DGPSEE -I/home/donny/gpsee/src  -I/usr/local/gpsee/jsapi/include/js
-I/usr/include/nspr -D_GNU_SOURCE" "LDFLAGS=-L/home/donny/gpsee/src 
-L/usr/local/gpsee/jsapi/lib -Wl,-rpath=/usr/local/gpsee/jsapi/lib 
-L/usr/local/gpsee/lib -Wl,-rpath=/usr/local/gpsee/lib  -lgpsee
-L/usr/local/gpsee/jsapi/lib -lmozjs -L/usr/lib -lplds4 -lplc4 -lnspr4
-lpthread -ldl -ldl -lm  -lm -ldl " gpsee-js
make[1]: Entering directory
`/home/donny/gpsee/src/spidermonkey/builds/debug/shell'
gpsee-js.cpp
c++ -o gpsee-js.o -c -I../dist/system_wrappers_js -include
/home/donny/gpsee/tracemonkey/js/src/config/gcc_hidden.h -DEXPORT_JS_API
-DOSTYPE=\"Linux2.6\" -DOSARCH=Linux -I/home/donny/gpsee/tracemonkey/js/src
-I.. -I/home/donny/gpsee/tracemonkey/js/src/shell -I. -I../dist/include
-I../dist/include/nsprpub  -I/usr/include/nspr      -fPIC 
-I/usr/local/gpsee/include -I/home/donny/gpsee/tracemonkey/js/src -DGPSEE
-I/home/donny/gpsee/src  -I/usr/local/gpsee/jsapi/include/js
-I/usr/include/nspr -D_GNU_SOURCE   -DMOZILLA_CLIENT -include
../mozilla-config.h -Wp,-MD,.deps/gpsee-js.pp gpsee-js.cpp
gpsee-js.cpp: In function ‘void panic(const char*)’:
gpsee-js.cpp:84: warning: ‘void panic(const char*)’: visibility attribute
ignored because it
/usr/local/gpsee/include/gpsee.h:304: warning: conflicts with previous
declaration here
In file included from
/home/donny/gpsee/tracemonkey/js/src/nanojit/nanojit.h:300,
                 from /home/donny/gpsee/tracemonkey/js/src/jsbuiltins.h:45,
                 from /home/donny/gpsee/tracemonkey/js/src/shell/js.cpp:58,
                 from gpsee-js.cpp:110:
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h: In member function
‘void nanojit::LIns::staticSanityCheck()’:
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:494: warning: invalid
access to non-static data member ‘nanojit::LInsOp1::ins’ of NULL object
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:494: warning: (perhaps
the ‘offsetof’ macro was used incorrectly)
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:494: warning: invalid
access to non-static data member ‘nanojit::LInsOp1::oprnd_1’ of NULL object
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:494: warning: (perhaps
the ‘offsetof’ macro was used incorrectly)
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:494: warning: invalid
access to non-static data member ‘nanojit::LInsOp2::ins’ of NULL object
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:494: warning: (perhaps
the ‘offsetof’ macro was used incorrectly)
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:494: warning: invalid
access to non-static data member ‘nanojit::LInsOp2::oprnd_1’ of NULL object
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:494: warning: (perhaps
the ‘offsetof’ macro was used incorrectly)
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:496: warning: invalid
access to non-static data member ‘nanojit::LInsOp2::ins’ of NULL object
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:496: warning: (perhaps
the ‘offsetof’ macro was used incorrectly)
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:496: warning: invalid
access to non-static data member ‘nanojit::LInsOp2::oprnd_1’ of NULL object
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:496: warning: (perhaps
the ‘offsetof’ macro was used incorrectly)
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:496: warning: invalid
access to non-static data member ‘nanojit::LInsLd::ins’ of NULL object
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:496: warning: (perhaps
the ‘offsetof’ macro was used incorrectly)
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:496: warning: invalid
access to non-static data member ‘nanojit::LInsLd::oprnd_1’ of NULL object
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:496: warning: (perhaps
the ‘offsetof’ macro was used incorrectly)
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:498: warning: invalid
access to non-static data member ‘nanojit::LInsLd::ins’ of NULL object
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:498: warning: (perhaps
the ‘offsetof’ macro was used incorrectly)
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:498: warning: invalid
access to non-static data member ‘nanojit::LInsLd::oprnd_1’ of NULL object
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:498: warning: (perhaps
the ‘offsetof’ macro was used incorrectly)
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:498: warning: invalid
access to non-static data member ‘nanojit::LInsSti::ins’ of NULL object
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:498: warning: (perhaps
the ‘offsetof’ macro was used incorrectly)
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:498: warning: invalid
access to non-static data member ‘nanojit::LInsSti::oprnd_1’ of NULL object
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:498: warning: (perhaps
the ‘offsetof’ macro was used incorrectly)
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:503: warning: invalid
access to non-static data member ‘nanojit::LInsOp2::ins’ of NULL object
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:503: warning: (perhaps
the ‘offsetof’ macro was used incorrectly)
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:503: warning: invalid
access to non-static data member ‘nanojit::LInsOp2::oprnd_2’ of NULL object
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:503: warning: (perhaps
the ‘offsetof’ macro was used incorrectly)
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:503: warning: invalid
access to non-static data member ‘nanojit::LInsSti::ins’ of NULL object
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:503: warning: (perhaps
the ‘offsetof’ macro was used incorrectly)
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:503: warning: invalid
access to non-static data member ‘nanojit::LInsSti::oprnd_2’ of NULL object
/home/donny/gpsee/tracemonkey/js/src/nanojit/LIR.h:503: warning: (perhaps
the ‘offsetof’ macro was used incorrectly)
c++ -o gpsee-js -I/usr/local/gpsee/include
-I/home/donny/gpsee/tracemonkey/js/src -DGPSEE -I/home/donny/gpsee/src 
-I/usr/local/gpsee/jsapi/include/js -I/usr/include/nspr -D_GNU_SOURCE 
gpsee-js.o    -L/home/donny/gpsee/src  -L/usr/local/gpsee/jsapi/lib
-Wl,-rpath=/usr/local/gpsee/jsapi/lib  -L/usr/local/gpsee/lib
-Wl,-rpath=/usr/local/gpsee/lib  -lgpsee -L/usr/local/gpsee/jsapi/lib
-lmozjs -L/usr/lib -lplds4 -lplc4 -lnspr4 -lpthread -ldl -ldl -lm  -lm -ldl
 -L../dist/bin -L../dist/lib -L/usr/lib -lplds4 -lplc4 -lnspr4 -lpthread
-ldl ../editline/libeditline.a ../libjs_static.a -ldl -lm      
gpsee-js.o: In function `gpseejs_NewObject(JSContext*, JSClass*, JSObject*,
JSObject*)':
gpsee-js.cpp:(.text+0x31e0): undefined reference to `gpsee_getGlobalClass'
gpsee-js.o: In function `gpseejs_NewRuntime(unsigned int)':
gpsee-js.cpp:(.text+0x3474): undefined reference to `environ'
gpsee-js.cpp:(.text+0x3486): undefined reference to `gpsee_createInterpreter'
gpsee-js.o: In function `fatal(char const*)':
gpsee-js.cpp:(.text+0x350a): undefined reference to `gpsee_log'
gpsee-js.o: In function `gpseejs_DestroyRuntime(JSRuntime*)':
gpsee-js.cpp:(.text+0x355f): undefined reference to `gpsee_destroyInterpreter'
gpsee-js.o: In function `split_setup(JSContext*)':
gpsee-js.cpp:(.text+0x4856): undefined reference to `gpsee_initGlobalObject'
gpsee-js.o: In function `EvalInContext(JSContext*, JSObject*, unsigned int,
int*, int*)':
gpsee-js.cpp:(.text+0x4ae7): undefined reference to `gpsee_initGlobalObject'
gpsee-js.cpp:(.text+0x4ba5): undefined reference to `gpsee_initGlobalObject'
gpsee-js.o: In function `Scatter(JSContext*, unsigned int, int*)':
gpsee-js.cpp:(.text+0x516a): undefined reference to `gpsee_initGlobalObject'
gpsee-js.o: In function `.L715':
gpsee-js.cpp:(.text+0x5bbf): undefined reference to `gpsee_initGlobalObject'
gpsee-js.o:gpsee-js.cpp:(.text+0x5fb1): more undefined references to
`gpsee_initGlobalObject' follow
/usr/bin/ld: gpsee-js: hidden symbol `panic' in gpsee-js.o is referenced by DSO
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: ld returned 1 exit status
make[1]: *** [gpsee-js] Error 1
make[1]: Leaving directory
`/home/donny/gpsee/src/spidermonkey/builds/debug/shell'
make: *** [gpsee-js] Error 2

Pre-compiled scripts are not created with the same ownership and
permissions as the uncompiled version. This can lead to a privilege
escalation attack.

Suggested Fix:
 - Try to unlink old .jsc files before creating new (don't need to log failure)
 - Open new .jsc files with mode == (script stat buf->st_mode & 0666)
 - Open new .jsc files with oflag == O_WRONLY | O_CREAT | O_EXCL 
 - fchown the new .jsc file, before writing tokens. fchown failure leads to
unlink() and log message
 - ftruncate the .jsc file, after fchown, before writing tokens 

While the BuildingGypsee docs are "fine", after getting the required packages 
in (like wget, nspr, 
etc), the rest of the steps could be completely automated.   Actually it would 
not be hard 
automating all of it.

I might take a stab at that since I seem to be doing this a lot.

      debug: JS strict warning #110 in shellalike.js at line 576 ch 29 -
function Pipeline_run does not always return a value
      debug: JS strict warning #110 in shellalike.js at line 589 ch 29 -
function Pipeline_run does not always return a value
      debug: JS strict warning #110 in shellalike.js at line 595 ch 29 -
function Pipeline_run does not always return a value
      debug: JS strict warning #110 in shellalike.js at line 597 ch 4 -
function Pipeline_run does not always return a value

Added some configuration for FreeBSD too, such as
looking in /usr/local for iconv and using MAKE=gmake.

Didn't want to use /usr/local/gpsee since that would
violate hier(7), so used /opt/gpsee for prefix instead.

When a Javascript language module is installed, it ought to be in a
directory that doesn't have write access to just anyone (really, no one but
root.) As a result, precompilation cannot be taken advantage of, because
most users that use these modules do not have sufficient privileges to
precompile.

The obvious way to fix this is to precompile all modules when they are
installed.

An interesting alternate solution would be a new SUID GPSEE command
specifically for compiling modules that don't belong to you. This does have
more complex security considerations.

For now, GPSEE logs the problem with SLOG_ERR.

donny@pacemates:~/gpsee/src/tests/gffi$ ./test1.js
JS exception #147 - uncaught exception:
gpsee.module.ca.page.gffi.MutableStruct.cast.type:
gpsee.module.ca.page.gffi.Memory objects are not castable to MutableStruct
Failed loading program module './test1.js': module initialization failed

Currently, modules are not JITable to due to the way they are implemented
and changes in Tracemonkey; a scope object which is not the global object
in JS_ExecuteScript() and friends makes the script code interpreter-only.

Three places we need to examine:

1 - Program Modules (top priority)
2 - Pure-JS modules
3 - Blended modules' JS components

It is possible that JITable program modules executing script code which is
exported from a JS module will be able to JIT the module's script. If this
is the case, 2 and 3 become significantly less interesting and may not be
worth solving.

While pointer values themselves are of dubious value through FFI, being
able to compare pointer values, such as (caddr_t -1) for mmap or simple
equality are necessary.

At the very least, the Memory object must support this; other objects could
be handled via Memory-casting.

 - Ability to cast JS integers to Memory, for values like -1
 - toString should return something with the pointer address in it

So, syntax like this should work:

/************************/
const _mmap = new ffi.CFunction(ffi.pointer, "mmap", ...);
var c = _mmap.call(/*invalid file or something */);

if (c == ffi.Memory(-1))
  throw("mmap failed due to " + _strerror.call(ffi.errno));
/************************/

This might work by coercing both c and ffi.Memory(-1) toString and
comparing. valueOf() might be needed instead of toString(), though, I'm not
100% sure about the JS casting rules. BUT valueOf() cannot be an unadorned
pointer address; it is not big enough on 64-bit systems. 

This also might not be possible with the == operator, which would be
unfortunate. If it's not, we'll simply to have add ffi.addressCompare() or
something.

In prmain() around line 656 there is some old-style error reporting using
JS_ReportPendingException(). I think this should be using
gpsee_reportUncaughtException(), but I want Wes' opinion on reporting
errors that occur in preload scripts.

Really minor issue here. Just need to decide what the exception string
should be and make it so.

donny@pacemates:~/gpsee/src/tests/binary$ ./ByteString.js 
EXCEPTION MISMATCH
V1 gpsee.module.ca.page.binary.ByteString.substr.arguments.0: negative
number is invalid
V2 gpsee.module.ca.page.binary.ByteString.substr.arguments.0.underflow
the test that failed this way:
function (t) {
    t.ex = t.sw(EX_SUBSTR_UNDERFLOW(0));
    (new ByteString("1234")).substr(-1, 3);
}
EXCEPTION MISMATCH
V1 gpsee.module.ca.page.binary.ByteString.substring.arguments.0: negative
number is invalid
V2 gpsee.module.ca.page.binary.ByteString.substring.arguments.0.underflow
the test that failed this way:
function (t) {
    t.ex = t.sw(EX_SUBSTRING_UNDERFLOW(0));
    (new ByteString("underflow")).substring(-10);
}
passed 40 tests
failed 2 tests
FAIL

Binary module tests, and possibly others, need to be tweaked a bit to take
the update to gpsee_throw().

It seems that libdb binaries (libdb.so) and development files (db.h) are
needed to build GFFI. IIRC this is in order to get functions which were
part of the C Standard Library on Solaris. I'm filing this issue because
derek was building GPSEE and ran into this problem, and I think Wes will
know better the right thing to do. In the mean time I just told Derek to
install the right libdb<VERSION>-dev package (he's on Ubuntu.)

** XDR serialization / deserialization should not be a leaky abstraction **

I spotted the problem as follows:
 - upgraded spidermonkey by a significant jump (enough to have new opcodes)
 - ran gsr -f X.js
 - noticed the XDR failure error message, and that the program did not run
 - ran gsr -f X.js again
 - worked fine second time

Until this problem is fixed, here is a quick and dirty way to blow away any
build residue:

cd modules/gffi
rm `hg status * | grep ^? | sed 's/..\(.*\)/\1/'`

On mac/linux at least, a .js source file in the form of NAME.js is compiled and 
saved as .NAMEc

Since it doesn't have a unique suffix, it is hard to write  svn/hg ignore rule, 
and hard to create a 
"make clean" rule.

Perhaps save them as 

NAME.jsc (no leading dot)

or

.gpsee/NAME.jsc

or

NAME.js.xdr?



"Prior Art" -- not that you don't know these examples... just brainstorming

Python stores the object file as NAME.pyo in the same directory as the source.
Emacs stores the .elc file in the same directory as the source, typically under 
a .emacs.d/ 
directory
Subversion -- stores metadata files  in .svn directory

--nickg

We need to figure out another way to provide panic() in libgpsee.so;
providing it from the linking binary while surelynx-convenient is a pain
for the rest of the world.

We either need to come up with a way to define it, or add a weak reference
to a boilerplate panic in libgpsee.so.

Either way, linking-but-not-using libgpsee.so should not cause runtime errors!

gpsee_loadModule() needs to be more careful. we need a special reentrant
mutex. should probably take another reading of the securablemodules spec to
see how to handle cyclical module dependencies before tackling this bug.

Implement the -n option for gsr

Remove the "dd" from "// gpsee: dd" and run from the command line:

gsr -f tests/warnings/program.js

We get a lot of noisy output. There is probably some clash between recent
verbosity reworks and "// gpsee: flags" comments.

Right now, the ByteArray() and ByteString() casting calls create copies if
not casting if either the argument's type or the target type are mutable
((clasp == byteString_clasp) && (hnd->btFlags & bt_immutable))

byteThing_Cast() will need to be looked at soon when COW (issue 13) gets going.

donny@pacemates:~/gpsee/src/tests/gpsee-core$ make
make: gpsee-config: Command not found
../../outside.mk:48: ../../_stream.mk: No such file or directory
make: gpsee-config: Command not found
make: *** No rule to make target `../../_stream.mk'.  Stop.

I have written a mostly complete wrapper around libcurl.  It is very bare-metal 
(almost identical 
to the easy_curl interface), however from that, more "javascript-y" like apis 
could be created 
(using javascript).   I think it's "OK" -- my guess it could use some other 
eyes, but at least the 
boiler plate is created.

I also got a partial collection of common posix system calls all wrapped up -- 
so you can 
pretend you are writing in C

var fd = posix.open("/tmp/foobar", posix.O_CREAT | posix.O_TRUNC | 
posix.O_WONLY);

somewhat in the spirit of python's 
http://docs.python.org/library/os.html#module-os

I don't think it would take too much work to convert them into GPSEE format, 
and use the 
module loader/require stuff.    So the million dollar question is do you want 
them?  I'm happy to 
re-assign all rights and use whatever license.

Strict-mode warnings should be reported by GSR, regardless of whether they
are from preload scripts, -c or program.js.

The test in tests/strict/program.js should trigger a strict-mode warning
about an undeclared variable.

Problem 1: /usr/bin/env does not (portably) pass on shebang options.
Problem 2: flusspferd, gpsee, and other engines do not have compatible
shebang options
Problem 3: it would be nice to have cross-engine scripts 
Solution: Parse all leading comments with gsr; once we have a non-//
comment, or empty // comment, we switch to JS parser

Sample script:

#! /usr/bin/commonjs
// gpsee: zzdd
// flussfperd: -i -Ilib
//
// This comment would be parsed by JS API
function hello()
{
  print("world");
}

Doco bugs on page http://code.google.com/p/gpsee/wiki/Building


My comments are prefixed with "#"

cd
wget ftp://sources.redhat.com/pub/libffi/libffi-3.0.8.tar.gz
tar -zxvf libffi-3.0.8.tar.gz
# NOTE: there is now 3.0.9

mkdir hg
cd hg

hg clone https://code.google.com/p/gpsee
# Wrong.  Should be "hg clone https://gpsee.googlecode.com/hg/ gpsee"

hg clone http://hg.mozilla.org/tracemonkey
# this is "expensive" -- take a look at 
#   http://code.google.com/p/packaging-spidermonkey/
#   and see if we can figure out a way of using a tarball that is
#   already autoconf213'd.  
#   I can start posting snapshots of the trunk as well.

cd gpsee

#
# all below should be "local_config..." not "local-config" (underscore vs. dash)
#
cp local-config.mk.sample local-config.mk
cp spidermonkey/local-config.mk.sample spidermonkey/local-config.mk
cp libffi/local-config.mk.sample libffi/local-config.mk
vi local-config.mk edit suitably, then quit
vi spidermonkey/local-config.mk edit suitably, then quit
vi libffi/local-config.mk edit suitably, then quit

otherwise ok..

$/usr/bin/gsr -c 'print("Hello World");'
Hello World

tada!

I have not even scratched the surface on this one. Just noticing it and
filing a report for now!

donny@pacemates:~$ cat test.js
#!/usr/bin/gsr -zzdd
var bs = new (require('binary').ByteArray);
var bs2 = new bs.constructor([65, 66, 67]);
print(bs.decodeToString('ascii'));
donny@pacemates:~$ ./test.js
Could not create compiler cache
'/usr/local/gpsee/libexec/.binary_module.jsc' (Permission denied)
JS error #137 in test.js at line 4 - out of memory

donny@pacemates:~$ 

 - Add COW capability flag

 - Add boiler plate to copy-cast ByteThings
   - Immutable to Immutable: copy buffer pointer, length, memoryOwner        
   - Mutable(+COW) to Mutable(+COW): copy buffer pointer, length,
memoryOwner  (note: Mutable cannot alter length unless memoryOwner or friend)
   - Immutable to Mutable: duplicate buffer
   - Immutable to Mutable+COW: NULL buffer pointer, copy length & memoryOwner
   - Mutable+COW to Immutable: Complicated, see below
   - Mutable to Immutable: duplicate buffer

 - Add read/write-intercept boilerplate 
   - Mutable+COW setters to duplicate buffer
   - Mutable+COW getters to access memoryOwner->buffer rather than hnd->buffer

 - Add boilerplate to cast from Mutable+COW->Immutable
   - Must cast first to a generic intermediate type, ImmutableMemory
   - ImmutableMemory instance steals memoryOwner status and finalizer,
adjusting Mutable+COW's handle accordingly
   - ImmutableMemory holds (parent?) reference to original memoryOwner, to
prevent finalizer from getting collected (DSO module case)
   - Now we can do Immutable->Immutable cast

I pretended I was a complete n00b using Mac OS X 10.6 to install gpsee 
using
http://code.google.com/p/gpsee/wiki/Building with a fresh MacPorts 
install.

There is no mention of how to install mercurial in the doco.

This "sudo port install mercurial" is a nightmare as it appears into install 
every X library and takes 45 minutes.

Recommend: download hg directly from

http://mercurial.selenic.com/

Afterwards, start a new terminal to continue the install (new terminal gets 
fresh path information)

I may have some issues to raise with CJS about the spec, but as it stands
now, GPSEE violates (in spirit only, in some cases) 1.6.1 through 1.6.4:

1.6.1:  The "paths" property must not exist in "sandbox" (a secured module
system).

We have no option to remove support for require.paths, and we don't
pre-create require.paths, thus removing the possibility feature-test.

1.6.2: The "paths" attribute must be referentially identical in all modules.

We actually define a "require" property in the global scope and then a new
one in each module scope, and when we are searching module paths we
actually do a property lookup to find the "require" property of the module
scope, and then in turn, the "paths" property of that.

1.6.3: Replacing the "paths" object with an alternate object may have no
effect.

This is strongly related to the previous one. The common pattern seems bent
toward efficiency: the "host" (GPSEE) usually holds its own references to
the objects it will use, thus simply changing the "paths" property of
"require" does not change the internal reference to the original, thus
invalidating the expression "require.paths."

Javascript follows the same pattern elsewhere: You can change the "String"
property of the global scope to another value, say with this expresion:
"String = Number." This does not, however, change the "constructor"
property of the value of a string expression.

1.6.4: If the "paths" attribute exists, in-place modification of the
contents of "paths" must be reflected by corresponding module search behavior. 

Assuming "in-place modification" means modifying the contents of the Array
object and not changing the "paths" property of "require," we sort of
comply, but 1.6.4 sort of exists to back up 1.6.3.

http://wiki.commonjs.org/wiki/Modules/1.1

Conclusion:

Each GPSEE interpreter needs an internal, rooted reference to its own
"require.paths" Array object, and the extra property look-ups on each
individual module scope object need to be bypassed.

Sorry, being pedantic on the doco

wget is not installed on Mac OS X and it's referenced later to get libffi.

might as well add

sudo port install wget

to the Mac OS section


Maybe I also recommend the doco not to use bulleted items, and instead 
use preformatted text, so someone can cut-n-paste the script.

When a GPSEE program module stops running due to an uncaught exception, GSR
should dump the error stack to stderr.

This feature should either be separately enableable with a letter-flag, or
perhaps only on at high debug levels (gpsee_verbosity(0) > 2?)

One way to implement this without writing stack-printing code in C would be
to have gsr detect the uncaught exception state, send it to the error
reporter as usual, then print the stack using a javascript function
included with a utility module, by doing something like

JS_TurnOffTheErrorReporter();
JS_EvaluateScript("require('themodule').dumpStack()");

Make sure the fix for this gets into the repository!

Refactor the binary module using new casting boilerplate code (see issue
13). Methods, like slice or splice or should be implemented as either
Immutable or Mutable+COW casts (with tweaked buffer offsets as appropriate.

We should be able to implement code like this with no buffer copies:

function yieldLines(stream)
{
  var buf = new Memory(1024);

  _fgets.call(buf, buf.size, stream);
  buf = ByteArray(buf);
  buf.length = _strlen.call(buf);

  if (baBuf[baBuf.length - 1] == '\n'.charCodeAt(0))
    baBuf.length--;

  yield ByteString(baBuf);
}

Note that ByteString and ByteArray are friends, and thus ByteString will
correctly inherit the ByteArray length, even though the size of the
original Memory instance is probably much larger.

I'm not quite sure what is going on here.  Everything exists.  It's like bash 
doesn't like the pipe.

{{{
spidermonkey nickg$ make build
[ -d /Users/nickg/dev/gpsee2/hg/gpsee/spidermonkey/builds/release ] || 
mkdir -p 
/Users/nickg/dev/gpsee2/hg/gpsee/spidermonkey/builds/release
cd /Users/nickg/dev/gpsee2/hg/gpsee/spidermonkey/builds/release && 
CC="/bin/@echo "@executable_path/libmozjs.dylib " | 
/Users/nickg/dev/gpsee2/hg/gpsee/darwin-ccld.sh gcc" CXX="/bin/g++" 
/Users/nickg/dev/gpsee2/hg/tracemonkey/js/src/configure --with-
system-nspr --enable-threadsafe --prefix=/opt/local/gpsee/jsapi --
libdir=/opt/local/gpsee/jsapi/lib --
includedir=/opt/local/gpsee/jsapi/include
/bin/sh:  | /Users/nickg/dev/gpsee2/hg/gpsee/darwin-ccld.sh gcc: No 
such file or directory
make: *** 
[/Users/nickg/dev/gpsee2/hg/gpsee/spidermonkey/builds/release/Makefi
le] Error 127
}}}

http://wiki.commonjs.org/wiki/Modules/1.1

 - module.id         -> abs path
 - require(moduleid) -> abs path allowed if already loaded
 - require.paths

We probably want, in the future, the ability to set the exit status code of
the program, and tailor the error message very exactly. Right now a special
exception seems the way to do this.

FFI auto-finalizers are needed to insure that non-JS resources are properly
released when JS objects are finalized, regardless of whether or not the JS
programmer remembered to release the resources or not.

FFI finalizers are tricky, because you can't call into JS from a GC
finalizer. This means that all the "calculations" needed to perform
finalization must be done at declaration time.  This severely limits the
utility of FFI finalization, but that's okay. We really only want to use
them for things like cleaning up file descriptors and freeing memory.

FFI finalizers can be invoke two ways: programmatically, via ffi.destroy()
and via JS garbage collection. When invoked from the GC, a gpsee_log()
message should be generated at the SLOG_NOTICE level mentioning the name of
the finalizer.

A sample program using this facility would look something like this:

const ffi = require("gffi");
const libc = new Library(rtldDefault);
const _fopen = new libc.CFunction(ffi.pointer, "fopen", ffi.pointer,
ffi.pointer):
const _fclose = new libc.CFunction(ffi.pointer);

function openFile(filename, mode)
{
  var file = _fopen.call(filename, mode);

  if (file)
    file.finalizer = _fclose.finalCall(file);

  return file;
}

function fclose(file)
{
  ffi.destroy(file);
}

In this example, _fclose.finalCall() sets up the FFI function call
arguments just like _fclose.call() would, except instead of invoking the
ffi call, it stores the call details in a CFinalizer object's private
handle.  When the CFinalizer is itself finalized, it will then invoke the
ffi call (without suspending the request!) and clean up the memory in the
private handle.

Note that the CFinalizer must copy ALL details into its own private storage
and cannot rely on /anything/ set up by CFunction. This is because the
CFunction could be finalized long before the CFinalizer.

Something like this

/opt/local/gpsee/libexec/gffi.js:70: too much recursion
   ()                  in /opt/local/gpsee/libexec/gffi.js        at 70 
   ()                  in /opt/local/gpsee/libexec/gffi.js        at 70 
   [last line repeated 123 more times]
   ([object Object])   in /opt/local/gpsee/libexec/gffi.js        at 74 
   mmap("/etc/passwd") in /Users/wes/hg/gpsee/tests/gffi/test2.js at 44 
                       in /Users/wes/hg/gpsee/tests/gffi/test2.js at 53 

Spotted by ondras via code review.

Looks like iconv_close is not called unless there is an error.