"Out of memory" error on Ubuntu 8.10 with 8Gb ram. about ps3mediaserver HOT 12 CLOSED

It happens only when you open two files consecutively ?

No no, I meant it happens with either file type.

After trying a few more filetypes AVI with transcoding no longer works either.

Almost as if a memory buffer has run out of memory but is not reset afterwards.

The error I get in the terminal is this: "[Thread-87] TRACE 22:37:32.933 FATAL 
ERROR:
OutOfMemory / dumping stats"

check the debug.log file, you should have more infos on this dump, with stats 
(free 
memory, used memory, etc.)

There is quite a lot of information in the debug.log file. Can't really seem to 
find
any relavant information regarding out of memory.

search the "OutOfMemory" word, it should be around

Sorry. Found it.

"[Thread-101] TRACE 22:57:46.850 FATAL ERROR: OutOfMemory / dumping stats
[Thread-101] DEBUG 22:57:46.850 freeMemory: 526584608
[Thread-101] DEBUG 22:57:46.851 totalMemory: 582877184
[Thread-101] DEBUG 22:57:46.851 maxMemory: 715849728
[Thread-12] INFO  22:57:46.854 Sending BYEBYE..."

what JDK vendor/version are you using ? also, you can try reducing the 
transcode 
buffer size (even if the default value of 400 mb should be ok, according to 
your log)

When I first installed PMS I assumed more cache would be beneficial, I was 
wrong.
Setting the buffer back to 400 from the 650 I changed it to got rid of the out 
of
memory error. 

Now I get this error on MKVs only:
"[mencoder] DEBUG 23:08:40.252 EOF
[mencoder] TRACE 23:08:40.252 Process mencoder has a return code of 139! Maybe 
an
error occured... check the log file
[mencoder] INFO  23:08:40.252 Stopping process: mkfifo"

Should I file it as a new Issue?

650mb should work actually. But the thing is your maxmemory value is around 
700mb, or 
with the launch script, it should be close to 770mb (don't know why it's not), 
hence 
your error

"Should I file it as a new Issue?"

yep, it's better to do this