CLaSH runs out of memory when translating a simple 96-way case... about clash-compiler HOT 8 CLOSED

It's odd that it says out of memory for only 2 Mb of memory. How much memory does the compiler use according to the operating system?

Regardless, the compilation system that the CLaSH compiler uses probably belongs to an exponential complexity class for certain use cases. So I won't know how easy it will be to fix this bug. Especially as I plan to completely overhaul the system within the next 6 months.

from clash-compiler.

I have 12G laptop, and it visibly goes up to the limits. So 2MB is just an allocation chunk in this case. Sure at least >=8GB of memory is used before the crash.

from clash-compiler.

Is there any way to avoid exponential blowup, possibly by decreasing optimization?
This case expression is a simple ROM, so there is nothing to optimize, and nothing to keep in memory.

from clash-compiler.

Perhaps, but that's not how the compiler works. It tries to reach a normal form, and can only translate that normal form to VHDL/Verilog. The only true optimisation it does right now is common subexpression elimination.

As a work around, what you could do, is replace the inner case-statements by Vec indexing, e.g.:

glyph 33 scanLine =  (!! scanLine) --'!'
      (0b0000001111000000
    :> 0b0000011111100000
    :> 0b0000111111110000
    :> 0b0000111111110000
    :> 0b0000111111110000
    :> 0b0000111111110000
    :> 0b0000011111100000
    :> 0b0000001111000000
    :> 0b0000001111000000
    :> 0b0000001111000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> 0b0000001111000000
    :> 0b0000001111000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> Nil)

from clash-compiler.

Actaully, if you turn it into code like:

glyph :: Unsigned 8 -> Unsigned 3 -> Unsigned 8

glyph c scanLine = (!! scanLine) $ (!! (c - 32))  --' '
    ( (0b0000000000000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> Nil)

    :> --'!'
      (0b0000001111000000
    :> 0b0000011111100000
    :> 0b0000111111110000
    :> 0b0000111111110000
    :> 0b0000111111110000
    :> 0b0000111111110000
    :> 0b0000011111100000
    :> 0b0000001111000000
    :> 0b0000001111000000
    :> 0b0000001111000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> 0b0000001111000000
    :> 0b0000001111000000
    :> 0b0000000000000000
    :> 0b0000000000000000
    :> Nil)

....

    :> Nil)

It has an even likelier chance that it won't use too much memory. Because the then the vector literal is one BIG constant, which is normal form. However.... it will still take a long time (45 seconds and 900 Mb on my machine) because every literal 0b0000001111000000 is internally seen as fromInteger $dUnsigned 0b0000001111000000, where $dUnsigned is a so-called class dictionary: a record data-type containing the functions of the class instance, in this case the functions of the Unsigned instance for the Num type-class.

from clash-compiler.

Workaround works great so far, but it would be nicer to treat the other case. (Multiway cases are going a long way in some projects.)

from clash-compiler.

As of 75a34ec, CLaSH no longer runs out of memory for this test-case... as long as you have 4GB or more...

Also note, that the GHC front-end, which CLaSH uses to go from Haskell to it's internal Core representation just creates really bad code for the given example. However, if we change the code to:

{-# LANGUAGE BinaryLiterals #-}
{-# LANGUAGE DataKinds      #-}
module Font(glyph, topEntity) where
import CLaSH.Prelude

topEntity = glyph

glyph :: Unsigned 8 -> Unsigned 3 -> Unsigned 8
glyph x y = glyph' (fromEnum x) (fromEnum y)

glyph' :: Int -> Int -> Unsigned 8
glyph' 32 scanLine =case scanLine of --' '
    0 -> 0b0000000000000000

We get much, MUCH, better case-statements out of GHC, and CLaSH finishes in less than 8s and using less than 600 MB.

from clash-compiler.

CLaSH now uses less than 800 MB, and finishes within 20s on my machine. I consider this issue solved.

from clash-compiler.