coverband / spitfire Goto Github PK

I'm including a patch that runs a speed test on the compile for benchmarking.

You can use the attached template to test this out. When you attempt to 
compile, the compiler 
chokes on the last close token (see attached stacktrace.

Adding a space (whitespace) after the token yields the same error.

Adding a newline will work, and cause the template to compile.

1.) Create a spitfire file with the following content:

#def foo($s)
$s
#end def

#def bar($s)
$s
#end def

<script type="text/javascript">
var bar={ cat: $foo($bar("dog")) };
</script>

2.) Run SpitFire on it.

It produces the following error:
Trying to find START_PLACEHOLDER on line 11:
>  var bar={ cat: $foo($bar("dog")) };
>                                  ^
List of nearby tokens: 
  (@106)  PLACEHOLDER_OPEN_PAREN  =  u'('
  (@107)  START_PLACEHOLDER  =  u'$'
  (@108)  ID  =  u'bar'
  (@111)  PLACEHOLDER_OPEN_PAREN  =  u'('
  (@112)  '"'  =  u'"'
  (@113)  DOUBLE_QUOTE_STR  =  u'dog'
  (@116)  '"'  =  u'"'
  (@117)  CLOSE_PAREN  =  u')'
  (@118)  CLOSE_PAREN  =  u')'
  (@119)  PLACEHOLDER_CLOSE_BRACE  =  u' }'

3.) I expect it to produce the following:
<script type="text/javascript">
var bar={ cat: "dog" };
</script>

Source:

#for $x in $foobar
    #set $broken_variable = 'test'
    $broken_variable
#end for

With -O4, this compiles to:

    _fphB50896DF = _self_filter_function(broken_variable) # << use before 
definition
    for x in _rph_foobar:
      broken_variable = u'test'

The stop-gap fix is (checked in r115) is to not hoist any variables that have 
other dependencies in the loop. A warning is emitted instead. Long term 
fix should be to hoist both the loop invariant variable and all invariant 
dependencies.

What steps will reproduce the problem?
1. create a simple template that includes an "or" operator with a placeholder 
on each side
2. compile it with -O4


What is the expected output? What do you see instead?

If the first argument is True, the second should not be evaluated.  This is not 
the case.

There is code in optimizer.py in AnalyzeBinOp to not hoist if the operator is 
"and", and the same 
logic should apply to "or".

Source code:

#set $a = 'foo'
#set $b = $a

Compiling with -Xcheetah-cheats (no optimization) produces:

    _self_search_list = self.search_list + [_globals]
    _buffer_write(u'\n')
    a = u'foo'
    b = resolve_placeholder(_self_search_list, 'a')

This always fails during runtime with: NotFound: cannot find 'a'

This is probably because locals() is not checked by resolve_placeholder(). 
_self_search_list does not contain locals.

The following two cases compile and run without issues:

1. Compiling without -Xcheetah-cheats produces:

    a = u'foo'
    b = resolve_placeholder('a', template=self, local_vars=locals(), 
global_vars=_globals)


2. Compiling with -Xcheetah-cheats and -O2 produces:

    _self_search_list = self.search_list + [_globals]
    _buffer_write(u'\n')
    a = u'foo'
    b = a

The following case also reproduces the runtime error (compiled with -
Xcheetah-cheats and -O2):

#if True
 #set $a = 'foo'
#end if
#set $b = $a

Summary: If cheetah-cheats is enabled and the optimizer doesn't hit the 
resolve_placeholder node, there is a runtime error.

With both translate1 and translate2 in the function registry:

Source:

#set $a = 'a'
$translate1(a)
$translate2(123)

Compiled:

## translate1 ok:

 25     a = u'a'
 26     _buffer_write('%s' % _self_filter_function(translate1(a), translate1))
 27     _buffer_write(u'\n')
 28     if _cphAC9C6496 is None:

## translate2 not optmized: uses resolve_placeholder()

 29       _globals['_cphAC9C6496'] = 
_self_filter_function(resolve_placeholder('translate2', template=self, 
local_vars=locals(), global_vars=_globals)(123), 
resolve_placeholder('translate2', template=self, local_vars=locals(), 
global_vars=_globals))

This is because CacheNode doesn't get visited by the optimizer.

trying to encode unicode chars with default filter throws unicode decode 
exception.

'test.spt' / 'test_utf8.py' exercises the traceback. 'patch' include an attempt 
at a solution.

In my application (with a deeply nested hierarchy of templates), this patch 
provides a 20% and up improvement to compile times by caching pattern lists, 
and speeding up restriction checks.  Testing speed against the included tests 
provides a more modest 5% speedup.