aliakhtar / spock Goto Github PK

additional classes to be filtered:
groovy.lang.MetaClassImpl
groovy.lang.MetaMethod

feature method names should be fixed up in closures too:
org.spockframework.smoke.OldValues$___feature4_closure1.doCall

Example:

{{{
expect:
x == y

where:
x << [1, 2, 3]
y << [1, 2]
}}}

Currently this passes (number of iterations is minimum number of values 
returned by a data 
provider). Alternatively, we might issue a warning, insert nulls once a data 
provider runs out of 
values, or let the feature / last iteration fail.

Pros of the current approach:
 * Might be convenient if external data is involved (however this is speculation)
 * In line with the tolerant nature of scripting languages

Cons of the current approach:
 * No feedback if data is missing accidentally

Ideas:
 - Groovy script that imports @Speck is considered a specification
 - Script class is annotated with @Speck and @RunWith behind the scenes
 - Make sure that the specification is also run when the script's main method is invoked (important 
for tools that fail to identify the script as a specification (e.g. IDEs))

Selector can then be used in JUnit task to run all Specks regardless of
their name (cf. find-specks Maven goal).

Open question: Should we search for classes annotated with @Speck, classes
annotated with @RunWith, classes with at least one @Test method, or classes
satisfying any or a combination of these criteria?

Should have same effect as "def e = thrown(Throwable)".

Nothing more to say, really.

java.lang.NullPointerException
    at
org.spockframework.runtime.JUnitSupervisor.getFeatureDescription(JUnitSupervisor
.java:90)
    at org.spockframework.runtime.JUnitSupervisor.error(JUnitSupervisor.java:61)
    at
org.spockframework.runtime.SpeckInfoBaseRunner.invokeRaw(SpeckInfoBaseRunner.jav
a:181)
    at
org.spockframework.runtime.SpeckInfoBaseRunner.invoke(SpeckInfoBaseRunner.java:1
61)
    at
org.spockframework.runtime.SpeckInfoBaseRunner.invokeSetupSpeck(SpeckInfoBaseRun
ner.java:98)
    at
org.spockframework.runtime.SpeckInfoBaseRunner.run(SpeckInfoBaseRunner.java:78)
    at spock.lang.Sputnik.run(Sputnik.java:47)
    at com.intellij.rt.junit4.Junit4ClassSuite.run(Junit4ClassSuite.java:99)
    at junit.textui.TestRunner.doRun(TestRunner.java:116)
    at
com.intellij.rt.execution.junit.IdeaTestRunner.doRun(IdeaTestRunner.java:94)
    at junit.textui.TestRunner.doRun(TestRunner.java:109)
    at
com.intellij.rt.execution.junit.IdeaTestRunner.startRunnerWithArgs(IdeaTestRunne
r.java:22)
    at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter
.java:118)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:40)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.jav
a:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:90)

...because setup() is declared as "void setup()" in class Specification.

Too few invocations: Completely synthetic stack trace(s) that point(s) to all 
affected interactions
Too many invocations: Truncate after call to matching method, replace name of 
proxy class (e.g. 
$Proxy9)

Build should be upgraded to Gradle 0.6.

Make it possible to impose a partial order on interactions.

Example:

when: // ...
then: 1 * x.foo(); 1 * y.foo()
then: 1 * x.bar()

1 * x.foo() and 1 * y.foo() are unordered with respect to each other, but 1
* x.bar() comes after them (the exact meaning of which still has to be
worked out).

class Foo extends Specification {
  def x = Mock(List) // SyntaxException: Mock objects can only be created inside a Speck 

  def foo() { expect: true }
}

Stacktraces presented to the user should have all internal MOP calls
removed (no more, no less).

For example, multi-line values should be displayed with correct indentation.

I hit the case where I need a cleanup for a certain test. It would be nice if 
there was an 
after/cleanup clause…

def t() {
  setup:
    // swap in fake streams for System.out and System.err

  then:
    // do some stuff that produces output

  expect:
    // what was written to System.out was correct

  cleanup: // maybe ‘after’ as a synonym?
    // restore real System.out and System.err
}

Add a base class for specification, either as an alternative or a replacement 
for the current way to 
designate specifications. 

Old:
{{{
import org.junit.runner.RunWith
import spock.lang.Speck
import static spock.lang.Predef.*

@Speck
@RunWith(Sputnik)
class Foo {}
}}}

New:
{{{
import spock.lang.Specification

class Foo extends Specification {}
}}}

Plus:
 * shorter and nicer on the eye
 * saves static imports (static imports are difficult for IDEs to insert automatically, and "bind" 
stronger in Groovy than they do in Java)

Minus:
 * might inspire Spock extensions to use inheritance (two such extensions cannot be used 
together)
 * specification that inherits from specification in different compilation unit might be hard to 
recognize for Spock compiler and test runners
 * reputation as anti-pattern since JUnit switched to annotations

Should we do this even if IDEs can't cope with it?

if necessary, adapt SpeckInfoParameterizedRunner.estimateNumIterations() 
accordingly

@Speck
@RunWith(Sputnik)
class GlobalInteractions {
  @FailsWith(InteractionNotSatisfiedError)
  def "should be verified"() {
    List list = Mock()
    1 * list.size()
  }
}

alternatively, disallow global required interactions (compile-time error)

This is a design issue not an implementation issue.

Example:
{{{
when:
foo.bar("hi")
foo.bar("hi")

then:
1 * foo.bar("hi")
1 * foo.bar(_)
}}}

Currently this fails because the first interaction matches too often (twice
instead of once). But it might be reasonable to give the second interaction
a chance to match the second invocation before letting the first
interaction fail.
A related question is what should happen if the second interaction were global.

Invoking regular methods on such mocks should already work, but invoking
GDK and dynamic methods is not yet supported.
What should we do if a GDK/dynamic method is invoked that doesn't match any
interaction? Should we be graceful, or should we issue an error?

Currently, all mocks are lenient. It would be nice to also have mocks that
- are strict
- have other default return values (empty collection, dummy mock, etc.)

Syntax suggestions:
SMock(), StrictMock(), Mock(strict: true), Mock("strict")
IMock(), IntelliMock(), Mock(intelligent: true), Mock("intelligent")

It might also make sense to have mocks that combine several of these
properties.

Exception subclasses should reveal information beyond just the error message.

expect: false
cleanup: throw new Exception()

ConditionNotSatisfiedError should prevail

assert result == ""

"" is currently rendered as java.lang.String@hash, but should be rendered as "".

I have an enhancement request to make tests run in parallel. I like Spock
so much, I'm using it to run some validation tests that run over a large
dataset. I use the data provider way and @Unroll to run the tests as unique
tests. They work great, except they run serially and I'd like to use up all
the cores that the machine has (> 4).

What steps will reproduce the problem?
0. Use IntelliJ IDEA EAP build (I don't think this is IntelliJ issue though)
1. Create a new Groovy Script with the HelloSpock example
2. Set two breakpoints: at name<< and size<< 
3. Debug the script, wait a second and resume at each breakpoint. 
4. Error occurs for me roughly 1 out of 4 times. 

What is the expected output? What do you see instead?
I expect test to pass without a JVM error
The test fails with a JVM error: 
FATAL ERROR in native method: JDWP on getting class status,
jvmtiError=JVMTI_ERROR_WRONG_PHASE(112)
JDWP exit error JVMTI_ERROR_WRONG_PHASE(112): on getting class status
[../././rc/hare/ack/til.c:1265]

or 

FATAL ERROR in native method: JDWP Can't allocate jvmti memory,
jvmtiError=JVMTI_ERROR_INVALID_ENVIRONMENT(116)
JDWP exit error JVMTI_ERROR_WRONG_PHASE(112): on getting class status
[../././rc/hare/ack/til.c:1265]
JDWP exit error JVMTI_ERROR_INVALID_ENVIRONMENT(116): Can't allocate jvmti
memory [../././rc/hare/ack/til.c:1779]
ERROR: JDWP unable to dispose of JVMTI environment:
JVMTI_ERROR_INVALID_ENVIRONMENT(116)


What version of the product are you using? On what operating system?
Spock 0.1
Win XP
Junit 4.4
IDEA 9 EAP
Groovy 1.7 snapshot build today
Java 1.6.04

Please provide any additional information below.

Hopefully this is an IDEA or Java issue. Otherwise, I'm sorry. This one
doesn't look fun to debug. 

Should we provide them? How should they behave?

Depends on http://jira.codehaus.org/browse/GROOVY-3552

It's important for Spock that Groovy allows the following within a method:

expect: ...
and: ...
and ...

when: ...
then: ...
then: ...

Although both Java and Groovy do allow this, it should be made explicit with a 
unit test s.t it isn't 
changed unintentionally.

In the wiki or elsewhere.

Thanks.

via RunNotifier.fireTestIgnored()

Example:
{{{
when: stack.push("elem")
then: stack.size() == old stack.size() + 1
}}

Here, Predef.old(_expr_) refers to the value of _expr_ before the when
block was entered.

...that allows to jump to affected feature method

Features: 
 * Start up and shut down Tapestry modules
 * Inject service, symbols, and values into specifications

expect: true
where: [a, a] << [1, 2] // a occurs twice

Similar to http://groovyconsole.appspot.com/
Could fork http://github.com/glaforge/groovywebconsole/tree/master

@Ignore works on method level, but not on class level.

For example, "grails create-unit-spec" might create something like:

package <package name automatically inserted here>

import spock.lang.*
import grails.plugin.spock.*

class MySpec extends Specification {
  def "feature"() {

  }
}

It should be possible to call void methods from expect and when blocks. Such 
method calls should 
not be considered implicit conditions, just as variable declarations aren't 
considered implicit 
conditions.

Usages:
 * println "debug message"
 * conditionInHelperMethod()
 * assertionApiCall()

Either print just the message, or print the message and the rendered condition.

This snippet works ok:

then:
def e = thrown(HostException)
e.message == "No active testcase"

while following doesn't catch the exception:

then:
thrown(HostException).message == "No active testcase"

It could be good to support such shortcut definition.

Either disallow or support them. See specification 
StaticMethodsInSpecifications.

The following blows up:

expect:
new Person(name: "fred")

Should also verify that method calls with named parameters work as expected.

Pass an invocation object to result blocks instead of just the invocation's 
arguments. Invocation 
object could, for example, provide access to the invocation count:

setup:
Foo foo = Mock()
foo.calculate() >> { it.count * 3 }

It should be possible to apply @Timeout to a whole Speck
When applied to a feature method, @Timeout should (at least optionally) include 
all iterations, 
setup(), and cleanup()

This feature is a great challenge for our extension mechanism.

Example from class FeatureMethodParameters:

def "data value type can not be coerced to parameter type"(x, ClassLoader y) {
  expect:
  x == y

  where:
  x << [1, 2]
  y << [1, 2]
}

Stack trace should allow easy navigation to line y << [1, 2].

What steps will reproduce the problem?
1. Open GroovyConsole
2. Add spock/junit jars thru "add jar to classpath" menu item
3. Try to run HelloSpock

What is the expected output? What do you see instead?
I expected the code to at least compile. Instead I got a stack trace: 
Exception thrown: null line detected; lineNum=11, expr=(name.size() == size)

What version of the product are you using? On what operating system?
Win XP . Spock 0.1

Please provide any additional information below.
I don't actually care about running the specks in GroovyConsole, I just
wanted to use the ASTViewer to see what AST was being created so that I
could understand how Spock workd. 

i would expect to be able to do this:

def "my test"() {
  given:
      def a = new Apple()
      def o = new Orange()

  when:
      fruit = f
  then: 
      <something>
  where:
       f << [a, o, a, o, a, o]  
}

But I get MissingPropertyException for a & o.  I also tried declaring a & o in 
the where block - same issue.  Declaring them outside the 
scope of the method works but is not ideal.  I ended up unrolling the where 
into several when/then blocks.

I see that the documentation for the where block is coming (at light speed :) 
so maybe this is as designed.

Thanks

"Speck" is a cute word play, but we have decided that it doesn't work out for 
us.