There is a lonely SortTest.java in examples/locoGP

New tests are welcome :)

think about re-structuring to pull out common parts into one place for easier manipulation, e.g., have a fitness folder along the lines of edits folder

To support the final assessment of the impact of patches, it would be good to provide a bit of help to the users. Most of the following thoughts are around an optional testing phase that happens immediately after the actual optimisation phase (the evolution). This is a bit similar to what is implemented in the automated algorithm configurator irace.

Some random thoughts, in no particular order:

1. For the functional properties (e.g. "number of tests passed"), the final test phase provides an easy-to-use overview. This becomes especially important in case Gin works with more than just a single solution at a time in the future, e.g. by considering trade-offs or populations or multiple solutions that all satisfy the functional requirements are found. As functional tests are typically deterministic in their outcome, no special handling of noise is necessary.
   This also applies to measurements of deviation from the oracle (think of a target vector from which the actual calculations of the patched code deviate).
2. However, some measurements are not deterministic: runtime, memory consumption (debatable), energy consumption, ... For these cases, in order to achieve meaningful and trustworthy observations that can be used/published, it would be good if tests can be run repeatedly. In irace, I have to specify this for each test (== problem instance) individually. As tests can vary in the resources needed, we can go this way or simply let the user provide a global setting, e.g., "do 100 repetitions". If Gin does the repetitions, then the individual results should be piped into a CSV file, because you never know what kind of statistical analyses the user wants to run.
3. Statistics: if we do the processing on Gin's end, then we need to decide what statistics to provide. min/max/average/median/stdev/quartiles might be a good start - and already more than what the average user needs. The alternative is to let the user handle this, which is how irace handles it. It does not know anything about repetitions, but just runs the configurations on the provided test instances.
4. Testing & side-effects & plenty of noise: as some tests can be affected by noise and can have side-effects (e.g. due to warming up of the system, due to side-effects, due to dynamic overclocking of single cores, ...), and in case Gin is to do the repetitions, then Gin might have to execute patches A/B/C/D alternatingly, in order to bias the measurements as little as possible and to average out noise: ABCDABCDABCD...
   For two patches one might actually want to do ABABAB...BABABA, but it is not clear to me how to extend this to larger sets of patches - therefore the suggested round-robin.

A few of these thoughts have consequences for Gin's internal survival mechanism (read: which solution to pick in case of a local search), but that's a different story...

We should add a new test case that compiles all the examples in ./examples/ and runs their tests against them.

The examples are excluded from the main source tree to ensure dynamic class loading works correctly, but this also means they're not being properly built and tested.

An alternative would be to have gradle build and test them. Unsure what's best.

LocalSearch.neighbour() can generate a patch with no Edits in it, by deleting an edit when patch size == 1. Not sure if this is intentional or not!

Not sure what the reason is yet. If a method has signature:
"methodname Map<List, Integer>> methodname(int, Map<List, Integer>)"
the following is thrown when trying to setup SourceFile:
"gin.SourceFile.getTargetMethodRootNodesFromCU() ERROR: Couldn't find these methods:.."
I narrowed it down to gin.misc.FullyQualifiedNames.getFQName(MethodDeclaration m) which returns the wrong FQName, namely:
"Integer>methodname(int,Map<List,Integer>)"
instead of:
"methodname(int,Map<List,Integer>)"

similar issue occurs for the following method signature:
"List<Pair<Integer,Integer>> methodname(int , int , int)"
returns:
"Integer>>methodname(int,int,int)"
instead of:
"methodname(int,int,int)"

It would be nice to have a tutorial :) e.g. if you have a new edit/search how can you experiment with Gin? also a hello world (trianlge) example etc.

Looks like a regression after commit 60262c1 that I didn't spot when checking the PR.

Should be easily fixed. Working on it now.

Example:

Markuss-MBP:git-gin wagner$ java -jar build/gin.jar examples/locoGP/SortBubbleDouble.java
Optimising source file: examples/locoGP/SortBubbleDouble.java

Initial execution time: 5377250.0 (ns)

Step 1 | COPY 11 -> 1:0 |*** New best *** Time: 4127871.0(ns)
Step 2 |Time: 5673439.0
Step 3 |Time: 4937142.0
Step 4 | COPY 11 -> 1:0 | DEL 12 |*** New best *** Time: 2406271.0(ns)
Step 5 | COPY 11 -> 1:0 | DEL 12 | COPY 11 -> 5:0 |Time: 4719200.0
Step 6 | DEL 12 |Time: 4197840.0
Step 7 | COPY 11 -> 1:0 | DEL 12 | DEL 15 |Patch invalid
Step 8 | COPY 11 -> 1:0 |Time: 5805536.0
Step 9 | COPY 11 -> 1:0 |Time: 3091739.0
Step 10 | COPY 11 -> 1:0 | DEL 12 | DEL 4 |Patch invalid
Step 11 | COPY 11 -> 1:0 | DEL 12 | MOVE 8 -> 3:1 |Time: 4251221.0
Step 12 | DEL 12 |Time: 2804988.0
Step 13 | COPY 11 -> 1:0 | DEL 12 | MOVE 19 -> 0:2 |Time: 3854282.0
Step 14 | DEL 12 |Time: 2794879.0
Step 15 | DEL 12 |Time: 2700009.0
Step 16 | COPY 11 -> 1:0 | DEL 12 | COPY 1 -> 6:0 |Time: 1.5343549E7
Step 17 | COPY 11 -> 1:0 | DEL 12 | DEL 13 |Patch invalid
Step 18 | DEL 12 |*** New best *** Time: 2008354.0(ns)
Step 19 | DEL 12 | DEL 0 |Failed to compile
Step 20 | DEL 12 | COPY 5 -> 2:0 |Time: 4259756.0
Step 21 |Time: 4324886.0
Step 22 | DEL 12 | MOVE 12 -> 6:0 |Patch invalid
Step 23 | DEL 12 | COPY 10 -> 4:0 |Failed to compile
Step 24 |Time: 2329325.0
Step 25 |Time: 3040417.0
Step 26 |Time: 2725780.0
Step 27 |*** New best *** Time: 1742064.0(ns)
Step 28 | MOVE 5 -> 1:0 |Failed to compile
Step 29 | MOVE 3 -> 2:0 |Time: 8787057.0
Step 30 | COPY 2 -> 1:0 |Time: 4770382.0
Step 31 | COPY 18 -> 5:0 |Failed to compile
Step 32 | DEL 17 |Time: 2432983.0
Step 33 | DEL 16 |Failed to compile
Step 34 | MOVE 1 -> 4:0 |Failed to compile
Step 35 | MOVE 6 -> 0:1 |Patch invalid
Step 36 | COPY 2 -> 2:0 |Failed to compile
Step 37 | COPY 6 -> 0:2 |Failed to compile
Step 38 | COPY 13 -> 4:0 |Failed to compile
Step 39 | MOVE 9 -> 3:1 |Failed to pass all tests
Step 40 | MOVE 9 -> 0:2 |Failed to compile
Step 41 | COPY 19 -> 0:0 |Failed to compile
Step 42 | DEL 2 |Patch invalid
Step 43 | DEL 11 |Patch invalid
Step 44 | MOVE 0 -> 2:0 |Failed to compile
Step 45 | MOVE 3 -> 2:0 |Time: 8274254.0
Step 46 | COPY 10 -> 0:1 |Time: 4223412.0
Step 47 | MOVE 11 -> 4:0 |Patch invalid
Step 48 | COPY 3 -> 5:0 |Failed to compile
Step 49 | MOVE 18 -> 5:0 |Failed to compile
Step 50 | MOVE 5 -> 6:2 |Failed to compile
Step 51 | DEL 2 |Patch invalid
Step 52 | DEL 10 |Time: 2993312.0
Step 53 | COPY 11 -> 6:0 |Failed to compile
Step 54 | MOVE 8 -> 3:2 |Time: 2385970.0
Step 55 | COPY 11 -> 4:0 |Time: 2803413.0
Step 56 | DEL 5 |Time: 3657767.0
Step 57 | COPY 9 -> 1:0 |Failed to compile
Step 58 | DEL 13 |Patch invalid
Step 59 | MOVE 4 -> 0:2 |Patch invalid
Step 60 | DEL 15 |Patch invalid
Step 61 | DEL 10 |Time: 2856155.0
Step 62 | DEL 4 |Patch invalid
Step 63 | MOVE 18 -> 1:0 |Failed to compile
Step 64 | COPY 5 -> 1:0 |Failed to compile
Step 65 | COPY 0 -> 2:0 |Failed to compile
Step 66 | MOVE 7 -> 5:0 |Failed to compile
Step 67 | COPY 2 -> 0:0 |Time: 2157261.0
Step 68 | DEL 13 |Patch invalid
Step 69 | MOVE 3 -> 0:2 |Time: 5331342.0
Step 70 | DEL 5 |Time: 3313758.0
Step 71 | COPY 16 -> 6:1 |Failed to compile
Step 72 | COPY 2 -> 0:1 |*** New best *** Time: 1394517.0(ns)
Step 73 |Time: 5582996.0
Step 74 | COPY 2 -> 0:1 | DEL 0 |Failed to compile
Step 75 | COPY 2 -> 0:1 | DEL 0 |Failed to compile
Step 76 | COPY 2 -> 0:1 | DEL 12 |Time: 3002779.0
Step 77 |Time: 5694114.0
Step 78 | COPY 2 -> 0:1 | MOVE 7 -> 5:0 |Failed to compile
Step 79 |Time: 3539437.0
Step 80 | COPY 2 -> 0:1 | MOVE 8 -> 2:0 |Failed to pass all tests
Step 81 | COPY 2 -> 0:1 | COPY 16 -> 0:1 |Failed to compile
Step 82 | COPY 2 -> 0:1 | MOVE 3 -> 5:0 |Failed to compile
Step 83 |Time: 2307858.0
Step 84 |Time: 4806585.0
Step 85 |Time: 4796634.0
Step 86 | COPY 2 -> 0:1 | DEL 4 |Patch invalid
Step 87 |Time: 2615552.0
Step 88 | COPY 2 -> 0:1 | COPY 13 -> 6:1 |Failed to compile
Step 89 | COPY 2 -> 0:1 | MOVE 15 -> 4:0 |Patch invalid
Step 90 |Time: 3400359.0
Step 91 |Time: 1422670.0
Step 92 | COPY 2 -> 0:1 | DEL 19 |Failed to compile
Step 93 | COPY 2 -> 0:1 | COPY 14 -> 2:0 |Time: 2111640.0
Step 94 | COPY 2 -> 0:1 | MOVE 6 -> 2:0 |Patch invalid
Step 95 |Time: 8795765.0
Step 96 | COPY 2 -> 0:1 | MOVE 14 -> 0:2 |Failed to compile
Step 97 |Time: 7091829.0
Step 98 |Time: 6114283.0
Step 99 |Time: 8386180.0
Step 100 |Time: 1971654.0

Best patch found: | COPY 2 -> 0:1 |
Found at step: 72
Best execution time: 1394517.0 (ns)
Speedup (%): 0.7406635361941513

==> Does *** New best *** mean that this patch passes all test cases, or just more than before? I assume Gin does not optimise in a multi-objective fashion right away, but maybe it optimises lexicographically <testsFailed,runtime>? Or what does it do? Minimise time, but always passing all tests? Yes, I have yet to look into the source code... Given the distinction of "patch invalid" and "failed to compile", I think the goal is runtime minimisation under the condition that all tests are passed, right? If so, this might be a statement for the Readme.md, e.g., "Running a Simple Example" with an explanation of the terms used.

==> Maybe "Speedup (%): 0.7406635361941513" can be rewritten to "Speedup (%): 74.06635361941513"? Strictly speaking, the speedup is no 0.74%.

These methods create clones of the nodes to return, but use clone() rather than CloneVisitorCopyIDs so any child nodes don't get copied with their associated IDs. Only really an issue for debugging but could trip someone up sometime!

Ordinarily if there is a bug in the test class, then we get a compile error, and if a test fails, there is a "failed to pass all tests".

In examples/TriangleTest.java, if all references to the Triangle class are replaced with ExampleTriangleProgram, then Gin will run, but reporting much faster run times (one example just now has 1203275 ns for the unpatched program), like the delay code didn't trigger but everything else did.

I assume this is because the JUnit reflection code finds ExampleTriangleProgram in the Gin classpath and runs it, though it doesn't explain why the delay isn't firing. Possibly this is just "something to watch out for"?

Reported by a few users - the TriangleTest class file is not in Gin, and therefore the minimal example doesn't run.

Currently, the Edit classes are just wrappers for ints, which are then used by Patch.apply(). It would be cleaner to put the operation in each of the Edit classes - though this would mean they have access to the lists of statements and blocks in Patch so not easy to separate out.

A few people have reported problems with future versions of the JDK and Gin. This could be a regression, but we should do some proper evaluation and work out what's going on.

Basic GenProg-like GP search added in utilities. A few efficiency improvements could be made: history recording (to avoid fitness-recalculations), early termination (e.g. for bug fixing when fix is found, or on first timeout for runtime improvement). Tests are also missing.

As it stands for Gradle projects we can generate tests, but we will have to manually modify the build.gradle file in the given gradle project in order to add the EvoSuite dependency (as shown in testgeneration/init.gradle), so that they could be run with gradle test command (for profiling). For Maven projects the pom is updated programmatically. Theoretically one could use FileWriter to do this task for Gradle, but it would be much nicer to do this programmatically and it seems like wtih gradle tooling api version 6.6.1 it would be possible, hence I am not implementing the FileWriter hack. However, currently updating the tooling dependency breaks a Gin test, so perhaps it's best to put this issue on hold until we are ready for a general upgrade of dependencies.

Add new sampler (well, copy tidy versions of the one from private repo) from this paper: Brownlee AEI, Petke J & Rasburn AF (2020) Injecting Shortcuts for Faster Running Java Code. In: IEEE World Congress on Computational Intelligence, Glasgow, 19.07.2020-24.07.2020. Piscataway, NJ, USA: IEEE. https://wcci2020.org/

This sampler extends RandomSampler, by adding repeat runs for successful patches, interleaved with runs of the original code, to allow comparisons of run times.

We need the facility to output a Gin patch as a standard diff.

Currently, gin only supports single-class optimisation.

A paper in development, and general application, requires the ability to handle larger systems.

This issue is to discuss and record development to that end.

This is due to implmentation of JUnitBridge.annotateTestWithTimeout and TestRunner.testsForClass methods which assume each individual Junit test is annotated. This is to avoid running setup/cleanup as a test, I think. Temporary workaround is to get all declared methods and only run those that start with "test" in its name, i.e:
add the following at line 84 in TestRunner (and import java.lang.reflect.Method;):
if (methods.size()==0) { for (Method eachTestMethod : clazz.getDeclaredMethods()) { if (eachTestMethod.getName().startsWith("test")) { String methodName = eachTestMethod.getName(); UnitTest test = new UnitTest(testClassName, methodName); tests.add(test); } } }
Need a cleaner solution.

Gin needs CI. We need it to accelerate development and make things scale.

Someone want to step up for this?

Markuss-MBP:git-gin wagner$ java -jar build/gin.jar examples/locoGP/SortBubble.java
Optimising source file: examples/locoGP/SortBubble.java

Initial execution time: -1.0 (ns)

Step 1 | COPY 5 -> 1:0 |Failed to compile
Step 2 | MOVE 10 -> 3:0 |Failed to compile
...
Step 100 | DEL 3 |Failed to compile

Best patch found: |
Found at step: 0
Best execution time: -1.0 (ns)
Speedup (%): -0.0

==> This seems to be a broken test case?
==> The same for SortHeap.java, SortInsertion.java, SortMerge.java, SortQuick.java, SortRadix.java, SortSelection.java, SortSelection2.java, and SortShell.text - unless the goal here is to fix a bug, of course.

https://github.com/gintool/gin/blob/master/src/main/java/gin/util/Profiler.java

line67 now:
private static final String HPROF_ARG = "-agentlib:hprof=cpu=samples,lineno=y,depth=1,file=";

amend to:
private static final String HPROF_ARG = "-agentlib:hprof=cpu=samples,interval=10,lineno=y,depth=1,file=";

as interval=10 is hprof's current default.

source:
https://docs.oracle.com/javase/8/docs/technotes/samples/hprof.html

Line edits can target any lines in a source file, including empty and comment-only lines. The constructors currently target one or two of these lines chosen uniformly at random. It might make more sense that comment/empty lines are excluded from the edit source and destination lines.

It is already possible to get a list of lines excluding those that are empty/comments by calling SourceFileLine.getLineIDsNonEmptyOrComments(). The samplers and enumerators do this, choose a pair of line numbers from that list, and call the Line edit constructors with specific line numbers rather than just an RNG and a source file. Something like this:

[code]
List allLines = sourceFile.getLineIDsNonEmptyOrComments(false);
List targetMethodLines = sourceFile.getLineIDsNonEmptyOrComments(true);
CopyLine cl = new CopyLine(sourceFile,targetMethodLines.get(rng.nextInt(targetMethodLines.size())), sourceFile, allLines.get(rng.nextInt(allLines.size())));
[/code]

We could either change the standard (SourceFile, RNG) constructors for Line edits to do this; or make an extended version of Line edits that only target non-empty/comment lines. My preference is for the latter; anyone have any thoughts either way?

Add new operators (well, copy tidy versions of the ones from private repo) from this paper: Brownlee AEI, Petke J & Rasburn AF (2020) Injecting Shortcuts for Faster Running Java Code. In: IEEE World Congress on Computational Intelligence, Glasgow, 19.07.2020-24.07.2020. Piscataway, NJ, USA: IEEE. https://wcci2020.org/

Test Gin on Windows and ensure all code is cross-platform, particularly unit tests.

It seems that if inner classes appear before the main class declaration, this method can pick those up instead, with can throw off detection of target methods.

e.g. in Disruptor, trying to target
com.lmax.disruptor.Sequence.compareAndSet(long,long)
gets broken because Sequence.java contains classes LhsPadding,Value, and RhsPadding before Sequence.

Add a section to the Gin repo with papers that use Gin. We could add a note, "if you used Gin in your research, please let us know, so we could add your paper to the list below”

As the list of edits is likely to grow, it might be better to allow a user to specify specific edits that can be used in sampling/searching, rather than the "families" of edits specified by EditType at the moment. I expect something like allowing a list of string of edit names, which are then loaded using reflection.

This will mean that all edits will need a standard constructor (probably taking a sourcefile and RNG as arguments).

I was talking to Justyna and she raised up the idea of early termination of test executions when a single test fails.
Thus, instead of executing the test suite until the end, we can stop the execution at the first failing test case and return the maximum fitness value.

We can alternatively set it as default (since I believe it is going to speed up considerably the optimisation), and add a command line option such as "-ft" (full testing) to run all test cases even when one fails, just in case this functionality is needed.

We have to also think how to do it in case of multiple repetitions of test case executions.

gradle shadowJar
java -jar build/gin.jar ./examples/locoGP/SortCocktail.java

produces

Step 93 | COPY 2 -> 0:2 | DEL 12 | MOVE 16 -> 2:0 |Exception in thread "main" java.lang.AssertionError: I am not a child of my parent.

From Markus' original comments:

==> Does *** New best *** mean that this patch passes all test cases, or just more than before? I assume Gin does not optimise in a multi-objective fashion right away, but maybe it optimises lexicographically <testsFailed,runtime>? Or what does it do? Minimise time, but always passing all tests? Yes, I have yet to look into the source code... Given the distinction of "patch invalid" and "failed to compile", I think the goal is runtime minimisation under the condition that all tests are passed, right? If so, this might be a statement for the Readme.md, e.g., "Running a Simple Example" with an explanation of the terms used.

The tests in IntegrationTests.java test the dynamic loading of in-memory compiled classes and use java assertions. These need to be turned into proper integration tests. I was unable to get these working with junit, because it interferes with the custom class loading.

So for now, integrating these into cradle to run on "gradle test" would be a good idea.

Currently in-memory compilation assumes a single class under optimisation, and that the class does not contain inner classes. The InMemoryCompiler used by gin will actually compiler the inner class, but it doesn't return the inner class' compiled object. We need to fork InMemoryCompiler and add a method to return a collection of class objects, including any inner classes, rather than a single class.

This step is necessary to support #25

Basically patchValid check is not performed in ExternalTestRunner. Spoted by @GiovaniGuizzo , thank you.
We need to decide on default behaviour and consistently apply it for both.

These methods typically use spaces to split into tokens for parsing. This split will include any spaces in the source file name.

Some comparisons:
https://exygy.com/which-license-should-i-use-mit-vs-apache-vs-gpl/
https://www.quora.com/Whats-the-difference-between-Apache-v2-0-and-MIT-license-What-are-the-major-consequences-of-using-Apache-software-over-MIT
https://snyk.io/blog/mit-apache-bsd-fairest-of-them-all

Information overload: https://en.wikipedia.org/wiki/Comparison_of_free_and_open-source_software_licenses

TLDR: yes, the MIT one is short and sweet and in plain English, but I do like the Apache 2.0 one for being just a little bit more detailed. Maybe this is something to consider.

There are many compilation warnings - some due to deprecated calls to JavaParser but plenty of others.

Add this to build.gradle to see them all:

tasks.withType(JavaCompile) { configure(options) { options.compilerArgs << '-Xlint:deprecation' << '-Xlint:unchecked' } }

If Gin is run of one of it’s dependencies (e.g. commons libraries) a linkage error occurs if a different version off the same library is used. This is to be expected as two classes with the same name are loaded to the CacheClassLoader. In this case InternalTestRunner simply should not be run: we need to confirm that it's okay with ExternalTestRunner, and then add a note on that to the README.

Not Java-specific as such but I think will be heavily slanted towards Java syntax. The idea is to look within statements for possible modifications. So e.g.:

• for operators, look for things that take the same arguments and return values (e.g. == =! > < are all mutually swappable)
• for an assignment, swap in a new variable or value for the RHS. This will mean maintaining a list of variables for the class (or possibly just for the parent statement so they are in scope)

Mainly to benefit from short circuiting, add a mutation operator to vary the order content of boolean expressions (so, look for || or && and swap their child nodes)

Use Evosuite before applying GIN
(do we want to integrate evolsuite with gin?)

Using more tests in GIN should give a smoother gradient on a problem, provided each test is somewhat independent. Not sure what fitness evosuite uses, but it could certainly help to automatically boost the test cases in GIN.

The current Javadoc is not complete. I have already improved and am still improving the Javadoc of some of the classes. For example: CacheClassLoader.

My suggestion is to improve and standardise the Javadoc while modifying a class, rather than improving the Javadoc of the whole prohect in one go (too boring).

If you are willing to add Javadoc, here is a link to best practices of documentation: Oracle's Guide.

Current behaviour:

When multiple statements are inserted into the same block they are all applied at the location of the earliest insertion.

Expected behaviour:

Insertions should happen at the positions at which they are applied individually.

Steps to reproduce:

1. Run the following command, inserting the statement delay(); in the source code: ("35 -> 34:61")

java -cp build/gin.jar gin.PatchAnalyser -f examples/triangle/Triangle.java -p "| gin.edit.statement.CopyStatement examples/triangle/Triangle.java:35 -> examples/triangle/Triangle.java:34:61 |"

Then diff examples/triangle/Triangle.java.original examples/triangle/Triangle.java.patched -c yields:

*** examples/triangle/Triangle.java.original	2020-12-10 20:44:36.721198276 +0000
--- examples/triangle/Triangle.java.patched	2020-12-10 21:07:01.147888414 +0000
***************
*** 16,21 ****
--- 16,22 ----
              a = b;
              b = tmp;
          }
+         delay();
          if (a > c) {
              int tmp = a;
              a = c;

2. Similarly, run the following command, inserting the statement delay(); 10 lines later in the source code: ("35 -> 34:96")

java -cp build/gin.jar gin.PatchAnalyser -f examples/triangle/Triangle.java -p "| gin.edit.statement.CopyStatement examples/triangle/Triangle.java:35 -> examples/triangle/Triangle.java:34:96 |"

diff examples/triangle/Triangle.java.original examples/triangle/Triangle.java.patched -c yields:

*** examples/triangle/Triangle.java.original	2020-12-10 20:44:36.721198276 +0000                
--- examples/triangle/Triangle.java.patched	2020-12-10 21:15:36.664564081 +0000
***************
*** 26,31 ****
--- 26,32 ----
              b = c;
              c = tmp;
          }
+         delay();
          if (a + b <= c) {
              return INVALID;
          } else if (a == b && b == c) {

3. Applied at the same time, both insertions happen at the same position. ("35 -> 34:61 | 35 -> 34:96" is incorrectly applied as "35 -> 34:61 | 35 -> 34:61")

java -cp build/gin.jar gin.PatchAnalyser -f examples/triangle/Triangle.java -p "| gin.edit.statement.CopyStatement examples/triangle/Triangle.java:35 -> examples/triangle/Triangle.java:34:61 | gin.edit.statement.CopyStatement examples/triangle/Triangle.java:35 -> examples/triangle/Triangle.java:34:96 |"

The diff yields:

*** examples/triangle/Triangle.java.original	2020-12-10 20:44:36.721198276 +0000                
--- examples/triangle/Triangle.java.patched	2020-12-10 21:18:07.734566718 +0000
***************
*** 16,21 ****
--- 16,23 ----
              a = b;
              b = tmp;
          }
+         delay();
+         delay();
          if (a > c) {
              int tmp = a;
              a = c;

4. Expected behaviour: the diff should yield instead:

*** examples/triangle/Triangle.java.original	2020-12-10 21:21:21.964570109 +0000               
--- examples/triangle/Triangle.java.patched	2020-12-10 21:21:04.691236475 +0000
***************
*** 16,21 ****
--- 16,22 ----
              a = b;
              b = tmp;
          }
+         delay();
          if (a > c) {
              int tmp = a;
              a = c;
***************
*** 26,31 ****
--- 27,33 ----
              b = c;
              c = tmp;
          }
+         delay();
          if (a + b <= c) {
              return INVALID;
          } else if (a == b && b == c) {

I propose making all classes Serializable (i.e. making them implement java.io.Serializable).
This will allow Gin's objects to be serialized by projects that depend on it or by users that want to perform some kind of distributed computing (e.g. Apache Spark, Hadoop, etc).

There is a minor performance issue with the parsing of the method file.
This block is not very efficient when we have thousands of test cases.

A HashSet instead of a List solves this problem. I tried to change it, but apparently the method index in this List is used in some classes. The refactoring would change the functionality of such classes, so I decided to create this issue instead.

If I end up fixing this issue, I'll commit the changes.
Cheers,
Giovani.

It's rather behind, and much has been improved.

I came across a problem with test poisoning when executing multiple repetitions of a same test suite while using ExternalTestRunner.
Basically, in the second execution of a test case (more precisely, this one), it fails.
Apparently there are some interactions between tests. Probably a static variable is being changed by other tests and it is making this test fail.

I fixed the bug in my development branch by forcing the start of a new process after each repetition, i.e., after all test cases in the test set are executed, the next test set repetition is done in a clean process (see it here).
This makes the test pass (intended behaviour).

However, I am not sure if this should be considered default behaviour.
In any case, I am happy to fix this bug, I just need your input on how to properly do it.
A few options:

1. Add this feature as default;
2. Add this feature as default, but also add an option to Sampler to disable it;
3. Do not add the feature as default, but add an option to enable it in Sampler.

Cheers,
Giovani.

gin.util.regression.impl.EkstaziRTSTest.testConstructor fails as examples/unittests/ekstazi/.ekstazi is not created by Config.createStorer(); - no idea why; need to consult with Giovani, to be resolved later