TestGenie - an IntelliJ plugin that natively integrates EvoSuite into the IDE. EvoSuite is an automated test suite generation tool using evolutionary algorithms. Used for research @ SERG, TU Delft.
License: MIT License
For the pop-up, which appears during coverage visualisation in the gutter, create the ability to display tests that cover the line which you click the highlight on. The pop-up already displays the test names, these could be made clickable.
As a developer, i should be able to access advanced parameters in the Settings menu by pressing the corresponding button on Quick Access tool window panel, so that I can quickly access them in case i want to modify them.
As a developer, I could have my variables names in the generated tests be based on their usage, not just by their type (e.g if a string is passed as an argument of a method and that argument is named “username”, then the string in the test will also be named “username”), so that it is easier for me to read the generated tests.
UPDATE: after implementing the new features introduced by the client and having a discussion, it has been decided that our part of this issue is going to be implementing telemetry diff specifically for changes made in variable names.
Our client has given us feedback on the relevance of the parameters in our quick access window. Some of the parameters that we have included in the previous sprint are actually rarely (if at all) modified (like e.g local search budget). This issue will focus on incorporating their feedback and correcting the problems.
The fixes should ensure that the code is consistent throughout the entire code base, i.e. if we are removing a certain parameter from the quick access then we should make sure that it is removed correctly in all the places where it is no longer relevant/used.
search budget is the first);search budget and stopping condition have been placed in own category (since they are related).stopping condition is referring to the budget type of search budget (rename or extend a tooltip)search budget has a dynamic unit depending on the selected typeMake the evosuite runner class leverage Intellij's Backgroundable tasks so that there's a progress indicator shown.
Research the IntelliJ plugin development documentation and set up a basic plugin.
Make EvoSuite not generate a folder when tests are generated. (Deleting the folder after generation works.)
evosuite-tests folder is not generated (or deleted after EvoSuite exits)Implement UI tests for the code to ensure the quality/functionality of the plugin and code.
Possible sections to test:
As a developer, I must have a quick access to ~10 parameters in the side bar (tool window), so that i can easily and quickly tweak them.
Currently, the DTOs that we use (such as CompactReport) require us to import the entirety of EvoSuite into the classpath. This is not optimal, so it would be best to move those shared DTOs to a separate module, which can then be imported both to Thunderdome and to TestGenie.
Add a color picker to settings which allows the user to change the color of the highlight. This way, the user can use any color they like. Improves accessibility.
Test the part of our implementation that integrates evosuite.
Create a CI step that enforces a uniform code style accross the whole project
The generated tests tab is currently empty. This tab should not appear until tests are actually generated.
Furthermore, the "Generated tests" tab should be the one that is focused once test generation is complete, rather than the "Coverage visualisation" tab.
During our client meeting, we have discussed generating tests for classes and methods when clicking on them. The client has given us some feedback that we are going to incorporate in this issue (see Definition of Done)
Furthermore, the issue of generating tests for a class and a method from the last sprint must be continued: it is in its very basic version with a lot of limitations (overloads, java or third-party classes and methods). This will also be handled in this issue.
Write UI tests for the Quick Access Parameters tab in the tool window
Depends on #64
Since this plugin relies on a custom build of evosuite to work, the jar is currently sources as a raw binary inside the project's root which is not a good practice. Create a separate build process that allows developers to make custom build releases and integrate it with TestGenie's own build process.
As a developer, I would like have a quick overview of what amount of my code I tested.
Write tests for PSI selection (classes, methods and lines). Verify that only the classes that are supposed to be selected are selected, only the methods that are supposed to be selected are selected and only the lines that are supposed to be selected are selected.
Conduct research on how to test UI in IntelliJ Plugin. Consider reading the documentation from the intellij-platform-plugin-template and the official SDK. Perhaps implement a demo version of how to do testing on a separate test project.
As a developer, I should be able to use a Settings menu for the plugin, where I can set parameters such as Algorithms and Population, so that I can easily specify custom parameters where I want and leave the default ones I do not want to modify.
This issue has been modified to account for the new client requirements, which is reordering the options in the currently existing Settings menu.
As an TestGenie developer, I should be able to load the default values for parameters from a bundle rather than hardcoding them, so that it is more maintainable in case EvoSuite developers decide to change them.
This issue is primarily related to "Quick Access Parameters"
As a developer, I should be able to have the option to provide an EvoSuite jar executable to my plugin through a settings menu, so that I can actually use the plugin.
Setting menu for selecting a jar executableMove service classes to services package.
TestGenieSettingsService is moved into services package (@bolkhod )QuickAccessParametersService is moved into services package (@SergeyDatskiv )Ensure that calling TestGenie on non-java classes does not throw exceptions and the app does not crash
Note: it might be that no extra work has to be done for this issue. What is important is to verify that the Definition of Done is fulfilled.
As a developer, I must be able to manually select a method via TestGenie and generate tests for that method.
Depends on #7
Add issue templates for features and bugs.
Runner.kt is in dire need of a refactor. Currently the class has a lot of code duplication and a lot of inefficient usage of IDE real estate. This is a class that should have little to no runtime dependencies or mutable state (the only exception being if the user wants to use a custom java path than their system's default). However, due to the limitations of the build system, it doesn't seem to be possible to inject the required constants which are known at compile time.
As a developer, I should be able to provide the assertions that the generated tests should make use of, so that the algorithm can generate better tests and use better assertions in the future.
Note: this feature was tagged with an asterisk in the google doc, it had comments from Martin and Mitchell.
A few implementation things are unclear at the moment, i.e. how should the developer provide the assertions, where, should the developer be notified of successful input?
For more clarification refer to "Assertion Generation" feature in ProjectForum project description.
UPDATE: after implementing the new features introduced by the client and having a discussion, it has been decided that our part of this issue is going to be implementing telemetry diff specifically for changes made in assertions.
Ensure that class/method selection for generating tests does not crash when there are multiple carets
Note: it might be that no extra work has to be done for this issue. What is important is to verify that the Definition of Done is fulfilled.
As a developer, I must be able to manually select a class via TestGenie and generate tests for the entire class (i.e all its callable methods).
As a user, I should be able to have access to documentation for our plugin, so that I can learn how to use the tool.
Note: think about different ways of documenting the plugin for the user. Is documentation easy to find? Is there enough of it.
Create a menu in the tool window which can be used by other components of the plugin (generating tests for a class, generating tests for a method, etc.) to allow the user to view the generated tests.
As a developer, I should be able to see the visualisation of the coverage in the form of coloured highlights next to a line, denoting which tests cover which lines, so that I can see which lines and/or branches are covered by specific test cases.
As a developer, I must be able to change the default parameters, with which I run EvoSuite when I am generating tests. Namely, the following: the list of parameters the client suggested us, continuous test generation, test sequence creation, test execution, class under test, method under test via a Settings menu.
Test the Coverage Visualisation offered by TestGenie.
Currently, when tests are generated, they appear in the ToolWindow. However, the "Apply" button does not work.
This issue will focus on making it work and actually moving the files in a new test file in the tests folder.
As a developer, I could be able to conduct interactive learning via TestGenie using gathered feedback (e.g provide a proper string format (e.g regex) if the test generator used a wrong string format), so that it can be passed to the EvoSuite application in order to improve the quality and efficiency of the next generated tests.
UPDATE: after implementing the new features introduced by the client and having a discussion, it has been decided that our part of this issue is going to be implementing telemetry diff
The package name of the plugin as well as all identifiers must be changed from com.github.mitchellolsthoorn.testgenie to nl.tudelft.ewi.se.ciselab.testgenie.
Describe the bug
When the generated tests tool window is updated, it shows an "Unexpected tokens" syntax error over all tests.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
No syntax errors are shown
Screenshots
Currently, the plugin doesn't differentiate between different editors/workspaces of the user. For example, coverage visualization is always displayed on the editor window that's currently opened, even if the test target is not in that window.
Create basic integration between EvoSuite and the plugin
java -jar evosuite-1.2.0.jar -projectCP target/classes/ -class delft.ArrayUtils would have)Provide an option to supply a custom path to a Java executable instead using the default java as present in PATH.
Research how intellij and evosuite work when multiple test generation jobs are triggered in parallel
As a developer, I should be able to select specific lines in the IDE and tell the plugin to generate tests that cover those lines if they are valid (statements within a method).
Since EvoSuite only supports methods and classes for Units Under Test (UUT) but not specific lines, this feature would wrap around existing functionality and extend on it in the plugin layer.
As a developer, I should be able to access advanced parameters in the Settings menu by pressing the corresponding button on Quick Access tool window panel, so that I can quickly access them in case I want to modify them.
Finalise testing the Settings menu from the previous sprint.
As a developer, I should be able to have the option to provide an EvoSuite jar executable to my plugin through a settings menu, so that I can actually use the plugin.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.