The documentation at https://biometria-se.github.io/grizzly/command-line-interface/usage/#optional-arguments_2 links to https://biometria-se.github.io/grizzly/usage/variables/environment-configuration/ for environment specific configuration but that link gives HTTP 404.

update to the latest version of locust.

Add support for having wait time specified in a variable, i.e:

And ask for value of variable "wait_time"
..
And wait for "{{ wait_time }}" seconds

Say you write:

And repeat for "10" it

and hit ctrl+space for auto-completions, it will suggest erations, but when selecting that it will insert rations.

right now there's only tests for step implementations, but there is a gap behave and grizzly.

need to find a way to programatically call behave, that would parse and "execute" a feature file, and then being able to verify what the step implementation has been called with.

there is a problem with "module" tzdata not being found, even though it is installed.

fix that, and uncomment include: section in code-quality.yaml.

similar to Biometria-se/grizzly-cli@215522a

Adding a comment, similar to:

# grizzly-message: are you sure?

Should make grizzly-cli to prompt:

are you sure? [y/n]

before starting the test. it can be used to remind the operator to perform pre-checks etc.

and print locust statistics and error summary even though it was not finished.

also print Started: and Stopped: (but maybe Aborted) if run is ctrl + c'ed.

It would be convenient to be able to save all output to a log file, perhaps with a -l logfile option.

To get the steps listed in order in the output, as the order they are in the feature file, maybe the scenario hash could be changed to a numerical zero-filled index instead. Like:

001 firstscenario_step1_name
001 firstscenario_step2_name
002 secondscenario_step1_name
002 secondscenario_step2_name
[..]

feature for allowing async / parallell http calls, using requests. maybe even support for running any GrizzlyTask in parallell, in greenlets for non RequestTask implementations. but only allow the same type of task in an group.

Something in the line of:

Given parallell request group [with name "..."]
Then post/get/put request ....
And end parallell request group [with name "..."]

and the actual implementation would be a new task, called something like ParallellRequestTasks.

right now:

grizzly-cli run ... local ...
grizzly-cli run .. dist ...
grizzly-cli build ...

but since build is only applicable for dist, it would make more sense to have:

grizzly-cli local .. run ...
grizzly-cli dist .. run ...
grizzly-cli dist .. build ...

make it possible to register custom jinja2 filters to format variable values.

https://stackoverflow.com/a/67384801/3378455

create a grizzly decorator that automagically will register it as a filter when the code is loading.

by overloading any of the behave hooks, add handlers in GrizzlyContext, then register them on applicable runner in locust.run.

I.e, no gherkin steps for registering handlers.

also add locust runner to grizzly.state.locust, remove grizzly.state.enviroment and change usage from grizzly.state.environment to grizzly.state.locust.environment.

probably the image has to be built for every testcase.

They are quite secret right now...

if thrown from any of the wrapped tasks.

replace setup.py and setup.cfg with pyproject.toml.

When using the until task, an error could be expected on the first tries, so any errors should be buffered, and if the whole task fails then log the errors with locust. Normal logger.error should be done for all errors when they happen.

add with name "..." to the step expressions and implementations.

use the name when firing locust request-event.

https://github.com/Biometria-se/grizzly/blob/main/grizzly/locust.py#L83-L152

$ grep -E '^git\+.*grizzly.git' requirements.txt 
git+https://[email protected]/biometria-se/grizzly.git@main#egg=grizzly-loadtester[mq]
$ grizzly-cli --version all
Traceback (most recent call last):
  File "/home/vscode/.local/bin/grizzly-cli", line 8, in <module>
    sys.exit(main())
  File "/home/vscode/.local/lib/python3.8/site-packages/grizzly_cli/__main__.py", line 115, in main
    args = _parse_arguments()
  File "/home/vscode/.local/lib/python3.8/site-packages/grizzly_cli/__main__.py", line 62, in _parse_arguments
    grizzly_version, locust_version = get_dependency_versions()
  File "/home/vscode/.local/lib/python3.8/site-packages/grizzly_cli/utils.py", line 135, in get_dependency_versions
    rc += subprocess.check_call(
  File "/usr/local/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['git', 'checkout', '-b', 'main', '--track', 'origin/main']' returned non-zero exit status 128.

check if the branch specified in requirements.txt (@main) is the active branch after cloning the repository, and only checkout a branch if that is not the case.

"pytz has served the Python community well for many years, but it is no longer the best option for providing time zones. pytz has a non-standard interface that is very easy to misuse; this interface was necessary when pytz was created, because datetime had no way to represent ambiguous datetimes, but this was solved in in Python 3.6, which added a fold attribute to datetimes in PEP 495. With the addition of the zoneinfo module in Python 3.9 (PEP 615), there has never been a better time to migrate away from pytz."

https://pypi.org/project/pytz-deprecation-shim/

Intellisense for available step expressions that matches what the user has started to type.

Insertion of snippets, like http get would result in the necessary steps to create an RestApiUser and the step for a GET request.

when completing a step, if the step expression has any variables the cursor should be positioned for entering the first variable.

And value for variable "" is ""
  #                    ^       ^
  #                    |       |
  # cursor here -------´       |
  #  ...and not here ---------´

which would return a rendered clone of itself.

The container file for grizzly is always built med MQ libraries included;

Add some logic in grizzly_cli.build that checks if mq is included in the extras specification ([...]) in EXECUTION_CONTEXT/requirements.txt.
For package installation from pypi, it would be something like:
grizzly-loadtester[mq]

and for installation from git:
...#egg=grizzly-loadtester[mq]

The value inside of [] is a comma seperated list, so would have to take that into consideration.

Create a second container file, Container.mq, which would inherit from the ordinary image, and would run the mqm commands from the current Containerfile (which would then be removed from that file). Also move the setting of enviroment variable LD_LIBRARY_PATH from the current grizzly commands.

Might have to build the first image with a different name, depending on if building the second image can inherit FROM an image with the same name that it produces.

If a new name is needed, if mq is not in the extras specification, re-tag the base image with the final name.

See pypa/pip#11617

The new format is not supported if grizzly is specified as a git dependency when running grizzly-cli --version all

Must support:

foo @ git+https://github.com/some-org/some-repo.git@some-rev

And for now, also:

git+https://github.com/some-org/some-repo.git@some-rev#egg=foo

right now, grizzly-cli --version will not show the correct version.

simplify the release workflow, the same way that has been done for grizzly.

so it instead throws grizzly.scenario.failure_exception, if it's not fatal enough to continue with next iteration.

these should be investigated:

 $ find grizzly/ -type f -name '*.py' -exec grep -HE 'raise StopUser' {} \; | grep -v 'failure_exception'
grizzly/scenarios/iterator.py:                        raise StopUser()
grizzly/scenarios/iterator.py:            raise StopUser()
grizzly/scenarios/__init__.py:            raise StopUser()
grizzly/users/blobstorage.py:                    raise StopUser()
grizzly/users/servicebus.py:                raise StopUser()
grizzly/users/restapi.py:                raise StopUser()
grizzly/users/restapi.py:                raise StopUser()
grizzly/users/restapi.py:                raise StopUser()
grizzly/users/base/grizzly_user.py:            raise StopUser()
grizzly/users/sftp.py:                    raise StopUser()
grizzly/testdata/utils.py:                    raise StopUser()
grizzly/testdata/communication.py:            raise StopUser()
grizzly/testdata/communication.py:            raise StopUser()

the following step expression needs to be updated:

 $ find grizzly/steps/ -type f -name '*.py' -exec grep -HE '^@.* stop user' {} \;
grizzly/steps/scenario/response.py:@when(u'response {target:ResponseTarget} "{expression}" {condition:Condition} "{match_with}" stop user')

a new step/task to make it possible to have a constant scenario iteration time, which will give better control of the intensity of a scenario not depending on how good the backend services replies.

E.g. possible to say an iteration should take 2000 ms, and if the requests for that iteration takes 500 ms, it should sleep 1500 ms, and if the requests takes 2001 ms, no sleep at all.

Something in the style of:

And set iteration time to "2000" milliseconds

Easiest way would probably to implement something like IterationTimeTask, that checks how long since last time it executed, and compare that to the set limit.

in this version, zmq.sugar.constants is removed:

causing problems since we import constants from zmq.sugar.constants in various places.

I have installed grizzly-loadtest and grizzly-loadtester-cli on windows without problems.
One note is that C++ is huge to install setproctitle. That maybe you can look into as well?

But the problem I got when running yours example is this:

C:\Users\Pelle\PycharmProjects\grizzly-main\example> grizzly-cli --local -c environments/example.yaml features/example.feature
Traceback (most recent call last):
File "C:\Users\Pelle\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\Pelle\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Users\Pelle\PycharmProjects\grizzly-main\venv\Scripts\grizzly-cli.exe_main.py", line 4, in
File "c:\users\pelle\pycharmprojects\grizzly-main\venv\lib\site-packages\grizzly_cli\cli.py", line 21, in
from .build import main as build
File "c:\users\pelle\pycharmprojects\grizzly-main\venv\lib\site-packages\grizzly_cli\build.py", line 1, in
from os import environ, getuid, getgid
ImportError: cannot import name 'getuid' from 'os' (C:\Users\Pelle\AppData\Local\Programs\Python\Python310\lib\os.py)
PS C:\Users\Pelle\PycharmProjects\grizzly-main\example>

So os.py needs to be handled for windows user. Can you give me some tips?

RuntimeError('GET: PYIFError="PYMQI Error: not open"')

in case of an PYIFError with message "Error: not open" on an previously connected session occurs, we should try to re-connect.

see what can be done to make the container image smaller.

in after_feature, when printing started and stopped, only do it if not on_worker(context).

openlawlibrary/pygls#276

to get request statistics within a scenario in chronological order.

the sorting order should be based on a weight on RequestType, and if the request_type isn't defined in that enum (RequestMethod) it should be sorted on request name.

right now, something like SCEN 002 2nd scenario would be printed after GET 002 1-first-request.

similar to create_user_type and create_scenario_type, assuming any dots before Atomic is a namespace (package + module), if nothing is specified before Atomic, use default namespace grizzly.testdata.variables.

https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

change:

echo "::set-output name={name}::{value}"

to:

echo "{name}={value}" >> $GITHUB_OUTPUT

in all workflows, in all repos.

Example:

Then start timer with name "timer-01"
...
Then stop timer with name "timer-01"

in __call__ implementation:

• if start timer is None, set it to perf_counter()
• if start timer is not None, set response_time to int((perf_counter() - self.start) * 1000), set start to None and call parent.user.environment.events.request.fire, request type TIMR.

new attribute in GrizzlyContextScenario, which is a dict with name as key, and TimerTask as value.
start timer step implementation adds the TimerTask instance to this dict and to the scenario task list.
stop timer step implementations gets the TimerTask instance from this dict, adds it to the scenario task list and removes it from the dict.

in environment.py / after_scenario, check that there are no entries left in the dict (unclosed timers).

Is your feature request related to a problem? Please describe.
When running multiple tests in parallell, all logs end up in features/logs/, making it hard to tell which files come from which test

Describe the solution you'd like
An option to grizzly-cli, specifying where logs should be saved for the test, with default value features/logs as it works today

Describe alternatives you've considered
No alternatives :)

Additional context
N/A

Input, the URL to an OpenAPI/Swagger specification, and output would be a feature file with a scenario for each operation.

it is possible to inject grizzly-cli arguments via comments (metadata) in a feature file.

this is done by creating a line that starts with # grizzly-cli followed by the name of the subparser the argument is valid for and arguments for that subparser, e.g.:

# grizzly-cli run --verbose
Feature:
 ...

And the command:
grizzly-cli local run feature/test.feature -> grizzly local run --verbose feature/test.feature
grizzly-cli dist run feature/test.feature -> grizzly dist run --verbose feature/test.feature

If a metadata comment contains the name of a subparser that is not present in the command the user is executing, a message like ?? ignoring <arguments> will be printed.