zerodayyy / spotcli Goto Github PK
View Code? Open in Web Editor NEWCLI for Spot Elastigroups management
Home Page: https://github.com/zerodayyy/spotcli/releases/latest
License: MIT License
CLI for Spot Elastigroups management
Home Page: https://github.com/zerodayyy/spotcli/releases/latest
License: MIT License
Put essential stuff in README, full documentation — in GitHub Wiki or on readthedocs.org.
Can be implemented same as in tasks submodule
Describe the bug
SpotCLI is not working on Python 3.9 due to a bug in threading library.
To reproduce
Steps to reproduce the behavior:
Expected behavior
The action runs successfully
Console output
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/zerodayyy/.pyenv/versions/3.9.0/bin/spotcli:8 in <module> │
│ │
│ /Users/zerodayyy/.pyenv/versions/3.9.0/lib/python3.9/site-packages/click/core.py:829 in __call__ │
│ │
│ 826 │ │
│ 827 │ def __call__(self, *args, **kwargs): │
│ 828 │ │ """Alias for :meth:`main`.""" │
│ ❱ 829 │ │ return self.main(*args, **kwargs) │
│ 830 │
│ 831 │
│ 832 class Command(BaseCommand): │
│ │
│ /Users/zerodayyy/.pyenv/versions/3.9.0/lib/python3.9/site-packages/click/core.py:782 in main │
│ │
│ 779 │ │ try: │
│ 780 │ │ │ try: │
│ 781 │ │ │ │ with self.make_context(prog_name, args, **extra) as ctx: │
│ ❱ 782 │ │ │ │ │ rv = self.invoke(ctx) │
│ 783 │ │ │ │ │ if not standalone_mode: │
│ 784 │ │ │ │ │ │ return rv │
│ 785 │ │ │ │ │ # it's not safe to `ctx.exit(rv)` here! │
│ │
│ /Users/zerodayyy/.pyenv/versions/3.9.0/lib/python3.9/site-packages/click/core.py:1259 in invoke │
│ │
│ 1256 │ │ │ │ Command.invoke(self, ctx) │
│ 1257 │ │ │ │ sub_ctx = cmd.make_context(cmd_name, args, parent=ctx) │
│ 1258 │ │ │ │ with sub_ctx: │
│ ❱ 1259 │ │ │ │ │ return _process_result(sub_ctx.command.invoke(sub_ctx)) │
│ 1260 │ │ │
│ 1261 │ │ # In chain mode we create the contexts step by step, but after the │
│ 1262 │ │ # base command has been invoked. Because at that point we do not │
│ │
│ /Users/zerodayyy/.pyenv/versions/3.9.0/lib/python3.9/site-packages/click/core.py:1066 in invoke │
│ │
│ 1063 │ │ """ │
│ 1064 │ │ _maybe_show_deprecated_notice(self) │
│ 1065 │ │ if self.callback is not None: │
│ ❱ 1066 │ │ │ return ctx.invoke(self.callback, **ctx.params) │
│ 1067 │
│ 1068 │
│ 1069 class MultiCommand(Command): │
│ │
│ /Users/zerodayyy/.pyenv/versions/3.9.0/lib/python3.9/site-packages/click/core.py:610 in invoke │
│ │
│ 607 │ │ args = args[2:] │
│ 608 │ │ with augment_usage_errors(self): │
│ 609 │ │ │ with ctx: │
│ ❱ 610 │ │ │ │ return callback(*args, **kwargs) │
│ 611 │ │
│ 612 │ def forward(*args, **kwargs): # noqa: B902 │
│ 613 │ │ """Similar to :meth:`invoke` but fills in default keyword │
│ │
│ /Users/zerodayyy/.pyenv/versions/3.9.0/lib/python3.9/site-packages/spotcli/cli.py:157 in scale │
│ │
│ 154 │ KIND is the scaling direction: up or down. │
│ 155 │ GROUP is elastigroup name, alias or regex. │
│ 156 │ """ │
│ ❱ 157 │ action(action=f"{kind}scale", target=group, amount=amount, auto_approve=auto_approve │
│ 158 │
│ 159 │
│ 160 main.add_command(version) │
│ │
│ /Users/zerodayyy/.pyenv/versions/3.9.0/lib/python3.9/site-packages/spotcli/cli.py:185 in action │
│ │
│ 182 │ console.print("\n") │
│ 183 │ auto_approve or rich.prompt.Confirm.ask("Continue?") or sys.exit(1) │
│ 184 │ t_start = time.time() │
│ ❱ 185 │ task.run() │
│ 186 │ t_end = time.time() │
│ 187 │ duration = t_end - t_start │
│ 188 │ console.print(f"\n[bold green]Task run complete! Ran 1 task in {duration:.2f} second │
│ │
│ /Users/zerodayyy/.pyenv/versions/3.9.0/lib/python3.9/site-packages/spotcli/tasks.py:134 in run │
│ │
│ 131 │ │ loom = pexecute.thread.ThreadLoom(max_runner_cap=PARALLEL_THREADS) │
│ 132 │ │ for target in self.targets: │
│ 133 │ │ │ loom.add_function(work, [], dict(target=target, amount=self.amount, console= │
│ ❱ 134 │ │ return loom.execute() │
│ 135 │
│ 136 │
│ 137 @Task.register("downscale") │
│ │
│ /Users/zerodayyy/.pyenv/versions/3.9.0/lib/python3.9/site-packages/pexecute/base_loom.py:125 in │
│ execute │
│ │
│ 122 │ │ │ runner = self.runners.pop(0) │
│ 123 │ │ │ runner.start() │
│ 124 │ │ │ self.started.append(runner) │
│ ❱ 125 │ │ │ while self.get_active_runner_count() >= self.max_runner_cap: │
│ 126 │ │ │ │ time.sleep(self.POOL_FILL_PAUSE) │
│ 127 │ │ │
│ 128 │ │ while self.get_active_runner_count(): │
│ │
│ /Users/zerodayyy/.pyenv/versions/3.9.0/lib/python3.9/site-packages/pexecute/base_loom.py:141 in │
│ get_active_runner_count │
│ │
│ 138 │ │ │
│ 139 │ │ count = 0 │
│ 140 │ │ for runner in self.started: │
│ ❱ 141 │ │ │ if runner.is_running() or not runner.is_tracker_updated(): │
│ 142 │ │ │ │ count += 1 │
│ 143 │ │ │
│ 144 │ │ if count == 0: │
│ │
│ /Users/zerodayyy/.pyenv/versions/3.9.0/lib/python3.9/site-packages/pexecute/thread/runner.py:21 │
│ in is_running │
│ │
│ 18 │ def is_running(self): │
│ 19 │ │ """ Returns True if runner is active else False """ │
│ 20 │ │ │
│ ❱ 21 │ │ return self.runner and self.runner.isAlive() │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: 'Thread' object has no attribute 'isAlive'
System (please complete the following information):
Sometimes you need to revert exactly to the state before the scenario/action run. Stateful mode should solve this problem.
Is your feature request related to a problem? Please describe.
At some point the output of spotcli list x
will become way too large to be readable.
Describe the solution you'd like
A simple regex or substring filter, and a CLI option to enable it, like spotcli list scenarios -f ow
Describe alternatives you've considered
Using grep on the table is not convenient
Additional context
From implementation standpoint, the existing filter in spotcli.elastigroup
module can be reused if separated into its own module.
To make sure configs aren't edited by unauthorized users, a provider with access control is needed. S3 is a perfect choice for this use case.
TODO:
boto3
as a fallbackSince SpotCLI is a distributed app, it's important to track all kinds of exceptions in it, as well as the adoption rate of new versions. For this, Sentry can be utilized.
This should be an opt-out feature, which one can disable in the app config: no data should be sent once user opts out of telemetry.
We need a possibility to log all actions somewhere. Good first candidates for "somewhere" are Slack and ELK.
TODO:
log
parameter to Scenario and CLIIs your feature request related to a problem? Please describe.
This will add more visibility when doing operations using this tool
Describe the solution you'd like
A command like spotcli list instances example-group
that returns the list of group's instances with some metadata like IPs, healthchecks etc.
Describe the bug
After running suspend/unsuspend on any scenario.
The groups got suspended or unsuspended, but the following exception is thrown:
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ spotcli/cli.py:375 in │
│ │
│ 372 │
│ 373 │
│ 374 if name == "main": │
│ ❱ 375 │ main() │
│ │
│ /Users/lior.franko/.pyenv/versions/3.7.7/lib/python3.7/site-packages/click/core.py:829 in │
│ call │
│ │
│ 826 │ │
│ 827 │ def call(self, *args, **kwargs): │
│ 828 │ │ """Alias for :meth:main
.""" │
│ ❱ 829 │ │ return self.main(*args, **kwargs) │
│ 830 │
│ 831 │
│ 832 class Command(BaseCommand): │
│ │
│ /Users/lior.franko/.pyenv/versions/3.7.7/lib/python3.7/site-packages/click/core.py:782 in main │
│ │
│ 779 │ │ try: │
│ 780 │ │ │ try: │
│ 781 │ │ │ │ with self.make_context(prog_name, args, **extra) as ctx: │
│ ❱ 782 │ │ │ │ │ rv = self.invoke(ctx) │
│ 783 │ │ │ │ │ if not standalone_mode: │
│ 784 │ │ │ │ │ │ return rv │
│ 785 │ │ │ │ │ # it's not safe to ctx.exit(rv)
here! │
│ │
│ /Users/lior.franko/.pyenv/versions/3.7.7/lib/python3.7/site-packages/click/core.py:1259 in │
│ invoke │
│ │
│ 1256 │ │ │ │ Command.invoke(self, ctx) │
│ 1257 │ │ │ │ sub_ctx = cmd.make_context(cmd_name, args, parent=ctx) │
│ 1258 │ │ │ │ with sub_ctx: │
│ ❱ 1259 │ │ │ │ │ return _process_result(sub_ctx.command.invoke(sub_ctx)) │
│ 1260 │ │ │
│ 1261 │ │ # In chain mode we create the contexts step by step, but after the │
│ 1262 │ │ # base command has been invoked. Because at that point we do not │
│ │
│ /Users/lior.franko/.pyenv/versions/3.7.7/lib/python3.7/site-packages/click/core.py:1066 in │
│ invoke │
│ │
│ 1063 │ │ """ │
│ 1064 │ │ _maybe_show_deprecated_notice(self) │
│ 1065 │ │ if self.callback is not None: │
│ ❱ 1066 │ │ │ return ctx.invoke(self.callback, **ctx.params) │
│ 1067 │
│ 1068 │
│ 1069 class MultiCommand(Command): │
│ │
│ /Users/lior.franko/.pyenv/versions/3.7.7/lib/python3.7/site-packages/click/core.py:610 in invoke │
│ │
│ 607 │ │ args = args[2:] │
│ 608 │ │ with augment_usage_errors(self): │
│ 609 │ │ │ with ctx: │
│ ❱ 610 │ │ │ │ return callback(*args, **kwargs) │
│ 611 │ │
│ 612 │ def forward(*args, **kwargs): # noqa: B902 │
│ 613 │ │ """Similar to :meth:invoke
but fills in default keyword │
│ │
│ spotcli/cli.py:137 in run │
│ │
│ 134 │ │ console.print("\n") │
│ 135 │ auto_approve or click.confirm("Continue?", abort=True) │
│ 136 │ t_start = time.time() │
│ ❱ 137 │ s.run() │
│ 138 │ t_end = time.time() │
│ 139 │ duration = t_end - t_start │
│ 140 │ console.print( │
│ │
│ /Users/lior.franko/Repos/spotcli/spotcli/configuration/tasks.py:282 in run │
│ │
│ 279 │ def run(self): │
│ 280 │ │ results = [] │
│ 281 │ │ for task in self.tasks: │
│ ❱ 282 │ │ │ results.extend(task.run()) │
│ 283 │ │ return results │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
TypeError: 'NoneType' object is not iterable
To Reproduce
Steps to reproduce the behavior:
Expected behavior
No error
Console output
The above exception
Configurations
If applicable, add all existing configs.
System (please complete the following information):
Additional info
Add any other info about the problem here.
To avoid app distributions from becoming outdated, a new version detection mechanism is needed.
Implementation idea: compare app version to latest tag name on GitHub. Use packaging.version.parse
to compare versions. Display update command.
spotcli list [aliases|scenarios]
— pretty-print available aliases/scenarios.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.