pulp / pulp-cli Goto Github PK

View Code? Open in Web Editor NEW

33.0 10.0 41.0 1.81 MB

Home Page: https://docs.pulpproject.org/pulp_cli/

License: GNU General Public License v2.0

Shell 21.30% Python 77.19% Makefile 1.12% HTML 0.38%

pulp-cli's Introduction

Pulp command line interface

This is a command line interface for Pulp 3.

This software is in beta and future releases may include backwards incompatible changes.

Check out our docs and the quickstart guide to get started.

pulp-cli's People

Contributors

Stargazers

Watchers

Forkers

daviddavis mdellweg ggainey bmbouter gerrod3 mkot02 wbclark melcorr thmo fao89 latran goosemania ipanova vonsch vriesdemichael evgeni sthivagar rochacbruno tjev gauravtalreja1 dralley jaskarannarula yadnyawalkya lubosmj quba42 aranc23 manisha15 atix-ag pavelpicka zapp42 ziegenberg dkliban tjmullicani xiaoruiguo decko mayacostantini michalpysik pedro-psb viehoang68 stanleyz dimamarkovv

pulp-cli's Issues

Skip tests if a particular plugin is not installed

Make initial config file creation easier

Ok so the config file is already pretty easy to make: https://github.com/pulp/pulp-cli#configuration It would be easier still though if the user didn't have to make the dir and then then the config file. If they could just have to edit it to customize it instead of the default that would be easier.

Does it make any sense to do any of the following?

a) have the config file created by default on installation.

b) have a CLI command that would populate the config file (kind of like pulp smash has).

What do you think?

As a plugin writer, I have docs and/or tooling to help me get started

Add new python remote options

Pulp python 3.2 adding three new options to the remote: keep_latest_packages, exclude_platforms, and package_types

Please add a link to GH on the pypi page

I think it would be nice to add a link to https://github.com/pulp/pulp-cli on https://pypi.org/project/pulp-cli.

As a CLI user, I can specify which trusted CAs to use when connecting to Pulp

This may just be a docs story about how pulp-cli uses certs.

Make export/import process simpler for CLI

Users should be able to export the repositories they want in one command and shouldn't need to know the difference between 'export' and 'exporter'. I propose we remove the 'export' commands and re-brand them under 'exporter'. Exporter should get two new subcommands: 'export' and 'history'.

The 'history' command would get three subcommands from the original 'export' command: 'list', 'show', and 'delete'. Export objects are basically history records of previous exports and thus would fit nicely under this context. If a user doesn't specify a subcommand when using 'history' the default behaviour should be 'list' so they can see all the export objects aka the history.

The new 'export' sub command will take over the original export's 'run' command. This command will have extra functionality to allow for the creation of an exporter so the whole process can be done in one command.

Examples:

with Exporter named foo:

$pulp exporter pulp export --name foo [export options]
Started export task. Task href: ...

with no Exporter:

$pulp exporter pulp export --base-path tmp/exports/ --repository file_repo file --repository rpm_repo rpm [export options]
Would you like to save these configurations? [y/n]
$y
Please give a name to the exporter config options:
$bar
Config options saved in exporter object bar
Started export task. Task href: ...

If the user would specify 'no' on saving the config options, the command would give the created exporter a default name. The other commands under 'exporter' should have their help descriptions updated to reflect the exporter's role as a config object.

A similar process should be applied to the 'importer' commands so that the experience is consistent.

As a user I want to be able to specify types of RPMs to skip during a repository sync

The REST API for the Pulp RPM Plugin documents a parameter for "skip_types" that is not available via the pulp-cli wrapper. This is a very valuable filter because it allows skipping source RPMs in dnf repositories. Source RPMs tend to be many times larger than their respective compiled RPMs and take up significant space with very little return. When using Pulp to fully mirror a repository, the sync process often times out due to a default timeout of 5 minutes for any given operation. A single source RPM, for example, in the Appstream subscription for Oracle Linux 8 can be as large as 5 GBs.

Allow to specify plugin and type along with the name for a dependent resource

The referencing schema for resources (like remote on repositories) should follow the pattern: [[app_label:]type:]name

Given that "file" is the default type, all those should be equivalent:
pulp file repository update --name "test" --remote "test_remote"
pulp file repository update --name "test" --remote "file:test_remote"
pulp file repository update --name "test" --remote "file:file:test_remote"

If the name has colons: ::name:with:colons

The version checks need to understand app_labels of plugins and repository names as a fallback

pulp/pulpcore#1102
changes the way components are reported in the status and the api docs.
We need to change to use the new method and be able to still understand the old one.

Allow the fixture base url to be configurable for tests

We're running the CLI tests in various other repo CIs that use the docker_fixtures setting in plugin_template. The CLI does not honor this setting though and uses fixtures.pulpproject.org. We should make fixutre base url configurable and update the CLI code in plugin_template to configure it.

As a user I want to be able to pass a path to a YAML file with a list of Ansible Collections I wish to be represented by a Ansible Remote in the Ansible plugin

When creating or updating a remote Collection for the Ansible plugin the REST API documents a parameter, "requirements_file." This parameter is currently exposed via the pulp CLI tool. However, it only accepts a string directly. When you instead pass a file path to a YAML file it fails to serialize the YAML file correctly to JSON and throws an exception. My YAML file looked similar to the following:

---
collections:
  - testing.ansible_testing_content
  - pulp.squeezer

Misleading error message: 'core>=3.10.dev is not available

Edit: The described behavior is "as designed", but we should rephrase the error message to point to user to the real issue, a server side version mismatch for a particular command.

Hello,

There is an error while running pulp repository list on latest stable pulp-cli version:

# /usr/local/bin/pulp rpm repository list
[
  {
    "pulp_href": "/pulp/api/v3/repositories/rpm/rpm/b1dc7dc8-b540-496f-a8be-bf5b8aae3f5a/",
    "pulp_created": "2021-03-22T07:14:29.138535Z",
    "versions_href": "/pulp/api/v3/repositories/rpm/rpm/b1dc7dc8-b540-496f-a8be-bf5b8aae3f5a/versions/",
    "latest_version_href": "/pulp/api/v3/repositories/rpm/rpm/b1dc7dc8-b540-496f-a8be-bf5b8aae3f5a/versions/0/",
    "name": "infra_8",
    "description": null,
    "remote": null,
    "metadata_signing_service": null,
    "retain_package_versions": 0
  },
  {
    "pulp_href": "/pulp/api/v3/repositories/rpm/rpm/54f63365-dd05-4b6f-9fca-a447f0ef3491/",
    "pulp_created": "2021-03-22T07:14:20.466037Z",
    "versions_href": "/pulp/api/v3/repositories/rpm/rpm/54f63365-dd05-4b6f-9fca-a447f0ef3491/versions/",
    "latest_version_href": "/pulp/api/v3/repositories/rpm/rpm/54f63365-dd05-4b6f-9fca-a447f0ef3491/versions/0/",
    "name": "centos_8_3_base",
    "description": null,
    "remote": null,
    "metadata_signing_service": null,
    "retain_package_versions": 0
  }
]

# /usr/local/bin/pulp repository list
Error: 'core>=3.10.dev is not available

# pip3 list installed | grep pulp-cli
pulp-cli (0.7.0)

Error: no such option: --base_url

pulp-cli-0.7.0

$ pulp config create --base_url "http://localhost"
Usage: pulp config create [OPTIONS]
Try 'pulp config create --help' for help.

Error: no such option: --base_url

It works if I specify some other option, e.g. --username.
pulp config create --base_url "http://localhost" is an example specified in docs.

pulp config refuses to work in case the config file is not valid

When the existing config file is not valid, not even editing the config file works:

$ pulp config edit              
Usage: pulp [OPTIONS] COMMAND [ARGS]...
Try 'pulp --help' for help.

Error: Invalid value for '--format': invalid choice: jsonn. (choose from json, yaml, none)

In this special case, pulp --format=json config edit helps.

In general though, if the config file fails to parse, it would be nice to get a warning and still be able to run the pulp config ... commands.

Feature: expose retain-package-versions to pulp/rpm/repository

Ensure that client cert works with netrc

Change the `--requirements` option on ansible remote commands to accept a string or file

We should probably follow the pattern we use elsewhere and have --requirements accept a string or a file with @ instead of having --requirements and --requirements-file.

Feature: add a "pulp shell" command to start an interactive command-interpreter

Add "rpm repo uploads erratum" to pulp-cli

Pulp 2 pulp-admin had the "rpm repo uploads erratum" functionality to upload erratas. Can we have the same for Pulp 3 in pulp-cli?

Thank you.

Publish docs to http://docs.pulpproject.org/pulp_cli

The ssh key secret is already set up (PULP_DOCS_KEY).

Add support for autopublish and autodistribute

https://pulp.plan.io/issues/7469

File publication list command is missing options

It seems that the publication list command is missing options. See:

https://docs.pulpproject.org/pulpcore/restapi.html#operation/publications_file_file_list

Autopublish support - file plugin

Support should be added for

setting the "repository" field on distributions (all of them, after the new distribution models are rolled out)
setting the "manifest" field on file repositories
setting the "autopublish" field on file repositories

pulp/pulp_file#492 (comment)

Move changelog template and issue checker workflow to Github Actions

Add label command to ansible distributions

Related to this pulp_ansible issue: https://pulp.plan.io/issues/8441

As a CLI user, I can manage labels for resources

Show distribution information by base_path

I think it would be useful to be able to lookup a yum distribution by its base path, something like:

pulp rpm distribution list --base-path=FOO
or
pulp rpm distribution show --base-path=FOO

Move all operation parameters to the last command

This does not include "--type" and the global parameters on the main command.

Repository version url is malformed

Run these commands:

pulp file repository create --name cli_test_file_repository
pulp file repository version show --repository cli_test_file_repository --version 0

You'll see for the repository version show command, there are 3 requests to the server:

[05/May/2021 13:59:38] "GET /pulp/api/v3/repositories/file/file/?name=cli_test_file_repository&offset=0&limit=1 HTTP/1.0" 200 552
[05/May/2021 13:59:38] "GET /pulp/api/v3/repositories/file/file/346223ec-10f3-414c-b9e6-e0bd777a7402/versions/0 HTTP/1.0" 301 0
[05/May/2021 13:59:39] "GET /pulp/api/v3/repositories/file/file/346223ec-10f3-414c-b9e6-e0bd777a7402/versions/0/ HTTP/1.0" 200 322

The first and third I expect. The second one I do not. Looks like the CLI is forming the href without the slash at the end.

As a CLI user, I can manage and filter by labels

`repository version show` doesn't show options for specifying the repository

Only --version and --help show up.

`pulp <plugin> remote show --name “name”` fails with error

(.venv) vagrant@some-computer ~/pulp-cli $ pulp rpm remote show --name "Red_Hat_Enterprise_Linux_7_Server_-_Extras_RPMs_x86_64-32327"
Traceback (most recent call last):
  File "/home/vagrant/pulp-cli/.venv/bin/pulp", line 33, in <module>
    sys.exit(load_entry_point('pulp-cli', 'console_scripts', 'pulp')())
  File "/home/vagrant/pulp-cli/.venv/lib64/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/vagrant/pulp-cli/.venv/lib64/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/vagrant/pulp-cli/.venv/lib64/python3.6/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/vagrant/pulp-cli/.venv/lib64/python3.6/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/vagrant/pulp-cli/.venv/lib64/python3.6/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/vagrant/pulp-cli/.venv/lib64/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/vagrant/pulp-cli/.venv/lib64/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/vagrant/pulp-cli/.venv/lib64/python3.6/site-packages/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/vagrant/pulp-cli/.venv/lib64/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/vagrant/pulp-cli/.venv/lib64/python3.6/site-packages/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/vagrant/pulp-cli/.venv/lib64/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/vagrant/pulp-cli/pulpcore/cli/common/generic.py", line 311, in show_by_name
    entity = entity_ctx.show(href)
  File "/home/vagrant/pulp-cli/pulpcore/cli/common/context.py", line 308, in show
    return self.pulp_ctx.call(self.READ_ID, parameters={self.HREF: href})
AttributeError: 'PulpRpmRemoteContext' object has no attribute 'READ_ID'

AttributeError: 'PulpFileRemoteContext' object has no attribute 'READ_ID'

I'm hitting this on 0.2.0:

$ git checkout 0.2.0
$ pulp file remote create --name test --url test
$ pulp file remote show --name test
Traceback (most recent call last):
  File "/usr/local/lib/pulp/bin/pulp", line 11, in <module>
    load_entry_point('pulp-cli', 'console_scripts', 'pulp')()
  File "/usr/local/lib/pulp/lib64/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/pulp/lib64/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/pulp/lib64/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/pulp/lib64/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/pulp/lib64/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/pulp/lib64/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/pulp/lib64/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/pulp/lib64/python3.7/site-packages/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/pulp/lib64/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/pulp/lib64/python3.7/site-packages/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/pulp/lib64/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/vagrant/devel/pulp-cli/pulpcore/cli/common/generic.py", line 57, in show_by_name
    entity = entity_ctx.show(href)
  File "/home/vagrant/devel/pulp-cli/pulpcore/cli/common/context.py", line 302, in show
    return self.pulp_ctx.call(self.READ_ID, parameters={self.HREF: href})
AttributeError: 'PulpFileRemoteContext' object has no attribute 'READ_ID'

Have json options accept json or a file

Provide a generic way to have json options accept json or a file.

We did this for file repository modify:

pulp-cli/pulpcore/cli/file/repository.py

Lines 175 to 199 in c1f1774

    
           def _load_json_from_option( 
        
               ctx: click.Context, 
        
               option_name: Any, 
        
               option_value: str, 
        
           ) -> List[Dict[str, str]]: 
        
               """Load JSON from input string or from file if string starts with @.""" 
        
               json_object: List[Dict[str, str]] 
        
               json_string: Union[str, bytes] 
        
               if option_value.startswith("@"): 
        
                   json_file = option_value[1:] 
        
                   try: 
        
                       with click.open_file(json_file, "rb") as fp: 
        
                           json_string = fp.read() 
        
                   except OSError: 
        
                       raise click.ClickException(f"Failed to load content from {json_file}") 
        
               else: 
        
                   json_string = option_value 
        
               try: 
        
                   json_object = json.loads(json_string) 
        
               except json.decoder.JSONDecodeError: 
        
                   raise click.ClickException("Failed to decode JSON") 
        
               else: 
        
                   return json_object

As a user I want to specify the default container remote to be used during sync by a repository during creation

Currently when specifying the remote collection to be used by an Ansible repository during creation an exception is thrown that indicates the type does not match. The remote collection can only be specified successfully when creating a sync job.

pulp-cli could use a project description on pypi

https://pypi.org/project/pulp-cli/

Feature: list objects by their associations and/or see "deeper" associations

It would be nice to see object associations without having to search around.

Example: See publications and distributions associated with a repo. For now you have to list the distributions, see what publications are associated, then look at the publication to see what repositories are associated.

Another example in a different direction: list repositories from a publication.

From a Katello developer's perspective, we do this all the time from the Rails console. Pretty much all of the associations you could wish for are there and it really helps with debugging.

As a user I want to be able to set connection timeout parameters for an RPM Remote to Null

The aiohttp.ClientTimeout fields in Remote are expose as connect_timeout, sock_connect_timeout, sock_read_timeout, and total_timeout. Only total_timeout has a default 300 seconds. The type handling of the options as passed to pulp rpm remote update|create insists on a Float. This prevents setting the value back to None or Null. None or 0 should set the parameter to disabled.

`pulp <plugin> remote/distribution show --href` does not exist

It would be nice to view a remote or a distribution by the href, not just the name.

(.venv) vagrant@some-computer ~/pulp-cli $ pulp --client-cert /etc/pki/katello/certs/pulp-client.crt --client-key /etc/pki/katello/private/pulp-client.key rpm distribution show --help
Usage: pulp rpm distribution show [OPTIONS]

  Shows details of an entry

Options:
  --name TEXT  Name of the entry  [required]
  --help       Show this message and exit.

As a user, I can download and verify content

In Pulp 2, there was a command to do this (pulp-admin repo download --repo-id <repo_id> --verify-all). Support has asked for this in Pulp 3.

Misleading/unhelpful error message when requests are redirected

I spent quite some time this afternoon helping bmbouter debug a problem with his CLI. Basically, his CLI was pointing to http://localhost and whenever he submitted a POST request, he got a KeyError: '200'.

The issue turned out to be that his webserver was redirecting requests from http to https and thus POST :/pulp/api/v3/remotes/file/file was being turned into GET :/pulp/api/v3/remotes/file/file/ which returned a 200 response the cli didn't understand because it expected a 201 response.

Traceback (most recent call last):
  File "/usr/local/lib/pulp/bin/pulp", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/cli/common/generic.py", line 397, in callback
    result = entity_ctx.create(body=body)
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/cli/common/context.py", line 389, in create
    result = self.pulp_ctx.call(
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/cli/common/context.py", line 137, in call
    result = self.api.call(operation_id, *args, **kwargs)
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/cli/common/openapi.py", line 284, in call
    return self.parse_response(method_spec, response)
  File "/usr/local/lib/pulp/lib64/python3.9/site-packages/pulpcore/cli/common/openapi.py", line 222, in parse_response
    response_spec = method_spec["responses"][str(100 * int(response.status_code / 100))]
KeyError: '200'

As a user, I can import pulp exports

Need some docs on how The Magic works for a command-author

We've done a lot to make writing commands declarative and with as little boiler-plate as possible. This is great - but it can make writing new commands/changing existing ones, challenging for someone new to the codebase. It would be great if there were a short overview doc on "How It All Works", for would-be command-writers.

As a user I can specify a file path for cert options such as --ca-cert for `pulp rpm remote create`

Using -b with async ansible commands errors or gives incorrect results

The '-b' or '--background' option runs commands without waiting for any tasks generated by the commands to complete. Some of the async ansible commands show the results of the tasks expecting them to be finished. Using the '-b' option skips the waiting causing them to error or show an incorrect result. The commands currently affected are:

distribution create & update
remote update
repository update

Solution:
Use the passed pulp_context to check if the '-b' option is set and then appropriately show the results of the task or not.

Make task state a choice option for pulp task list

pulp task list --state nonexistent

Intuitive workflow for adding content to repositories

The command repository add --name foo --filename "shelf-reader-0.1.tar.gz" is a little unintuitive, because you might think it's adding a repository, but it's adding content to a repository.

Would a reverse order work? pulp python repository --foo add --filename "shelf-reader-0.1.tar.gz"

Or maybe the names could be changed slightly?

pulp python repository add-content --repo-name foo --filename "shelf-reader-0.1.tar.gz"

Deprecate old repository content add/remove/modify commands

When #194 is finished the old repository content commands should be marked deprecated.

Feature: Pulp 3 migration plugin support

It would be nice to be able to browse migration data.

	def _load_json_from_option(
	ctx: click.Context,
	option_name: Any,
	option_value: str,
	) -> List[Dict[str, str]]:
	"""Load JSON from input string or from file if string starts with @."""
	json_object: List[Dict[str, str]]
	json_string: Union[str, bytes]

	if option_value.startswith("@"):
	json_file = option_value[1:]
	try:
	with click.open_file(json_file, "rb") as fp:
	json_string = fp.read()
	except OSError:
	raise click.ClickException(f"Failed to load content from {json_file}")
	else:
	json_string = option_value

	try:
	json_object = json.loads(json_string)
	except json.decoder.JSONDecodeError:
	raise click.ClickException("Failed to decode JSON")
	else:
	return json_object