amarlearning / github-sectory Goto Github PK

View Code? Open in Web Editor NEW

43.0 9.0 12.0 2.89 MB

CLI for downloading sub-directory of any GitHub repository

License: MIT License

Python 100.00%

cli-app download github-sectory sub-directory python demo cli pip github-downloads hacktoberfest

github-sectory's Introduction

Github Sectory

CLI for downloading sub-directory of any Github repository using Github Content API!

Demo

Install

$ pip install github-sectory

Usage

Commands available

Pass the directory link as an argument-
	$ github-sectory <link_to_repo_directory>

Example:
	$ github-sectory https://github.com/amarlearning/PyStalker/tree/master/version01

OR

Pass the details as an arguments specified below-
	$ github-sectory -u <username> -r <repository-name> -d <directory-name> -b <branch-name>

Options:
	-r, name of the repository where the folder/directory is present.
	-d, name of the directory that you wish to download.
	-u, username/organisation name of owner of repository.

	-b, branch name of the repository, default is master [OPTIONAL].

Note: Default branch will be master

Issues

You can report the bugs at the issue tracker

You can tweet me if you can't get it to work. In fact, you should tweet me anyway.

License

Built with ♥ by Amar Prakash Pandey(@amarlearning) under MIT License

You can find a copy of the License at http://amarlearning.mit-license.org/

github-sectory's People

Contributors

Stargazers

Watchers

Forkers

pombredanne python3pkg jiyanminoscar cckim0805 krusenstiern hazemabdeen jiteshpabla lepy bibhuti93 seankessel souravaich fstavela

github-sectory's Issues

Directory not initialized with correct name.

Directory not initialized with correct name while downloading nested sub-direcotries

Add Test cases

It would be useful to have some basic smoke tests. I would like to create a package for openSUSE, but without any tests it isnt maintainable.

Also would be good to move the majority of the logic into a package, so the script is a thin wrapper, as that makes it more testable, and also the logic becomes re-usable.

Better directory structure

If I try to download a directory that is nested more than 1 level, I get this:

$ github-sectory https://github.com/chromium/chromium/tree/main/base/allocator/dispatcher/internal
[ Validating Input URL ]
[ Fetching folder content ]
[ Downloading all required files ]

[ Finished ]

$ du -a
4	./base/dispatch_data.cc
4	./base/dispatch_data.h
16	./base/dispatcher_internal.h
20	./base/dispatcher_internal_unittest.cc
4	./base/tools.h
48	./base
48	.

As you can see, the main directory naim is called base, even though I downloaded files from internal dir. I think the root of the downloaded dir structure should be called the same as the dir we are downloading, instead of the first level indention dir name.

Progress bar for ongoing download

I think there should be a dynamic progress bar to show the ongoing download status to the users. Something similar to what we find while using shell in linux.

Allow user to use their own github token

When we use github API without providing a token, we have requests limit set to 60 per IP per hour, which is not enough if we try to download a directory from a larger repo. When using the token, the limit is increased to 5000 per hour. We should allow the user to set their own token. It can be either through some config file, env variable or it can be passed as an option to the command.

Preserve commit history for files?

We are considering moving part of an open source project into a different project, and would like to preserve the commit history. That way the original authors still get attribution.

Does this tool preserve the commit history for files downloaded within the subfolder?

Show better error when I run out of Github API limit

If I run out of Github API limit, the script fails with this traceback:

Traceback (most recent call last):
  File "/home/fstavela/projects/github-sectory/github-sectory", line 182, in <module>
    GithubSectory().start()
  File "/home/fstavela/projects/github-sectory/github-sectory", line 142, in start
    self.downloadFile(self.path, self.response)
  File "/home/fstavela/projects/github-sectory/github-sectory", line 131, in downloadFile
    self.downloadFile(changedPath, newFileData)
  File "/home/fstavela/projects/github-sectory/github-sectory", line 122, in downloadFile
    if file['type'] == 'file':
TypeError: string indices must be integers

Process finished with exit code 1

This doesn't tell me anything useful and I had to do some debugging to find out what is the problem. The script should catch this issue and show me a useful error message informing me about exceeding the API limit.