crflynn / pypistats.org Goto Github PK

What I'd like to know is :
version 1.0.0 of my package foobar was downloaded X times, while version 1.1.0 was downloaded Y times.

This would give an idea of my package popularity over time, without requiring to record download count of every single day since the beginning of time (which is done with the overall endpoint of the API, only for a recent history).

(I just discovered this repo. For a long time, I did want to get pypi download counts without having to manipulate Big Query myself; http://pypistats.org solved this. Thanks!)

-- Louis

these pages have broken links to Google BigQuery

• https://pypistats.org/about
• https://pypistats.org/api/

working links

• https://console.cloud.google.com/search;q=bigquery-public-data:pypi
• https://console.cloud.google.com/marketplace/product/gcp-public-data-pypi/pypi

example SQL code

https://stackoverflow.com/a/71414912/10440128

moved from

• hugovk/pypistats#319
• pypi/warehouse#10900

Hello! You probably already know this, but just in case: it looks like the live site doesn't have any data for 7-28, even though we're almost to the end of 7-29.

(Very cool site by the way!)

It would be great to be able to get the list of packages from https://pypistats.org/top via the json api.

I can see this being used to see what the top packages were for the last day, for an application that wants to show those.

This is probably more of a hosting issue but I don't know where else to report it, and don't have the skills myself to debug it. I have noticed for the past approximately 3 to 5 days (can't remember exactly when it started but at least three days ago) the site pypistats.org is very slow. For most requests, it takes a very long time to come back; sometimes it times-out completely, returning "502 Bad Gateway" with "nginx/1.12.1" underneath.

I would like to use the data to correlate with openSUSE package names, which use the 'real' name supplied in setup.py, i.e. not-normalised.

I've been doing a bit of research at hugovk/top-pypi-packages#4, and the raw data from bigquery can include this, with a very small perf hit, or it can be added afterwards by doing lookups against PyPI directly, which would be a significant extra work.

I'm putting in the effort - I want to know where I should put the effort, here or somewhere else. If here, would you prefer it coming from bigquery, or added afterwards from PyPI. The latter makes sense if there are other details from PyPI that you believe are needed in this project which cant be obtained from the bigquery dataset.

Downloads stats for all packages can be found under the special keyword __all__:

• https://pypistats.org/packages/__all__

However, calling the API on __all__ returns 404:

• https://pypistats.org/api/packages/__all__/recent

Compare with a "real" package:

• https://pypistats.org/api/packages/pip/recent

Would it be possible to expose __all__ via the API in some way?

Thank you!

See e.g. https://pypistats.org/packages/pip

Method

1. Go to https://pypistats.org/packages/nearbeach

Expected Results

Download stats to be in single digits as it is an unknown app.

Actual Results

Downloads a month > 100

Notes

On the about page - https://pypistats.org/about - there is a link to the Google Big Data but it just shows the error "Unable to find table: the-psf:pypi.downloads". Do we know what is counted as a download?

I would like to propose a single extra button toggling the last 30/60/90 days for all the graphs.
The use case is simply comparing all the stats for a given period, without manually clicking different panels

Hi! When I queried the data from Google BigQuery, all download data was for 2016 onwards. Is there a way to obtain download data prior to 2016?

Has the project died?

If a dependency is listed more than once in install_requires or extra_requires (which is perfectly possible), e.g.,

    install_requires=["numpy"],
    extras_require={
        "all": ["netCDF4", "h5py", "lxml"],
        "exodus": ["netCDF4"],
        "hdf5": ["h5py"],
        "xml": ["lxml"],
    },

then it is listed multiple times in the web interface too, where it makes less sense:

Perhaps this can be uniqued.

Perhaps optional (extra) dependencies could be listed as such in the web interface, too.

Similar to NPM stats
I would like to be able to easily check the total downloads of a package.
I would like to be able to easily check the total downloads of all packages by a user.

Not all users are based on github, so oauth for github is a bit annoying for bitbucket / gitlab users

Don't die, please ❤️

pypistats.org has NO new data since July 19.

There is also ZERO for the data for all packages on April 8th, 2020, and on July 18, 2020. These may be related and/or due to the same cause (once that cause is discovered).

Hi,

Many thanks for this project! It's cool.

I'm a bandersnatch contributor and I'd love to see a page added for Mirroring software and the versions of each respective system used. Namely, bandersnatch, as I am bias.

Do you have an idea of how you'd like this done if I (or you) were to implement it? Today I query Google and a nice graph on your page would be amazing and simplify my life! :D

Hi, my package (https://pypi.org/project/xkcd-pass/) isn't being picked up by PyPIStats and not sure why.

I have a small script that queries the pypistats.org API to see how much my packages have been downloaded lately. It makes one request to /api/packages/:project/recent per project, and I only have eighteen projects. I usually only run it once every couple of weeks. However, the past few weeks, running it fails with a "429 TOO MANY REQUESTS" error somewhere in the middle of the project list. My best guess at an explanation is that some form of API rate-limiting was implemented after #20, but the limit seems set too low. What can I do to get my script working again?

Hi @crflynn !

I'm asking the following because you globaly return the wihout_mirror as the number of download but what happens in my case ?

Shouldn't we sums the with and without mirror totals ?

Python versions are done by straight comparison which means 3.11 is shown before 3.3.

What can cause null entries on the:

• Download * Python Major Graph
• Download * Python Minor Graph

What is the root cause of this on bandersnatch specifically?
https://pypistats.org/packages/bandersnatch

Is it bad data to Google?

since yesterday I can not reach pypistats.org anymore. All I get is a page saying „500“. I hope it will be up again soon. Thx for the great service.

Would you be willing to change the title name of the page like PyPI Download Stats - pypistats?

I can see which packages my package depends on. E.g. https://pypistats.org/packages/eliot shows Eliot depends on six, zope,interface, etc..

In order to track usage of my package, I'd love to be able to do the reverse lookup. I.e. be able to answer the question: which packages depend on eliot?

Hello, thank you very much for creating https://pypistats.org!

And thank you very much for creating an API as well, I've made a command-line tool pypistats that uses the API :)

https://pypi.org/project/pypistats
https://github.com/hugovk/pypistats

Here's example use:

$ pypistats python_minor pillow --last-month
| category | percent | downloads |
|----------|--------:|----------:|
|      2.7 |  46.64% | 1,512,429 |
|      3.6 |  30.34% |   983,838 |
|      3.5 |  12.53% |   406,429 |
|      3.7 |   6.12% |   198,558 |
|      3.4 |   3.41% |   110,552 |
| null     |   0.84% |    27,380 |
|      3.3 |   0.05% |     1,599 |
|      2.6 |   0.05% |     1,581 |
|      3.2 |   0.01% |       246 |
|      3.8 |   0.00% |       133 |
|      2.4 |   0.00% |         7 |
| Total    |         | 3,242,752 |

And the output is actually Markdown (though JSON is available via a switch), so can be pasted into GitHub like this:

This issue is just to let you know and to say thanks, feel free to close it once you've read it :)

Hi,

thanks for this project.
I noticed on https://pypistats.org/packages/solara that it listed dask as a dependency, while it is only an optional dependency.
Looking at:

where the extra is not directly after the ;

Regards,

Maarten

The /api/packages/<package>/recent endpoint does not currently have a mirrors argument, which would be helpful to see the number of downloads per month including mirrors.

It seems like the text totals for a package don't add up to what the graphs show. In the below image, the text shows 0, 7, and 15 downloads for day, week, and month, respectively, but the associated graph seems to show at least an order of magnitude more downloads. Am I reading the graph incorrectly? Perhaps the text totals are for downloads without mirrors, only?

It would be great to see a breakdown of source vs wheel downloads. As pointed out by @hugovk at hugovk/pypistats#245, the data could be derived from the download filenames.

Downloads have a distinct day-of-week pattern, where they're much lower on Saturday/Sunday. A 7-day moving average in the graph would make long term trends much easier to see.

Problem with CORS as mentioned below.

Whenever I go to https://pypistats.org/, I get greeted with this message:

I'm thinking that this is a problem on the websites and PyPI API's side. Please fix this as soon as possible.

For packages with zero downloads, these return 404:

• https://pypistats.org/api/packages/sugar/recent
• https://pypistats.org/api/packages/sugar/overall

However, these return data but with empty data:

https://pypistats.org/api/packages/sugar/python_major
https://pypistats.org/api/packages/sugar/python_minor
https://pypistats.org/api/packages/sugar/system

{
"data": [],
"package": "sugar",
"type": "python_major_downloads"
}

{
"data": [],
"package": "sugar",
"type": "python_minor_downloads"
}	

{
"data": [],
"package": "sugar",
"type": "system_downloads"
}

Similarly, for packages which don't exist:

https://pypistats.org/api/packages/notfound123444/recent
https://pypistats.org/api/packages/notfound123444/overall

These return "data": []:

https://pypistats.org/api/packages/notfound123444/python_major
https://pypistats.org/api/packages/notfound123444/python_minor
https://pypistats.org/api/packages/notfound123444/system

Should they all return 404?

It'd be handy if the by-python-version and by-OS graphs showed percentages of downloads that fell into each category on each day, instead of absolute numbers. We already see the total downloads in the graph at the top of each page, and then on the graphs below that it's hard to see how the different categories compare to each other because there's so much day-to-day variation in total downloads.

Note sure if the BigQuery API allows this, but I'd like to have download stats per pypi, user, not per project.

First, thanks for creating this awesome website 😄 👍

Second, here are some screenshots and links to illustrate

Source: https://pypistats.org/packages/__all__

Worth noting this is specific to linux downloads ...

This trend can be observed with individual packages too:

• https://pypistats.org/packages/pip
• https://pypistats.org/packages/cmake

Current page shows package dependencies, but not yet the dependent packages. I'll be great if we showed that as well.

Hi,

I am intermittently faced with a 429 RATE LIMIT EXCEEDED error when browsing pypistats.org

I'm only browsing the odd page, not making tonnes of requests through the API, so this is a little odd. It doesn't happen all of the time; maybe 25-30% of requests succeed. I have confirmed this on a different device using a different internet connection, so I think this is an issue at your end rather than mine.