Code Monkey home page Code Monkey logo

itc-reporter's Introduction

iTC Reporter

What?

This script mimics the official App Store Connect Reporter by Apple which is used to automatically retrieve Sales- and Financial Reports for your App Store sales. It is written in pure Python and doesn't need a Java runtime installation. Opposed to Apple's tool, it can fetch App Store Connect login credentials from the macOS Keychain in order to tighten security a bit. Also, it goes the extra mile and unzips the downloaded reports.

Before Apple in 2018 decided to rebrand it, App Store Connect used to be called iTunes Connect or iTC in short – hence the name of this script.

Why?

Once upon a time, there was autoingestion …

In the past, Apple has provided a tool called autoingest for automated retrieving of sales reports from App Store Connect. While this tool worked quite reliably, it needed a full blown Java Runtime Environment installed.

… nowadays, it's all about Reporter

Apple has shut down autoingestion on December 13th, 2016. Fortunately, it has been replaced by a new and a bit more streamlined tool called Reporter. Unfortunately, Reporter is based on Java, too. It also suffers from a minor but annoying security threat as it needs to read App Store Connect login credentials from a cleartext file.

Java really is a sledge hammer to crack this nut

There really is no compelling reason to employ a Java tool with its somewhat heavyweight dependency on JRE in order to download a few reports from App Store Connect. Apart from the Apple backend engineering staff really being into Java, apparently. Also, storing login credentials in cleartext isn't state of the art anymore.

iTC Reporter (this Python script), on the other hand, solves both problems. It is written in pure Python and doesn't need Java or any external dependencies to function. Also, it can optionally read the App Store Connect access token from the macOS Keychain (though it generally is still advisable to use an extra Apple ID specifically created for retrieving reports!).

How?

The argument names and values of this script have mostly been chosen to be consistent with Apple's documentation for Reporter. To get a quick overview, here is the output of ./reporter.py -h:

usage: reporter.py [-h] [-a ACCOUNT] [-m {Normal,Robot.XML}] -u USERID
                   {getStatus,getAccounts,getVendors,getVendorsAndRegions,getReportVersion,getFinancialReport,getSalesReport,getSubscriptionReport,getSubscriptionEventReport,getSubscriberReport,getNewsstandReport,getOptInReport,getPreOrderReport,generateToken,viewToken,deleteToken}
                   ...

Reporting tool for querying Sales- and Financial Reports from App Store Connect

optional arguments:
  -h, --help            show this help message and exit
  -a ACCOUNT, --account ACCOUNT
                        account number (needed if your Apple ID has access to
                        multiple accounts; for a list of your account numbers,
                        use the 'getAccounts' command)
  -m {Normal,Robot.XML}, --mode {Normal,Robot.XML}
                        output format: plain text or XML (defaults to
                        'Normal')

required arguments:
  -u USERID, --userid USERID
                        Apple ID for use with App Store Connect

commands:
  Specify the task you want to be carried out (use -h after a command's name
  to get additional help for that command)

  {getStatus,getAccounts,getVendors,getVendorsAndRegions,getReportVersion,getFinancialReport,getSalesReport,getSubscriptionReport,getSubscriptionEventReport,getSubscriberReport,getNewsstandReport,getOptInReport,getPreOrderReport,getPodcastsSubscriptionSnapshotReport,generateToken,viewToken,deleteToken}
    getStatus           check if App Store Connect is available for queries
    getAccounts         fetch a list of accounts accessible to the Apple ID
                        given in -u
    getVendors          fetch a list of vendors accessible to the Apple ID
                        given in -u
    getVendorsAndRegions
                        fetch a list of financial reports you can download by
                        vendor number and region
    getReportVersion    query what is the latest available version of reports
                        of a specific type and subtype
    getFinancialReport  download a financial report file for a specific region
                        and fiscal period
    getSalesReport      download a summary sales report file for a specific
                        date range
    getSubscriptionReport
                        download a subscription report file for a specific day
    getSubscriptionEventReport
                        download an aggregated subscriber activity report file
                        for a specific day
    getSubscriberReport
                        download a transaction-level subscriber activity
                        report file for a specific day
    getNewsstandReport  download a magazines & newspapers report file for a
                        specific date range
    getOptInReport      download contact information for customers who opt in
                        to share their contact information with you
    getPreOrderReport   download a summary report file of pre-ordered items
                        for a specific date range
    getPodcastsSubscriptionSnapshotReport
                        download an aggregated Apple Podcasts Subscription Snapshot
                        report file for a specific day
    generateToken       generate a token for accessing App Store Connect (expires
                        after 180 days) and optionally store it in the macOS
                        Keychain
    viewToken           display current App Store Connect access token and its
                        expiration date
    deleteToken         delete an existing App Store Connect access token

For a detailed description of report types, see
https://help.apple.com/itc/appssalesandtrends/#/itc37a18bcbf

Prerequisites

Creating an app-specific password

Assuming you are using two-factor authentication (2FA) with your Apple ID, you need to create an app-specific password for reporter.py by following these instructions.

Obtaining an App Store Connect access token

Since end of July 2017, Apple requires the use of access tokens instead of passwords for Reporter. To generate an access token for an Apple ID, log in to App Store Connect using the Apple ID that you plan to use with reporter.py. Go to Sales and Trends > Saved > Sales and Trends - Reports, then click on the help icon next to About Reports. Click Generate Access Token.

Or, instead of doing these steps manually, you can just let reporter.py fetch a token for you from App Store Connect. In addition, let's conveniently store it for future use in the macOS Keychain as an item named "iTC Access Token":

./reporter.py -u [email protected] generateToken -P <AppSpecificPassword> --update-keychain-item "iTC Access Token"

Your new access token has been generated.
AccessToken:4fbd6016-439d-4cef-a72e-5c465f8343d4
Expiration Date:2024-01-27
Keychain has been updated.

Storing credentials securely with Keychain

As access tokens expire after 180 days it probably is desirable to make the process of getting a new one automatable. But what about the cleartext app-specific password following the -P parameter? It definitely should be fetched from Keychain, too, instead of having to pass it on the command line! To accomplish this, you need to manually create a keychain item holding your password. To do so, open the Keychain Access.app, select the default keychain, press ⌘N and fill in the app-specific password. The item name you set for this new keychain entry is going to be what you have to supply for the -p (now lowercase!) parameter from now on. The command for obtaining a new access token now looks like this:

./reporter.py -u [email protected] generateToken -p "iTC App-Specific Password" --update-keychain-item "iTC Access Token"

Please note: Refreshing your access token using this command is only necessary after expiration (usually meaning every 180 days)! You can check for expiration with the viewToken command.

Usage examples

You are now equipped for regular use of this script by supplying your access token with the -t parameter. The following example queries App Store Connect's availability status for financial reports while fetching the access token from the Keychain item named "iTC Access Token":

./reporter.py -u [email protected] getStatus Finance -t "iTC Access Token"

Querying accessible accounts

Because your Apple ID could have access to multiple accounts, you will sometimes need to specify the account number you’d like to use. Use the following query to find out which accounts are available:

./reporter.py -u [email protected] getAccounts Sales -t "iTC Access Token"

The result is a list of account numbers you can then specify with the -a or --account argument in later queries regarding sales reports. Similarly, you'd use getAccounts Finance in order to find out account numbers that can be used for financial report queries.

Retrieving reports

Let's get to the point of this tool now: Retrieving reports from App Store Connect. To find out which vendor numbers you can query, you'll first need to get a list of available vendors, using (one of) the account number(s) you have found out with getAccounts before:

./reporter.py -u [email protected] --account 2821955 getVendors -t "iTC Access Token"

The resulting vendor number(s) can then be used to get the actual reports. In the following example, a sales report listing the sales of a single day (2023/07/18) for vendor 85442109 is going to be retrieved:

./reporter.py -u [email protected] -a 2821955 getSalesReport 85442109 Daily 20230718 -t "iTC Access Token"

Likewise, the following example fetches a financial report for sales in the US region in the first period of 2023 (according to Apple's fiscal calendar):

./reporter.py -u [email protected] -a 2821955 getFinancialReport 85442109 US 2023 01 -t "iTC Access Token"

These examples should do for a quick introduction. Don't forget to read Apple's reference documentation for Reporter. Also, you can get further help for a specific command by supplying -h after the command's name. For example:

./reporter.py getFinancialReport -h

What's still missing

There seem to be additional report types available for retrieving Apple Music and Apple Podcasts related data, but I wonder if anybody using this script would really need it.

Obligatory disclaimer

There is absolutely no warranty. I do not guarantee in any way that this tool works as intended or is fully compatible with Apple's official Reporter tool.

Pull Requests

Neither English nor Python are my native language – corrective PRs (even for style only) are very welcome!

itc-reporter's People

Contributors

fedoco avatar ixti avatar jakob avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

itc-reporter's Issues

this tool is great but how about FX

This tools is great in terms of grabbing the financial reports. thanks a lot to the author!

is there a tool that can download the FX in the Finanical Report! it was be tremoudous help!

Python3 Support

Hey,

I want to use Python3 with this library.

I have made all the changes locally and it seems to work for me.

Can you give me access to push branches so I can make a pull request? @fedoco

HTTP Error 504

This is an awesome project!

However, I can't seem to figure out what's going on here... I keep getting HTTP Error 504 when trying to download a financial report:

./reporter.py -u [email protected] -p "iTC Access" -a 219903 getFinancialReport 80092992 US 2016 01
HTTP Error 504. Did you choose reasonable query arguments?

I'm able to getStatus, etc, and have tried changing the server to add -sh to the URL per some other recommendations.

API documentation?

Hi!

Thanks for providing this tool! This will come in handy when Apple finally turns off the old autoingestion API.

I currently use curl to download sales reports.

I'm looking for the documentation for the JSON API that Reporter talks to. reporter.py is reasonable readable, but ideally I'm looking for some real docs. How did you find out how the API works?

Does Apple provide documentation for the JSON API?

Did you reverse engineer Reporter?

Best regards,
Jakob

getSubscriberReport cannot recognize date entry

Thank you so much for creating this Java-less solution.

However, I am having an issue with the getSubscriberReport method. My goal is to retrieve subscriber data daily for a Podcast.

My code for the getSalesReport works just fine

./reporter.py -u [email protected] -a ACCOUNTNUM getSalesReport VENDORID Daily 20230103 -T TOKEN

Returns There were no sales for the date specified.

But when I attempt to access the getSusbcriberReport, I run into a few issues.

  1. By simply replacing getSalesReport, I get this output.

INPUT: ./reporter.py -u [email protected] -a ACCOUNT_NUM getSubscriberReport VENDOR_ID Daily 20230103 -T TOKEN

OUTPUT:

usage: reporter.py [-h] [-a ACCOUNT] [-m {Normal,Robot.XML}] -u USERID
                   {getStatus,getAccounts,getVendors,getVendorsAndRegions,getReportVersion,getFinancialReport,getSalesReport,getSubscriptionReport,getSubscriptionEventReport,getSubscriberReport,getNewsstandReport,getOptInReport,getPreOrderReport,generateToken,viewToken,deleteToken}
                   ...
reporter.py: error: unrecognized arguments: 20230103
  1. I tried to leave out the date to see what would happen, and get a different error:

INPUT: ./reporter.py -u username[@apple-id.com] -a ACCOUNT_NUM getSubscriberReport VENDOR_ID Daily -T TOKEN

OUTPUT: Invalid date.

I've successfully retrieved this report using the Reporter download and CLI, however, I'm trying to automate this report in a Python environment. Are there any major errors I am making in my code? Is there an issue with this specific method?

Thanks

Downloaded reports are empty

Since December 12th, reports aren't decompressed properly anymore, resulting in files that only contain the header.

Is itc reporter is down?

Hello.
Is the service is down?
Since this morning we get 502 error (Bad Gateway) in the requests.
Thanks!

Forbidden Issue When Writing Similar Wrapper in GO

Hi, (Wasn't sure how to contact you so making an issue)

I am trying to create a wrapper for apple reporter in GO but my initial request to get Access Token keeps returning the following response:


response Status: 200 OK
response Headers: map[X-Content-Type-Options:[nosniff] X-Frame-Options:[SAMEORIGIN] X-Xss-Protection:[1; mode=block] Content-Language:[en-US] Set-Cookie:[JSESSIONID=1rfilqn1n5w3zgmjz6bqn6o25;Path=/reportservice] Strict-Transport-Security:[max-age=31536000; includeSubdomains] Date:[Mon, 11 Dec 2017 22:01:19 GMT] Server:[Shield] Connection:[keep-alive] Content-Length:[317] Expires:[Thu, 01 Jan 1970 00:00:00 GMT] Host:[reportingitc-reporter.apple.com] Content-Type:[text/html;charset=iso-8859-1]]
response Body: <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Forbidden</title>
</head>
<h1>Forbidden</h1>
<h5>You do not have permission to access this page.</h5>
<body>

</body>
</html>

Did you face this issue? If so, how did you solve it?

Thank you for your time!

Here is my code:

        command := "Sales" + ".generateToken"
	command = fmt.Sprintf("[p=Reporter.properties, %s]", command)
	m := map[string]interface{}{
		"account":    "None",
		"version":    "2.2",
		"mode":       "Normal",
		"password":   Password,
		"userid":     Username,
		"queryInput": command,
	}

	jsonStr, err := json.Marshal(m)
	if err != nil {
		fmt.Println("json.Marshal: ", err)
	}

	values := url.Values{}
	values.Add("jsonRequest", string(jsonStr))

	request, err := http.NewRequest("POST", "https://reportingitc-reporter.apple.com/reportservice/sales/v1", bytes.NewBuffer([]byte(values.Encode())))
	request.Header = http.Header{
		"Accept":     []string{"text/html", "image/gif", "image/jpeg; q=.2", "*/*; q=.2"},
		//"User-Agent": []string{"Java/1.8.0_92"},
		//"Content-Type": []string{"application/x-www-form-urlencoded"},
	}

	client := &http.Client{}
	resp, err := client.Do(request)
	if err != nil {
		fmt.Println("client.Do: ", err)
	}

	fmt.Println("response Status:", resp.Status)
	fmt.Println("response Headers:", resp.Header)
	body, _ := ioutil.ReadAll(resp.Body)
	fmt.Println("response Body:", string(body))
	resp.Body.Close()```

Can't figure out how to run the script

Hi there!

I'm trying to use the script on a Windows laptop. I already have an Access Token but I just don't get what I have to do now. I've read the whole README.md and it talks about the Keychain, which support is limited to macOS.

Is there any way to make this work on Windows?

There is something wrong

My system is Win7;
python version: 2.7 with anaconda

runfile('D:/python/Web Crawler/ios/reporter.py', wdir='D:/python/Web Crawler/ios', args = '-u ***@***.com generateToken -P ***** --update-keychain-item "iTC Access Token"')
Your new access token has been generated.
AccessToken:7b7***b5-2813-4**6-8**5-1d9**51
Expiration Date:2018-03-04

Traceback (most recent call last):

  File "<ipython-input-2-2ca4d77b16ef>", line 1, in <module>
    runfile('D:/python/Web Crawler/ios/reporter.py', wdir='D:/python/Web Crawler/ios', args = '-u ***** generateToken -P ***** --update-keychain-item "iTC Access Token"')

  File "C:\ProgramData\Anaconda3\envs\trade\lib\site-packages\spyder\utils\site\sitecustomize.py", line 688, in runfile
    execfile(filename, namespace)

  File "C:\ProgramData\Anaconda3\envs\trade\lib\site-packages\spyder\utils\site\sitecustomize.py", line 86, in execfile
    exec(compile(scripttext, filename, 'exec'), glob, loc)

  File "D:/python/Web Crawler/ios/reporter.py", line 356, in <module>
    args.func(args)

  File "D:/python/Web Crawler/ios/reporter.py", line 113, in itc_generate_token
    keychain.set_generic_password(None, args.update_keychain_item, '', token)

NameError: global name 'keychain' is not defined

Automatically generate access tokens

Apple has just sent out the following email:

We’ve also updated Reporter to let you generate an access token with the new generateToken command. To use this new command, download Reporter 2.2.

Find out how to generate an access token, and get details on automating downloads of your Sales and Trends reports and Financial reports in the Reporter User Guide.

I think this is great news! Generating the token in iTunes Connect is really cumbersome. It would be really nice if we could add a command to automatically generate the token.

The first step would be to reverse engineer what command iTunes Reporter sends to generates tokens.

How to run it without keychain ?

Thanks so much for this script, it's perfect. Only one small question, how can i run this script without keychain , from a linux or a windows machine for example ? Thanks

FX rate?

This is a great tool. any plan to add the function to download the payment report which include the FX rate?

Use the Tool with Windows

I´m getting a error message and i have no idea why. Maybe you can help me?

python reporter.py -u xxxxxxx generateToken -P xxxxxxxx
File "reporter.py", line 111
if not args.mode == 'Robot.XML': print "Keychain has been updated."
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(if not args.mode == 'Robot.XML': print "Keychain has been updated.")?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.