Download raw file contents from a specific commit about gitiles HOT 51 OPEN

curl "https://android.googlesource.com/device/lge/bullhead/+/master/mixer_paths.xml?format=TEXT"| base64 --decode > mixer_paths.xml

curl -s 'https://android.googlesource.com/device/lge/bullhead/+/master/mixer_paths.xml?format=TEXT' | python -m base64 -d

So does that mean this isn't going to be fixed?

base64 decoding is not a solution.

A normal person should be able to go to 

https://chromium.googlesource.com/chromium/src/+/master/tools/bisect-builds.py

And simply click a link to download the file.

Support downloading the contents of a single file "as is" from Gitiles, for example:

 https://android.googlesource.com/platform/manifest/+raw/master/default.xml

to get the raw XML rather than it wrapped inside of HTML.

This is a bit of a challenge because the server has cookies, and this is possibly unsafe
user supplied data. Raw HTML or JavaScript could cause the user's session to be able
to be hijacked.

Would providing a json format (issue 27) have less security concerns?  Chrome team has
tools which need to get current state of certain files.  This is currently done via
chromium svn servers but I'd like to transition these tools to use GoB.  An example
file which tools fetch is:  https://chromium.googlesource.com/chromium/src/+/master/DEPS

Would it be sufficient to always set the content-type to something like "application/octet-stream"?
Are there any browsers that would attempt to parse/display that, rather than just downloading?
That might help avoid the browser hijacking issue, while still allowing easy programmatic
access from our tools.

Or another option, maybe always put the file(s) into a zip archive, which would force
downloading and unzipping, again avoiding browser handling of the content, but still
being pretty easy for scripts to deal with.

#1: JSON is a good idea, that would be easy to implement in the short term. Works for
the scripting case but we still want true raw file support for e.g. viewing in-repo
HTML.

#2: Unfortunately if you include a file with an <img> or <script> tag browsers are
still liable to do content sniffing regardless of Content-Type in the header :(. Modern
browsers will respect Content-Disposition: Attachment but older IEs may not. And again
we still want to support browsing raw HTML stored in the repo.

#3: Zip archive is another idea, but between that and JSON I think I prefer JSON :)

Also an issue for Google's "Skia" open-source project.  Details are at https://code.google.com/p/skia/issues/detail?id=1898
('git transition blocker: skia_tools.js needs to download tip-of-tree global_variables.json')

GitHub has this functionality (they serve raw blobs as text/plain). Are they just unaware
or unconcerned about the content-sniffing issue alluded to in #2?

github solves the problem of cookies by serving the raw URL from a different hostname;
i.e. https://github.com/dataarts/tailbone/blob/master/Guardfile is served raw from
https://raw.github.com/dataarts/tailbone/master/Guardfile

Could that work for gitiles?

Yes, for googlesource.com we plan to use a new, cookieless domain (and for open-source
Gitiles the domain will simply be configurable and we trust administrators to understand
the risks and implement their own cookie policy).

This quarter I plan to do JSON support and at least get the ball rolling with our security/domain
teams on the cookieless domain part.

Blocking:
https://code.google.com/p/chromium/issues/detail?id=337030
https://code.google.com/p/chromium/issues/detail?id=345087
and some aspects of:
https://code.google.com/p/chromium/issues/detail?id=343251

Base64-encoded raw file support is checked in as:
https://gerrit-review.googlesource.com/55202

Example URL (won't work until this is released for googlesource.com):
https://gerrit.googlesource.com/gitiles/+/master/gitiles-servlet/src/main/java/com/google/gitiles/PathServlet.java?format=TEXT

FWIW, the base64-encoded delivery is fine for my Chromium use cases. If you want to
leave this open pending "true" raw file support, that's fine, but I don't think it
should be a priority.

From the Skia perspective, base64 and JSON cover all of our bases too.

Blocking crbug.com/369329

sebmarchand, that crbug appears to be about a Python script. Any reason base64 doesn't
work for you?

Yeah, I updated that bug with some notes on how to fetch. I don't think this is blocking.

Another issue is with pointing bug reporters and others to download bisect-builds.py
from the Chromium repo. We used to link to the file on src.chromium.org. This page
seems to now point to the gitiles page:
http://www.chromium.org/developers/bisect-builds-py

But that's not especially useful as there's no clear way to download it. For this use,
I think we'd have to be able to download the file itself, and not some encoded version.

I was able to download bisect-builds.py by adding ?format=TEXT on the end, then manually
base64 decoding it, but it was a bit of a pain...

This is a real PITA!! Why on earth google doesn't provide a "raw" code link?? Absolutely
absurd. Please fix!

please increase the priority of this - it's a real pain to have to attach bisect-builds.py
to tickets when I want people to help me bisect Chromium.  Asking them to base64 decode
a raw download is too much of a barrier to entry.

wfh, the way to bump priority is to raise this with your local chrome-infra team member
and they can work it into feature planning with our team.

As for bisect-builds.py, I'm not yet convinced base64 decoding is too much of a barrier
to entry.

The documentation page linked above says, paraphrasing:
1. Download bisect-builds.py from this link.
2. Run this python command.

Would the barrier to entry really be that much higher if the documentation were rewritten
to say:
1. Run this command to download bisect-builds.py:
$ curl 'https://chromium.googlesource.com/chromium/src/+/master/tools/bisect-builds.py'
| base64 -d > bisect-builds.py
2. Run this python command

?

d:\src>curl
'curl' is not recognized as an internal or external command,
operable program or batch file.

d:\src>base64
'base64' is not recognized as an internal or external command,
operable program or batch file.

So I think it does for some less technical users (that would be typical of bisect-builds
users -- if they were devs they'd already have the tree pulled and just use it from
there.)

Ah. Windows. Point taken.

In addition to bisect-builds.py, when reporting compiler regressions I used to say "1.
Download file <link> 2. Build it with these flags". This is no longer possible either.
(Again, Windows.)

Issue 83 has been merged into this issue.

INVALID_ARGUMENT: Request contains an invalid argument
[type.googleapis.com/google.rpc.LocalizedMessage]
locale: "en-US"
message: "The specified format type is not supported"

const url = document.querySelector('.Footer a[href*="?format=TEXT"]').href;
const response = await fetch(url);
const contents = await response.text();
const dataUrl = `data:text/plain;base64,${contents}`;
const anchor = document.createElement('a');
anchor.href = dataUrl;
anchor.download = true;
anchor.textContent = 'download file';
document.body.append(anchor);