Comments (15)
Verified from both local IIS 10 and IIS on Azure App Service as well, thank you :)
from acr.
@pbering Sorry you are right.
I could pull successfully because I already had the image on my machine. So docker pull only pulls the manifest but not the layer image.
If there is no layer image already on the local client, docker client will send GET request; our server will return 307 to redirect to Azure storage...But due to a bug I mentioned earlier, we may send positive content-lenght but not body. IIS is strict on that and will fail.
Our fix is checked in...but you may need to wait for the server to be patched to make your scenario working.
from acr.
@pbering We have updated our service. Could you please try again and let us know? Thanks!
from acr.
Here's a screenshot of the failed request tracing log:
from acr.
@pbering ARR uses WinHttp APIs to communicate with the backend server. This error is returned by async callback from WinHttp. It seems WinHttp could not understand the response from the backend. Could you please provide an iDNA trace so that we can take a look at it? Thanks!
from acr.
@pan-wang I'm not familiar with "iDNA" - found some old KB here https://support.microsoft.com/en-us/help/2716889/-sdp-3-3fc3cd2a-d297-4e37-a501-35cf7303e436-time-travel-tracing-diagnostic but with no download links. Can you point to some place where I can get the tool?
from acr.
I reproduced the failure locally. Per the FREB log, it seems ACR returned 307 response with non-zero content-length header value but empty body. ACR team is working on the fix. Let's wait for this fix to see whether it solves the problem.
from acr.
The ACR server is forwarding the 307 status code on blob GET request. However, there is a bug that the body of the response may not be forwarded to client.
Docker client seems to be more resilient since the body content is normally only be used by very old client. But IIS ARR module is more strict. We are working on a fix.
from acr.
I think the problem is the proxy configuration. You need to unselect the following option (which is selected by default I think):
Reverse rewrite host in response header
Our redirect is to the Storage SAS Url. With this being selected, IIS will override the Location header to point back to the Host (someregistry.azurecr.io). Our registry receives these calls and returns 404. So upload sees unknown blob.
Now after I unselected it, I could Pull/Push with ARR with no problem. Please try it and see if it can unblock you.
from acr.
@yuwaMSFT2 I've try with that setting both on an off with the same results. Also tried to toggle the other settings. My config looks like so:
from acr.
This is the configuration working for me:
This is the web.config
<?xml version="1.0" encoding="utf-8"?> <configuration> <system.webServer> <httpErrors errorMode="Detailed" /> <rewrite> <rules> <rule name="Proxy" stopProcessing="true"> <match url="(.*)" /> <action type="Rewrite" url="https://yuwatestcan1.azurecr.io/{R:1}" logRewrittenUrl="true" /> <serverVariables> <set name="HTTP_ACCEPT_ENCODING" value="" /> <set name="HTTP_HOST" value="yuwatestcan1.azurecr.io" /> <set name="HTTP_X_REAL_IP" value="{REMOTE_ADDR}" /> <set name="HTTP_X_FORWARDED_FOR" value="{REMOTE_ADDR}" /> <set name="HTTP_X_FORWARDED_PROTO" value="https" /> </serverVariables> </rule> </rules> <allowedServerVariables> <add name="HTTP_HOST" /> <add name="HTTP_ACCEPT_ENCODING" /> <add name="HTTP_X_REAL_IP" /> <add name="HTTP_X_FORWARDED_FOR" /> <add name="HTTP_X_FORWARDED_PROTO" /> </allowedServerVariables> </rewrite> </system.webServer> <location path="" overrideMode="Allow"> <appSettings /> </location> </configuration>
One of our engineer also tried it. After turning off the reverseHostOverride, he could pull/push.
What is the error you are getting? If you can share with me your registry name, I can also help checking our server logs.
from acr.
@yuwaMSFT2 mmm still not working with the exact same proxy setting and your web.config. It seems that I have a few more setting than in your screenshot, for example "stealthMode", which IIS / OS are you trying on? I'm on IIS 10 (Windows 10 CU Enterprise) and have ARR 3.0 x64 installed (also tryied to reinstall it). My ACR is running on chreg-pmbcodehouse392.azurecr.io.
The error from the docker client is:
PS C:>docker pull chreg-proxy.ad.codehouse.com/sitecore:8.2.170407.0
8.2.170407.0: Pulling from sitecore
error pulling image configuration: received unexpected HTTP status: 502 Bad Gateway
The error from IIS and FRT is in the beginning of this issue.
from acr.
Glad that the issue is confirmed and fixed! I would prefer to keep this issues open so others that try the same, can see that a fix is on its way. Could you please update this issue when fix in "live"?
from acr.
Sure. Will update here once the fix is applied to the service in all regions.
from acr.
This looks like an old issue but it really helped me out a bit. In my case, I'm using an IIS ARR Proxy, but I had to explicitly remove the ":" character from the requestPathInvalidCharacters attribute in the httpRuntime.
IIS 8 would proxy the manifest but not the blobs and that was why.
Anyway, just leaving that here in the event that someone else with the same problem comes across this particular Github issue just like me.
I added the following under the element in my web.config:
<system.web> <!-- <customErrors mode="Off"/> --> <httpRuntime requestPathInvalidCharacters="<,>,%,&,\,?" /> </system.web>
(I used the customErrors element to figure out what the problem was.)
from acr.
Related Issues (20)
- Manifests - Get API returns 404 for multi arch images
- Fail pulling image - manifest unknown
- Failing to pull image when Artifact streaming is enabled
- Dockerfile with extension is interpreted as YAML HOT 1
- Storage used per repository
- Pull Through Caching from Another Azure Container Registry HOT 2
- Add support of registry.k8s.io type in cache rules HOT 1
- Rest api for get tags doesn respect n parameter (pagesize) HOT 2
- Scope Security/ Vulnerability scan to certain image tags only
- Catalog API only works with scope map * HOT 2
- Allow configuration of CORS headers for API access from web clients HOT 1
- Use Entra security principals with scope maps HOT 3
- [Docker Hub] ACR Cache error: too many requests to source registry for cache rule HOT 52
- connectivity_challenge_error grcsharedacr
- Unable to login into azure acr HOT 2
- Unable to login to container registry shazdevops HOT 1
- Auth Endpoint seems to require account parameter which is not part of the API Spec
- Cache elastic images HOT 1
- ACR Build with public access disabled HOT 1
- ACR streaming: failed to open remote file as tar file error HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from acr.