anderssonpeter / compressedstaticfiles Goto Github PK

I have a lot of files in my projects and running the gulp file currently takes about half an hour all while utilizing around 8% of my cpu capacity on this entirely cpu bound work.

Currently all tasks for a file are parallelized (e.g. I can compress brotli and gzip at the same time )and while that is a 2x speedup it would be better to parallelize file processing too (e.g. files a,b,c,d are processed at the same time doing brotli and gzip = 8x speed up etc).

I tried and failed to incorporate https://github.com/mafintosh/parallel-transform to allow processing more than one file at a time but I have very limited experience with Gulp and Node so that doesn't mean much.

Ratioanle

If StaticFileOptions has RequestPath path specified and static files are queried as {RequestPath}/path-to-a-static-file then cached versions are not served.

It says that originalFile doesn't exist

Test

        [TestMethod]
        public async Task Should_serve_the_compressed_file_if_a_compressed_version_exists_and_the_browser_supports_it()
        {
            // Arrange
            var builder = new WebHostBuilder()
                .Configure(app =>
                {
                    app.UseCompressedStaticFiles(new StaticFileOptions
                    {
                        RequestPath = "/assets"
                    });
                    app.Use(next =>
                    {
                        return async context =>
                        {
                            // this test should never call the next middleware
                            // set status code to 999 to detect a test failure
                            context.Response.StatusCode = 999;
                        };
                    });
                }).UseWebRoot(Path.Combine(Environment.CurrentDirectory, "wwwroot"));
            var server = new TestServer(builder);

            // Act
            var client = server.CreateClient();
            client.DefaultRequestHeaders.Add("Accept-Encoding", "br, gzip");
            var response = await client.GetAsync("/assets/i_also_exist_compressed.html");
            var content = await response.Content.ReadAsStringAsync();

            // Assert
            response.StatusCode.Should().Be(200);
            content.Should().Be("br");
            response.Content.Headers.TryGetValues("Content-Type", out IEnumerable<string> contentTypeValues);
            contentTypeValues.Single().Should().Be("text/html");
        }

Hello,
now that i have replaced static file middleware with yours and removed responsecompression middleware, how do you suggest we do compression on fly for dynamic resources embeded in dll files (e.g. RCL in my case).

thanks

https://docs.nuget.org/create/creating-and-publishing-a-package#automatically-displaying-a-readme.txt-file-during-package-installation

Hello,

i have my own custom implementation for image resizing/compression that compress(or resize or change formats) images on fly and save them for future use, for other static files i found your solution very useful, so if possible can you add a parameter to configuration to disable certain lookups (in my case images).

regards

Hello, when i am using the compressed middleware i am getting this issue on trying to access embedded files related to #34

info: CompressedStaticFiles.CompressedAlternativeFileProvider[1]
      Sending file. Request file: '/embedbackend/plugins/global/plugins.bundle.css'. Served file: '/embedbackend/plugins/global/plugins.bundle.css.gz'. Original file size: 567386. Served file size: 72449
fail: Microsoft.AspNetCore.Server.Kestrel[13]
      Connection id "0HMHK8J2D283R", Request id "0HMHK8J2D283R:00000009": An unhandled exception was thrown by the application.
      System.NullReferenceException: Object reference not set to an instance of an object.
         at CompressedStaticFiles.CompressedAlternativeFile.Prepare(IContentTypeProvider contentTypeProvider, StaticFileResponseContext staticFileResponseContext)
         at CompressedStaticFiles.CompressedStaticFileMiddleware.<>c__DisplayClass6_0.<InitializeStaticFileOptions>b__0(StaticFileResponseContext context)
         at Microsoft.AspNetCore.StaticFiles.StaticFileContext.ApplyResponseHeaders(Int32 statusCode)
         at Microsoft.AspNetCore.StaticFiles.StaticFileContext.SendAsync()
         at Microsoft.AspNetCore.StaticFiles.StaticFileContext.ServeStaticFile(HttpContext context, RequestDelegate next)
         at Ultra.ImageTools.Services.Middleware.ImageResizerMiddleware.InvokeAsync(HttpContext context) in D:\Projects\Web\dotnet\UltraWeb\Net5\Ultra.WebHost\Ultra.ImageTools\Services\Middleware\ImageResizerMiddleware.cs:line 106
         at Microsoft.AspNetCore.Builder.Extensions.MapMiddleware.Invoke(HttpContext context)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
[23:57:59 ERR] Connection id "0HMHK8J2D283R", Request id "0HMHK8J2D283R:00000009": An unhandled exception was thrown by the application.
System.NullReferenceException: Object reference not set to an instance of an object.
   at CompressedStaticFiles.CompressedAlternativeFile.Prepare(IContentTypeProvider contentTypeProvider, StaticFileResponseContext staticFileResponseContext)
   at CompressedStaticFiles.CompressedStaticFileMiddleware.<>c__DisplayClass6_0.<InitializeStaticFileOptions>b__0(StaticFileResponseContext context)
   at Microsoft.AspNetCore.StaticFiles.StaticFileContext.ApplyResponseHeaders(Int32 statusCode)
   at Microsoft.AspNetCore.StaticFiles.StaticFileContext.SendAsync()
   at Microsoft.AspNetCore.StaticFiles.StaticFileContext.ServeStaticFile(HttpContext context, RequestDelegate next)   

as i checked the middleware doesn't handle the embedded file headers correctly, hence pass it on to next one so is the problem with this issue. Is there any way to provide it

Hello,
i think this feature will be helpful in case a static file is not already compressed on server, we can run a DI based or prefixed file minimizer (css/js minification through nuglify) & compress it for future usage when such files are not already present.
If you would like, i can make a PR on the same too.
thanks

I can see this library likely works great for Blazor Server, but could it be extended to add compressed static resource support in Blazor WASM applications?

Hi. I try using your package to get gziped js and css files.

I use webpack to gzip files in wwwroot folder. So, when I try to load page, browser takes static gzip files without any extra headers and can't use this files as styles and scripts.

Have I done something wrong? I can't understand this :(

Add support for .NETFramework 4.5.1 in nuget package

Whenever path contains ":" character, exception is thrown:

GET https://localhost/testtest - 404
GET https://localhost/test:test - 500

System.NotSupportedException

The given path's format is not supported.

I even tried to use StaticFileOptions to limit request path to certain folder, still, the same result.

Write unit tests that ensures that everything works as intended.

On ASP.NET Core 3.1, app.UseCompressedStaticFiles() fails.

Current workaround:

• Copy source files( /src/CompressedStaticFiles/*.cs ) into my ASP.NET Core 3.1 project
• Replace "IHostingEnvironment" with "IWebHostEnvironment"

and it seems working fine.

Here is the stack trace of "ASP.NET Core 3.1 with Razor Pages" project template with just replacing app.UseStaticFiles() to app.UseCompressedStaticFiles() .

System.MissingMethodException
HResult=0x80131513
Message=Method not found: 'Void Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware..ctor(Microsoft.AspNetCore.Http.RequestDelegate, Microsoft.AspNetCore.Hosting.IHostingEnvironment, Microsoft.Extensions.Options.IOptions1<Microsoft.AspNetCore.Builder.StaticFileOptions>, Microsoft.Extensions.Logging.ILoggerFactory)'. Source=CompressedStaticFiles スタック トレース: 場所 CompressedStaticFiles.CompressedStaticFileMiddleware..ctor(RequestDelegate next, IHostingEnvironment hostingEnv, IOptions1 staticFileOptions, ILoggerFactory loggerFactory)
場所 System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
場所 System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) (//src/System.Private.CoreLib/src/System/Reflection/RuntimeConstructorInfo.cs):行 440
場所 Microsoft.Extensions.Internal.ActivatorUtilities.ConstructorMatcher.CreateInstance(IServiceProvider provider) (//.packages/microsoft.extensions.activatorutilities.sources/3.1.2-servicing.20067.6/contentFiles/cs/netstandard1.0/ActivatorUtilities.cs):行 406
場所 Microsoft.Extensions.Internal.ActivatorUtilities.CreateInstance(IServiceProvider provider, Type instanceType, Object[] parameters) (//.packages/microsoft.extensions.activatorutilities.sources/3.1.2-servicing.20067.6/contentFiles/cs/netstandard1.0/ActivatorUtilities.cs):行 90
場所 Microsoft.AspNetCore.Builder.UseMiddlewareExtensions.<>c__DisplayClass4_0.b__0(RequestDelegate next) (//src/Http/Http.Abstractions/src/Extensions/UseMiddlewareExtensions.cs):行 92
場所 Microsoft.AspNetCore.Builder.ApplicationBuilder.Build() (//src/Http/Http/src/Builder/ApplicationBuilder.cs):行 103
場所 Microsoft.AspNetCore.Hosting.GenericWebHostService.d__31.MoveNext() (//src/Hosting/Hosting/src/GenericHost/GenericWebHostedService.cs):行 104
場所 System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() (//src/System.Private.CoreLib/shared/System/Runtime/ExceptionServices/ExceptionDispatchInfo.cs):行 63
場所 System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) (//src/System.Private.CoreLib/shared/System/Runtime/CompilerServices/TaskAwaiter.cs):行 180
場所 System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) (//src/System.Private.CoreLib/shared/System/Runtime/CompilerServices/TaskAwaiter.cs):行 151
場所 System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult() (//src/System.Private.CoreLib/shared/System/Runtime/CompilerServices/TaskAwaiter.cs):行 473
場所 Microsoft.Extensions.Hosting.Internal.Host.d__9.MoveNext() (//src/Hosting/Hosting/src/Internal/Host.cs):行 50
場所 System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() (//src/System.Private.CoreLib/shared/System/Runtime/ExceptionServices/ExceptionDispatchInfo.cs):行 63
場所 System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) (//src/System.Private.CoreLib/shared/System/Runtime/CompilerServices/TaskAwaiter.cs):行 180
場所 System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) (//src/System.Private.CoreLib/shared/System/Runtime/CompilerServices/TaskAwaiter.cs):行 151
場所 System.Runtime.CompilerServices.TaskAwaiter.GetResult() (//src/System.Private.CoreLib/shared/System/Runtime/CompilerServices/TaskAwaiter.cs):行 106
場所 Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.d__4.MoveNext() (//src/Hosting/Abstractions/src/HostingAbstractionsHostExtensions.cs):行 62
場所 System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() (//src/System.Private.CoreLib/shared/System/Runtime/ExceptionServices/ExceptionDispatchInfo.cs):行 63
場所 Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.d__4.MoveNext() (//src/Hosting/Abstractions/src/HostingAbstractionsHostExtensions.cs):行 74
場所 System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() (//src/System.Private.CoreLib/shared/System/Runtime/ExceptionServices/ExceptionDispatchInfo.cs):行 63
場所 System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) (//src/System.Private.CoreLib/shared/System/Runtime/CompilerServices/TaskAwaiter.cs):行 180
場所 System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) (//src/System.Private.CoreLib/shared/System/Runtime/CompilerServices/TaskAwaiter.cs):行 151
場所 System.Runtime.CompilerServices.TaskAwaiter.GetResult() (//src/System.Private.CoreLib/shared/System/Runtime/CompilerServices/TaskAwaiter.cs):行 106
場所 Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host) (/_/src/Hosting/Abstractions/src/HostingAbstractionsHostExtensions.cs):行 49
場所 WebApplication1.Program.Main(String[] args) (C:\Users***\Documents\Visual Studio 2019\Projects\CompressedStaticFiles\WebApplication1\Program.cs):行 16

It would be fantastic to be able to call something like return CompressedStaticFileResult("some/file.html"); from a webapi controller method or return Results.CompressedStaticFile("some/file.html"); from a minimal API endpoint to leverage the conditional request and compressed file serving logic in custom endpoints. This would allow things like serving a 401/403 page with the appropriate status code.

I might be able to have a crack at this myself, I think it would look very similar to the CompressedStaticFileMiddleware class, except it would implement IActionResult and defer to PhysicalFileResult instead of StaticFileMiddleware. Would you consider accepting a PR?

Just out of curiosity, I was wondering why do you use travis and AppVeyor and not only just one CI build service?

In file https://github.com/AnderssonPeter/CompressedStaticFiles/blob/master/src/CompressedStaticFiles/CompressedStaticFileMiddleware.cs

These lines have a potential issue:

 var originalPrepareResponse = staticFileOptions.Value.OnPrepareResponse;
            staticFileOptions.Value.OnPrepareResponse = ctx =>
            {
                originalPrepareResponse(ctx);

The above code essentially "monkey-patches" the OnPrepareResponse. By keeping a reference to the original & adding some extra code.

The problem with this approach is that .net core instantiates a new Middleware instance on each request. However, the options.value passed to the middleware constructor is usually a singleton (depends on the programmer).

This means the 1st request will work fine, but the 2nd request will monkey-patch the already monkey-patched method. The double-monkey-patched method still works, it just wastes cpu time running the extra code 2x. With every request this overhead gets worse until a stack overflow exception eventually happens and the app pool resets.

One potential solution would be to have the constructor to CompressedStaticFileMiddleware.cs clone the original StaticFileOptions before monkey-patching.

In my project, I can see that alternative encodings for images are correctly served (for example, PNG is requested, WEBP is returned).

But this doesn't seem to be the case for static .js, .css, .svg files etc. compressed via Brotli and GZip, at least when compressed and named as in the example Gulpfile.

For example, I'm serving app.js, app.js.br, and app.js.gz. In browser dev tools, a request/response for app.js shows a blank Content-Encoding value, whereas I expect it to be "br" or "gz". Am I misunderstanding the process? Or perhaps there's an extra initialization step necessary to enable Brotli and GZip?

(Note: initially I didn't even deploy app.js, since GZip is universally supported, so I didn't think the plain app.js file would ever be necessary. But then I observed that requests for app.js resulted in a 404, whereas I expected CompressedStaticFiles to intervene and return either the gz or br encoding, both of which were available and supported by the client. Depending on the outcome of this issue, I might open a separate issue on the necessity of unencoded resources.)

If you try to pass in StaticFileOptions that include setting the caching headers, they don't work since a new delegate is created. For example, if you pass in

app.UseCompressedStaticFiles(new StaticFileOptions
{
  OnPrepareResponse = ctx =>
  {
    if (env.IsDevelopment()) return;
    const int durationInSeconds = 60 * 60 * 24;
    ctx.Context.Response.Headers[HeaderNames.CacheControl] = "public,max-age=" + durationInSeconds;
    ctx.Context.Response.Headers[HeaderNames.Vary] = HeaderNames.AcceptEncoding;
  }
});

It won't work because it is overwritten in CompressedStaticFileMiddleware.cs

staticFileOptions.Value.OnPrepareResponse = ctx =>
{
  foreach (var compressionType in compressionTypes.Keys)
  {
    var fileExtension = compressionTypes[compressionType];
    if (ctx.File.Name.EndsWith(fileExtension, StringComparison.OrdinalIgnoreCase))
    {
      string contentType = null;
      if (contentTypeProvider.TryGetContentType(ctx.File.PhysicalPath.Remove(ctx.File.PhysicalPath.Length - fileExtension.Length, fileExtension.Length), out contentType))
        ctx.Context.Response.ContentType = contentType;
      ctx.Context.Response.Headers.Add("Content-Encoding", new[] { compressionType });
    }
  }
};

The original value needs to be stored and invoked.

I created a pull request that addresses this issue #6 Adding ability to execute OnPrepareResponse that is passed in

I've just had issues trying to use this when running inside Azure. I had some brotli files and they would not get served.

I've tracked the issue down to missing content types. Pull request coming up.

I also noticed that you don't need to do app.UseStaticFiles() either. The compressed static files uses these by default.