Code Monkey home page Code Monkey logo

Comments (4)

FZambia avatar FZambia commented on June 1, 2024

Hello @richelieu-yang

How did you enable GZIP? Share the entire configuration of Centrifugo and your proxy, steps to reproduce

from centrifugo.

richelieu-yang avatar richelieu-yang commented on June 1, 2024

@FZambia Thanks for help and here are details.

entire configuration of Centrifugo

single node
config.yaml.zip

a proxy service(127.0.0.1:80) which proxy request to centrifugo(127.0.0.1:8000)

centrifugo-cluster-proxy.zip
enable gzip by github.com/gin-contrib/gzip

from centrifugo.

FZambia avatar FZambia commented on June 1, 2024

I just tried enabling gzip in Nginx and point it to http_stream endpoint. It worked:

Screenshot 2024-05-13 at 18 23 08

Nginx config I used:

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
  worker_connections 1024;
}

http {
  upstream centrifugo {
    server host.docker.internal:8000;
  }

  server {
    listen 80;

    server_name localhost 127.0.0.1;

    location /connection/http_stream {
      proxy_pass          http://centrifugo;
      proxy_http_version  1.1;
      proxy_redirect      default;
      proxy_set_header    Upgrade $http_upgrade;
      proxy_set_header    Connection "upgrade";
      proxy_set_header    Host $host;
      proxy_set_header    X-Real-IP $remote_addr;
      proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header    X-Forwarded-Host $server_name;

      gzip            on;
      gzip_min_length 1;
      gzip_proxied    expired no-cache no-store private auth;
      gzip_types      text/plain;
    }
  }
}

So I suppose the issue is not with Centrifugo itself here, but with the custom proxy you are using. Please note that http stream in Centrifugo is a persistent HTTP connection with chunked transfer encoding. So make sure your proxy does not buffer responses and supports chunked transfer.

from centrifugo.

FZambia avatar FZambia commented on June 1, 2024

@richelieu-yang If it helps, here is a snippet in Go of reverse proxy which GZIPs chunked HTTP stream.

It works, but I just made proof of concept - ❗❗❗ totally untested ❗❗❗

package main

import (
	"compress/gzip"
	"context"
	"errors"
	"io"
	"log"
	"net/http"
	"net/http/httputil"
	"net/url"
	"strings"
)

func main() {
	targetURL := "http://localhost:8000" // Replace with your backend server URL
	proxyURL, err := url.Parse(targetURL)
	if err != nil {
		panic(err)
	}

	proxy := httputil.NewSingleHostReverseProxy(proxyURL)

	// Customize the Director function to handle request modifications
	proxy.Director = func(req *http.Request) {
		req.URL.Scheme = proxyURL.Scheme
		req.URL.Host = proxyURL.Host
		req.Host = proxyURL.Host
		req.Header.Add("Accept-Encoding", "gzip")
	}

	// Customize the ModifyResponse function to handle responses
	proxy.ModifyResponse = func(resp *http.Response) error {
		// Check if the response is already gzip-encoded
		if strings.Contains(resp.Header.Get("Content-Encoding"), "gzip") {
			return nil
		}

		// Remove Content-Length because the content length will change after gzip compression
		resp.Header.Del("Content-Length")
		// Set the Content-Encoding header to indicate that the response is gzip-compressed
		resp.Header.Set("Content-Encoding", "gzip")
		// Add the Vary header to indicate that the response varies based on the Accept-Encoding header
		resp.Header.Add("Vary", "Accept-Encoding")

		pr, pw := io.Pipe()
		gw := gzip.NewWriter(pw)

		// Store the original body to close it later
		origBody := resp.Body
		resp.Body = io.NopCloser(pr)

		// Use a goroutine to compress the response body and write to the pipe
		go func() {
			defer origBody.Close()
			defer pw.Close()
			defer gw.Close()

			buf := make([]byte, 1024)
			for {
				n, err := origBody.Read(buf)
				if n > 0 {
					if _, err := gw.Write(buf[:n]); err != nil {
						log.Printf("Error writing to gzip writer: %v", err)
						return
					}
					if err := gw.Flush(); err != nil {
						log.Printf("Error flushing gzip writer: %v", err)
						return
					}
				}
				if err != nil {
					if errors.Is(err, io.EOF) {
						break
					}
					if !errors.Is(err, context.Canceled) {
						log.Printf("Error reading from response body: %v", err)
					}
					return
				}
			}
		}()

		return nil
	}

	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		proxy.ServeHTTP(w, r)
	})

	log.Println("Starting proxy server on :9000")
	if err := http.ListenAndServe(":9000", nil); err != nil {
		log.Fatalf("Error starting server: %v", err)
	}
}

func flushResponse(w http.ResponseWriter) {
	if flusher, ok := w.(http.Flusher); ok {
		flusher.Flush()
	}
}

type gzipResponseWriter struct {
	io.Writer
	http.ResponseWriter
}

func (w gzipResponseWriter) Write(b []byte) (int, error) {
	n, err := w.Writer.Write(b)
	if err == nil {
		flushResponse(w.ResponseWriter)
	}
	return n, err
}

func newGzipResponseWriter(w http.ResponseWriter) http.ResponseWriter {
	gw := gzip.NewWriter(w)
	return gzipResponseWriter{Writer: gw, ResponseWriter: w}
}

from centrifugo.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.