You're probably aware of ghost.org & https://github.com/TryGhost

Cheers!
Pascal

After copying the configuration and changing just the port where apache is running, I checked the syntax and varnishd is returning this error:

Message from VCC-compiler:
Expected return action name.
('input' Line 269 Pos 14)
return (miss);
-------------####--
Running VCC-compiler failed, exited with 2
VCL compilation failed

Can you please help?

The version of Varnish I am running is:

varnishd -V
varnishd (varnish-4.0.2 revision bfe7cd1)
Copyright (c) 2006 Verdens Gang AS
Copyright (c) 2006-2014 Varnish Software AS

Thanks in advance!!!

On Nginx cache, you can do this to avoid keep on caching when you add something to the cart in Woocommerce :

Skip cache for WooCommerce query string

if ( $arg_add-to-cart != "" ) { 
      set $skip_cache 1;
}

Skip cache when WooCommerce cart is not empty

if ( $cookie_woocommerce_items_in_cart != "0" ) {   
    set $skip_cache 1;
}

How can you do this in Varnish ? With this config, wich was made specifically for Wordpress + Woocommerce, when you add something to cart, cache is not disabled store wide so you get a mix of cached and non cache pages, cart gets emptied : http://pastebin.com/TuYeMtUZ

I stumbled on an issue today where querysort was corrupting the URL and truncating them.

I'm not super familiar with the VCL language. I was wondering if there was a way we could check that querysort isn't corrupting a URL before replacing it. Maybe something simple like a length check?

Hi,

I've set up this vcl template. Thank you very much, its very useful.

Now, my setup is (client) --https--> apache 443 --proxy pass --> varnish 80 --> apache 8080

so apache does the ssl termination and proxies into varnish.
varnish then requests apache non https port

this works and requires in apache non https vhost:
SetEnvIf X_FORWARDED_PROTO "^https$" HTTPS=on

on vhost https I have:
ProxyPreserveHost On
ProxyPass / http://127.0.0.1:80/
#ProxyPassReverse / http://localhost:80/
RequestHeader set X-Forwarded-Port "443"
RequestHeader set X-Forwarded-Proto "https"

and for wordpress users, in wp-config.php i have:
define('FORCE_SSL_ADMIN', true);
// in some setups HTTP_X_FORWARDED_PROTO might contain
// a comma-separated list e.g. http,https
// so check for https existence
if (strpos($_SERVER['HTTP_X_FORWARDED_PROTO'], 'https') !== false)
$_SERVER['HTTPS']='on';

before loading wp-settings.php

Now, varnish should have a way of redirecting to https if client is not using it.. something like:

if(client.ip != "127.0.0.1"){
set req.http.x-redir = "https://" + req.http.host + req.url;
return(synth(850, "Moved permanently"));
}

and
sub vcl_synth {
if (resp.status == 850) {
set resp.http.Location = req.http.x-redir;
set resp.status = 302;
return (deliver);
}
}

I took this code from somewhere on the internet, but i'm afraid its not quite right..

Would it be possible for you to implement such feature?

Hello,

I tried using the configuration template, but I get following error:

Okt 05 16:42:45 litt varnishd[18997]: Message from VCC-compiler:
Okt 05 16:42:45 litt varnishd[18997]: Expected return action name.
Okt 05 16:42:45 litt varnishd[18997]: ('input' Line 273 Pos 15)
Okt 05 16:42:45 litt varnishd[18997]: return (miss);
Okt 05 16:42:45 litt varnishd[18997]: --------------####--
Okt 05 16:42:45 litt varnishd[18997]: Running VCC-compiler failed, exited with 2
Okt 05 16:42:45 litt varnishd[18997]: VCL compilation failed
Can you please investigate.
I am using varnish-4.0.4 revision 386f712 on CentOS 7 x64

Thank you for your work, cheers!

To use this, call the following from anywhere in vcl_recv: error 720 "http://host/new.html"

In Varnish 4

To use this, call the following from anywhere in vcl_recv: return (synth(720, "http://host/new.html"));

Hi,

It is not a bug, just a question. :)
Does the configuration store cache dynamic content for 6 hours?

Thank you,

Hi,
thanks for sharing your configuration! i just started using it with a Drupal 8 site an everything seems to be working great except for the comments urls, what i mean is for example, a user posts a comment, an admin goes to the admin/content/comments url to see the list of approved/un-approved comments and clicks on any of the links i.e https://example.com/comment/19#comment-19, the server returns an error 500, when visiting the page without using the varnish server but pointing directly to the web-server (apache), the page loads normally with no problems

hope you can provide some guide to fix this issue.
thank you!

i think there is a slight issue where the vcl_purge does return(restart) - it should probably do so only in case req.method != "PURGE" - for the CTRL-F5 case ... because it probably doesn't make much sense to restart a purge. And since in the other part that invokes purge there is no check for restarts, nor for the header, this causes errors :)

From the 4.0 docs it doesn't show that as a valid termination keyword, and the 3.0 docs say it's to be deprecated.

I'm guessing that the fetch keyword is still refreshing the content like the 3.0 documentation says.

The vcl Very usefully, thanks. now I use drupal 8, is possible to support drupal 8?

On Varnish 4.0, this code seems to duplicate the client IP into the header so it appears twice.

https://github.com/mattiasgeniar/varnish-4.0-configuration-templates/blob/master/default.vcl#L58

Is it now a default feature of Varnish?

Is there a default.vcl for wordpress?

Hi,

first of all, thank you for putting this together.

I'm using the untouched template from this repo to set up a fresh Varnish 4.1 installation.

My webserver is running on port 80. When browsing to port 6081 I see this in the output of varnishlog

-   FetchError     Director vdir returned no backend
-   FetchError     No backend

Performing curl 127.0.0.1 on the terminal of the server does return the html I'm expecting.

Do you have any clue why the Director vdir returned no backend message is given and how it can be solved?

Hi,

Using the VCL provided for a Drupal 7 installation seems to be working fine but when enabling multi language support ( just enable an extra language ) Varnish seems to stop caching pages, so i'm wondering if the VCL configuration provided is taking Language variations into consideration for caching? if you ever get a HIT on varnish it seems that cache works different per Browser.
There are multiple modules on Drupal 7 that are related to Languages that set cookies on the backed like Multilink (https://www.drupal.org/project/multilink) for caches pages in Drupal 7 so not sure if this is being handled as well on the provided config?.

i've added the following directive to the vcl_hash() routine to see if that might help but seems to be doing nothing.

sub vlc_hash(){
#...
#Uncomment if different languages are served at the same URL.
if( req.http.Accept-Language ) {
  hash_data(req.http.Accept-Language);
}
}

Thanks for your attention and clarifications.

I've configured a server to serve grace cached content to users like so, as described in Varnish docs.

sub vcl_hit {
   if (obj.ttl >= 0s) {
       // A pure unadultered hit, deliver it
       return (deliver);
   }
   if (obj.ttl + obj.grace > 0s) {
       // Object is in grace, deliver it
       // Automatically triggers a background fetch
       return (deliver);
   }
   // fetch & deliver once we get the result
   return (fetch);
}

I'm noticing that when a backend request is made in the grace period (e.g. if the TTL is 120 and the grace cache is 1 hour, 121 seconds after the initial request was cached) and the back-end is serving 500s, that request is served from the cache, but subsequent requests return 500s.

I've implemented this sort of logic to prevent cache busting when the backend serves 500s

sub vcl_backend_response {
    if (beresp.status == 500 || beresp.status == 502 || beresp.status == 503 || beresp.status == 504) {
        return (abandon);
    }
    # ... more code
}

Using this configuration i have managed to stream a TV channel but it got stack (freezes) for 60 seconds and then again automatically resume the stream with new segment and then again freezes for another 60 second and continuing the process..
Couldnt able to find any solution for this !!!
Any guidance would be appreciable...

Hi I might be the only one using Varnish 4 so far but... I wanted to let you know that on centos 6.5 your vcl is failing I enabled the debug mode and it shows the following:

Message from VCC-compiler:
Expected an action, 'if', '{' or '}'
('input' Line 238 Pos 6)
purge;
-----#####-

Running VCC-compiler failed, exit 1

VCL compilation failed

I removed the purge and it worked, is this happening to you as well?

This template is dangerous !!!

The following code allow anyone to purge all you cache and empty you "grace cache" if you backends are down.

I've been using your configuration for a while with WordPress sites and it's been great! I'm slowly moving everything to Xenial and I've started running into issues where pages are half rendered. If I remove Varnish, the backend returns everything correctly.

I've had this issue before. But it's more consistent now. I managed to dig down to inside varnishlog and find this:

-   Fetch_Body     4 eof stream
-   FetchError     Resource temporarily unavailable
-   FetchError     eof socket fail

When this happens, the request is incomplete. Yet Varnish returns it anyways. No retries are attempted.

Is there a way to make a check for these FetchError and trigger a retry? I tried using vcl_backend_error and doing return (retry);, but that didn't work. Is there a way to check for this in vcl_backend_response? I've looked around, but information on this is sparse (or my GoogleFu failed me lol).

I replace default default.vcl by your file.
Then I visit website, it says:

Error 503 Backend fetch failed
Backend fetch failed
Guru Meditation:
XID: 3
Varnish cache server

How to deal with this?

My webserver architecture is: Haproxy:80 => Varnish:6081 => Nginx:82
Before that, I use: Haproxy:80 => Nginx:82 (working fine)

Varnish delivers the wrong content for a number of page. I can't lay my finger on the pattern. When you do a CTRL-F5 the right content is returned. I have been searching on the internet for a solution to this, but I have not been able to find one.

Set-up:
1 core varnish server debian 7.8, 16Gb disk, 1G mem
8 core content server debian 7.8, 64Gb disk, 64Gb mem, plesk 12, php-fpm, nginx, APC

Website:
Joomla with user login on the front

Also running an ab -n 1000 -c 50 .... while checking the "varnishtop -c -i ReqURL" and "varnishtop -b -i BereqURL" in two separate putty windows shows that the requests from the front are send to the backend at a rate of 2:1. i.e. for every two requests on the front one request seems to this the back. I would have expected that there would be only one or just a handful or requests to the backend

As related to #34.

using this config there are a couple of pages in wordpress admin like widgets and rss importer admin (both of these well long loader / possible large pages before). That throw the error ERR_INCOMPLETE_CHUNKED_ENCODING in web browser and fail to load.

I'm 100% it is this varnish config file as I have been through everything else nginx/apache/wordpress/php-fpm etc. deactivated all modules, replaced all configs, checked permissions etc etc... It started working again right If I replace this varnish config with a default basic one with just a backend and a couple of lines in recv everything works, no errors.

So what is causing it to throw chunk errors when loading wp-admin/widgets.php with this config.
In the meantime I will comment out bits line by line until I find what part of vcl is causing the fail...

It seems something is wrong with ajax add to cart woocommerce. Although POST requests are not cached item cannot be added. Any suggestions?

Thank you

Hi,

Sometime I am getting URL with http://site.com:8080
My config is Varnis -> Nginx -> PHP-FPM
Nginx has this string to help port_in_redirect off;
However, 8080 is still appearing. Could you help me to understand where and what needs to change, please?

I have two .vlc's, one that I am already using and the one that you guys have coded.

My backend is @ 127.0.0.0, port 8081. I just change the ip and port configuration to try out your .vlc and I get a 503 error "can't fecth from backend" .... The backend configuration is identical to my old config :

My config

vcl 4.0;

import std;
import directors;

backend server1 {
  .host = "127.0.0.1";             # IP or Hostname of backend
  .port = "8081";                  # Port Apache or whatever is listening
  .max_connections = 800;          # That's it
  .first_byte_timeout = 300s;      # How long to wait before we receive a first byte from our backend?
  .connect_timeout = 300s;         # How long to wait for a backend connection?
  .between_bytes_timeout = 300s;   # How long to wait between bytes received from our backend?
}

# Only allow purging from specific IPs
acl purge {
    "localhost";
    "127.0.0.1";
    "domain.com";
}

sub vcl_init {
  # Called when VCL is loaded, before any requests pass through it. Typically used to initialize VMODs.
  new vdir = directors.round_robin();
  vdir.add_backend(server1);
}

And yours

vcl 4.0;
# Based on: https://github.com/mattiasgeniar/varnish-4.0-configuration-templates/blob/master/default.vcl

import std;
import directors;

backend server1 { # Define one backend
  .host = "127.0.0.1";    # IP or Hostname of backend
  .port = "8081";           # Port Apache or whatever is listening
  .max_connections = 300; # That's it

  .probe = {
    #.url = "/"; # short easy way (GET /)
    # We prefer to only do a HEAD /
    .request =
      "HEAD / HTTP/1.1"
      "Host: localhost"
      "Connection: close"
      "User-Agent: Varnish Health Probe";

    .interval  = 5s; # check the health of each backend every 5 seconds
    .timeout   = 1s; # timing out after 1 second.
    .window    = 5;  # If 3 out of the last 5 polls succeeded the backend is considered healthy, otherwise it will be marked as sick
    .threshold = 3;
  }

  .first_byte_timeout     = 300s;   # How long to wait before we receive a first byte from our backend?
  .connect_timeout        = 5s;     # How long to wait for a backend connection?
  .between_bytes_timeout  = 2s;     # How long to wait between bytes received from our backend?
}

acl purge {
  # ACL we'll use later to allow purges
  "localhost";
  "127.0.0.1";
  "::1";
}

/*
acl editors {
  # ACL to honor the "Cache-Control: no-cache" header to force a refresh but only from selected IPs
  "localhost";
  "127.0.0.1";
  "::1";
}
*/

sub vcl_init {
  # Called when VCL is loaded, before any requests pass through it.
  # Typically used to initialize VMODs.

  new vdir = directors.round_robin();
  vdir.add_backend(server1);
  # vdir.add_backend(server...);
  # vdir.add_backend(servern);
}

Is it too late and I am already blind or there is something I am missing ?

Thanks

Hi,

Looking at this code https://github.com/mattiasgeniar/varnish-4.0-configuration-templates/blob/master/default.vcl#L347

I thought if I shut down my backend, varnish should pick up for 6 hours

I get Error 503 Backend fetch failed

Hello i've just copy and paste your configuration and i've just change backend ip/port.
As of info : My backend isn't in the same server where varnish is installed apache is in a different server and varnish is in a different server.
I'm asking here as i'm very new with varnish this is my first time working with it and i hope someone from here can gave some help about this.

So as i said i've just copy and paste your configuration but idk why i always get this output

< HTTP/1.1 200 OK
< Date: Thu, 02 Nov 2017 21:10:56 GMT
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< Pragma: no-cache
< Set-Cookie: PHPSESSID=ko7urmd74ar9v6ej6ghambs1h0; path=/; HttpOnly
< Content-Type: text/html; charset=UTF-8
< Age: 0
< X-Cache: MISS
< X-Cache-Hits: 0
< Accept-Ranges: bytes
< Content-Length: 8229
< Connection: keep-alive

Can someone help about this?
Thanks.

Hi @mattiasgeniar,

I have tried to use default.vlc template in varnish-4.0-configuration-templates. It seem like it is not working. Is their any configuration I need to do in the template in order to make it work. Please let me know?

My drupal 7 setting are
$conf['cache_backends'][] = 'sites/all/modules/varnish/varnish.cache.inc'; $conf['cache_class_external_varnish_page'] = 'VarnishCache'; $conf['reverse_proxy'] = true; $conf['reverse_proxy_addresses'] = array('172.31.10.68'); $conf['reverse_proxy_header'] = 'HTTP_X_FORWARDED_FOR'; $conf['cache_lifetime'] = 0; $conf['page_cache_maximum_age'] = 21600; $conf['page_cache_invoke_hooks'] = false;

The reply I got from curl -I http://stage.mysite.com/ is
HTTP/1.1 200 OK Date: Fri, 08 Jul 2016 05:27:39 GMT X-Content-Type-Options: nosniff Cache-Control: max-age=0, no-cache, must-revalidate, post-check=0, pre-check=0 X-Content-Type-Options: nosniff Content-Language: en X-UA-Compatible: IE=edge Vary: Accept-Encoding X-Mod-Pagespeed: 1.9.32.14-0 Content-Type: text/html; charset=utf-8 Age: 0 X-Cache: MISS X-Cache-Hits: 0 Connection: keep-alive

When I used configuration from NITEMAN. It works but issue is that its not setting max-age properly

HTTP/1.1 200 OK Date: Fri, 08 Jul 2016 05:06:32 GMT Server: Apache X-Content-Type-Options: nosniff Cache-Control: max-age=0, no-cache, must-revalidate, post-check=0, pre-check=0 X-Content-Type-Options: nosniff Content-Language: en X-UA-Compatible: IE=edge X-Generator: Drupal 7 (http://drupal.org) Link: <http://stage.mysite.com/>; rel="canonical",<http://stage.mysite.com/>; rel="shortlink" Vary: Accept-Encoding X-Mod-Pagespeed: 1.9.32.14-0 Content-Type: text/html; charset=utf-8 X-Varnish-Cacheable: NO:Not Cacheable X-Varnish: 163872 Age: 0 Via: 1.1 varnish-v4 X-Varnish-Cache: MISS X-Varnish-Server: varnish Connection: keep-alive

Please help me I am new to this.

Kind regards,
Javed Gardezi

I built a RPM of vmod-named for centOS6. When trying to reload varnishd I'm getting an "Expected an action" error: http://pastebin.com/0b6BpZ7f , rpm seems to properly install libvmod_named.so as I'm able to load it.

I'm using a subroutine in vcl_recv with your code to remove the utm_ parameters from the URL but seems not to be working as you can guess by the logs of the backend:

127.0.0.1 - Anonymous [13/Mar/2017:16:26:41 -0200] "GET /politica/eduardo-fagnani-o-deficit-da-previdencia-e-uma-pos-verdade/@@nitf_galleria?ajax_load=1489433196711 HTTP/1.1" 200 8587 "https://www.cartacapital.com.br/politica/eduardo-fagnani-o-deficit-da-previdencia-e-uma-pos-verdade?utm_campaign=CartaCapital+Newsletter&utm_content=Eduardo+Fagnani%3A+%22O+%27d%C3%A9ficit%27+da+Previd%C3%AAncia+%C3%A9+uma+p%C3%B3s-verdade%22+%E2%80%94+CartaCapital+%282%29&utm_medium=email&utm_source=EmailMarketing&utm_term=Newsletter" "Mozilla/5.0 (Linux; Android 5.0.2; XT1078 Build/LXB22.99-24.12) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.85 Mobile Safari/537.36"
127.0.0.1 - Anonymous [13/Mar/2017:16:35:30 -0200] "GET /internacional/estado-islamico-assume-autoria-do-ataques-em-paris-4536.html HTTP/1.1" 200 64271 "http://www.cartacapital.com.br/internacional/estado-islamico-assume-autoria-do-ataques-em-paris-4536.html?utm_content=bufferf651c&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer" "Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:11.0) Gecko/20100101 Firefox/11.0"

this is the whole code I'm using:

sub vcl_recv {
    set req.backend = app;
    call sanitize_url_parameters;
}

# Remove Google Analytics parameters from the URL
# https://github.com/mattiasgeniar/varnish-4.0-configuration-templates
sub sanitize_url_parameters {
    if (req.url ~ "(\?|&)(utm_campaign|utm_content|utm_medium|utm_source|utm_term)=") {
        set req.url = regsuball(req.url, "&(utm_campaign|utm_content|utm_medium|utm_source|utm_term)=([A-z0-9_\-\.%25]+)", "");
        set req.url = regsuball(req.url, "\?(utm_campaign|utm_content|utm_medium|utm_source|utm_term)=([A-z0-9_\-\.%25]+)", "?");
        set req.url = regsub(req.url, "(\?&|\?|&)$", "");
    }
}