varnishcache / pkg-varnish-cache Goto Github PK
View Code? Open in Web Editor NEWPackage building scripts for official Debian and Redhat packages of Varnish Cache.
Package building scripts for official Debian and Redhat packages of Varnish Cache.
Moved from varnishcache/varnish-cache#2006.
Doing a service varnish reload on ubuntu 16 fails with Management port disabled. $DAEMON_OPTS must contain '-T hostname:port' if the /etc/default/varnish is CRLF (dos/windows newlines) instead of LF (*nix newlines).
That happens if /etc/systemd/system/varnish.service points to ExecReload=/usr/share/varnish/reload-vcl.
Doing a vi /etc/default/varnish and :set ff=unix to convert, solves the problem.
CC @cherouvim
The rework introduced in a59b3bd has left out PIDFILE. Since NAME has been re-factored to SERVICE changing NAME in PIDFILE does the trick.
diff --git a/debian/varnish.init b/debian/varnish.init
index c169397..b2d332b 100644
--- a/debian/varnish.init
+++ b/debian/varnish.init
@@ -20,7 +20,7 @@ SERVICE="$(basename ${0})"
DESC="HTTP accelerator"
PATH=/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/sbin/varnishd
-PIDFILE=/run/$NAME.pid
+PIDFILE=/run/${SERVICE}.pid
test -x $DAEMON || { echo "${DAEMON} has no execute bit"; exit 0; }
Original issue: varnishcache/varnish-cache#2119
Varnish is awesome. It would work more nicely with RPM if the RPMs were signed. Please sign RHEL 7 RPMs from your repo (e.g., https://repo.varnish-cache.org/redhat/varnish-4.1.el6.rpm)
RPM signature available.
Workaround missing signatures:
rpm --nosignature -i https://repo.varnish-cache.org/redhat/varnish-4.1.el7.rpm
Varnish version: 4.1.x
OS: Ubuntu 16.04 x86_84
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.1 LTS"
NAME="Ubuntu"
VERSION="16.04.1 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.1 LTS"
VERSION_ID="16.04"
I am using the varnish-cache repository for Debian Jessie (as there is none specific for Ubuntu xenial yet, see #36).
$ cat /etc/apt/sources.list.d/varnish-cache.list
deb https://repo.varnish-cache.org/debian/ jessie varnish-4.1
Yesterday I updated varnish from 4.1.1-1 to 4.1.3-1~jessie on our test environment and shortly after our QA testers found out, that some http headers, Varnish was supposed to add, disappeared.
It turned out that the installation of the update has overwritten the systemd service config file located under /lib/systemd/system/varnish.service.
Due to the fact, that systemd does not read /etc/default/varnish anymore (see https://bugs.launchpad.net/ubuntu/+source/varnish/+bug/1573561), my daemon specific configuration changes (as malloc for example) were done directly in /lib/systemd/system/varnish.service.
Before the update:
[Unit]
Description=Varnish HTTP accelerator
Documentation=https://www.varnish-cache.org/docs/4.1/ man:varnishd
[Service]
Type=simple
LimitNOFILE=131072
LimitMEMLOCK=82000
ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :6081 -T localhost:6082 -f /etc/varnish/titan.vcl -S /etc/varnish/secret -s malloc,4096m
ExecReload=/usr/share/varnish/reload-vcl
ProtectSystem=full
ProtectHome=true
PrivateTmp=true
PrivateDevices=true
[Install]
WantedBy=multi-user.target
After the update:
[Unit]
Description=Varnish Cache, a high-performance HTTP accelerator
[Service]
Type=forking
# Maximum number of open files (for ulimit -n)
LimitNOFILE=131072
# Locked shared memory (for ulimit -l)
# Default log size is 82MB + header
LimitMEMLOCK=82000
# On systemd >= 228 enable this to avoid "fork failed" on reload.
#TasksMax=infinity
# Maximum size of the corefile.
LimitCORE=infinity
ExecStart=/usr/sbin/varnishd -a :6081 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m
ExecReload=/usr/share/varnish/reload-vcl
[Install]
WantedBy=multi-user.target
The problem? The update process asked whether or not to replace /etc/default/varnish (although irrelevant with systemd):
Configuration file '/etc/default/varnish'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
What would you like to do about it ? Your options are:
Y or I : install the package maintainer's version
N or O : keep your currently-installed version
D : show the differences between the versions
Z : start a shell to examine the situation
The default action is to keep your current version.
*** varnish (Y/I/N/O/D/Z) [default=N] ? N
But it didn't ask whether or not to replace /lib/systemd/system/varnish.service.
Please add the same behaviour to the package, ask whether or not to replace the existing /lib/systemd/system/varnish.service file.
We currently ship an init script/service file to start varnishlog. This is not enabled by default.
If someone enables it with default config, they will get a binary VSL dump in /var/log/varnish/varnish.log
which most likely will fill their hard drive within a day. (on my debian jessie system)
I'm assuming the original thought here was that server errors should be logged, so the sysadmin can handle them. I guess logging "all" traffic for a single URL could be another use case?
First of all, it is sub-optimal that the near-default config will fill the disk and stop everything. We need to handle that somehow. Sampling or token-bucket based rate limiting have been suggested before. (I don't see increasing logrotate interval as very viable.)
Are we really expecting that people use our tool to read the logs in /var/log/? Shouldn't they be in ASCII format by default, like every other log in there? (=> add -A to the default invocation)
We should also provide examples of VSL queries that can be useful. My quick hack is using RespStatus ~ "^(40[^4]|50.)"
but that is probably too wide for most systems.
(as in: /usr/bin/varnishlog -a -A -w /var/log/varnish/varnish.log -D -P /run/varnishlog/varnishlog.pid -q RespStatus ~ "^(40[^4]|50.)"
)
Input welcome.
[4.1] Update 4.1.4-3~jessie held back because of libvarnishapi1 dependency problems
Previous to Install:
root@skynet:~# dpkg -l | grep varnish
ii libvarnishapi1 4.1.3-1~jessie amd64 shared libraries for Varnish
ii varnish 4.1.3-1~jessie amd64 state of the art, high-performance web accelerator
Install Simulation:
root@skynet:~# apt-get -s install varnish
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
varnish-dev
The following packages will be REMOVED:
libvarnishapi1
The following packages will be upgraded:
varnish
1 upgraded, 0 newly installed, 1 to remove and 24 not upgraded.
Remv libvarnishapi1 [4.1.3-1~jessie] [varnish:amd64 ]
Inst varnish [4.1.3-1~jessie] (4.1.4-3~jessie debian jessie:repo.varnish-cache.org [amd64])
Conf varnish (4.1.4-3~jessie debian jessie:repo.varnish-cache.org [amd64])
I am using Debian Jessie latest:
Linux skynet 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19) x86_64 GNU/Linux
Install Attempt
root@skynet:~# apt-get install varnish
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
varnish-dev
The following packages will be REMOVED:
libvarnishapi1
The following packages will be upgraded:
varnish
1 upgraded, 0 newly installed, 1 to remove and 24 not upgraded.
Need to get 2,399 kB of archives.
After this operation, 4,098 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 https://repo.varnish-cache.org/debian jessie/varnish-4.1 amd64 varnish amd64 4.1.4-3~jessie [2,399 kB]
Reading changelogs... Done
dpkg: libvarnishapi1: dependency problems, but removing anyway as you requested:
varnish depends on libvarnishapi1 (= 4.1.3-1~jessie).
(Reading database ... 92077 files and directories currently installed.)
Removing libvarnishapi1 (4.1.3-1~jessie) ...
(Reading database ... 92078 files and directories currently installed.)
Preparing to unpack .../varnish_4.1.4-3~jessie_amd64.deb ...
Unpacking varnish (4.1.4-3~jessie) over (4.1.3-1~jessie) ...
Setting up varnish (4.1.4-3~jessie) ...
Installing new version of config file /etc/init.d/varnishncsa ...
Installing new version of config file /etc/logrotate.d/varnish ...
insserv: warning: script 'K01aeroo-docs' missing LSB tags and overrides
insserv: warning: script 'aeroo-docs' missing LSB tags and overrides
insserv: warning: script 'K01aeroo-docs' missing LSB tags and overrides
insserv: warning: script 'aeroo-docs' missing LSB tags and overrides
Processing triggers for systemd (215-17+deb8u5) ...
Processing triggers for man-db (2.7.0.2-5) ...
[ Rootkit Hunter version 1.4.2 ]
File updated: searched for 177 files, found 146
Install version check:
root@skynet:~# dpkg -l | grep varnish
rc libvarnishapi1 4.1.3-1~jessie amd64 shared libraries for Varnish
ii varnish 4.1.4-3~jessie amd64 state of the art, high-performance web accelerator
This looks to be a packaging issue so I am posting this issue here. I assume that I should rollback to the previous version. Which is what I have done.
root@skynet:~# apt-get install varnish=4.1.3-1~jessie
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
libvarnishapi1
Suggested packages:
varnish-doc
The following NEW packages will be installed:
libvarnishapi1
The following packages will be DOWNGRADED:
varnish
0 upgraded, 1 newly installed, 1 downgraded, 0 to remove and 24 not upgraded.
Need to get 603 kB of archives.
After this operation, 4,098 kB disk space will be freed.
Do you want to continue? [Y/n] y
Get:1 https://repo.varnish-cache.org/debian jessie/varnish-4.1 amd64 varnish amd64 4.1.3-1~jessie [543 kB]
Get:2 https://repo.varnish-cache.org/debian jessie/varnish-4.1 amd64 libvarnishapi1 amd64 4.1.3-1~jessie [59.9 kB]
dpkg: warning: downgrading varnish from 4.1.4-3~jessie to 4.1.3-1~jessie
(Reading database ... 92345 files and directories currently installed.)
Preparing to unpack .../varnish_4.1.3-1~jessie_amd64.deb ...
Unpacking varnish (4.1.3-1~jessie) over (4.1.4-3~jessie) ...
Selecting previously unselected package libvarnishapi1.
Preparing to unpack .../libvarnishapi1_4.1.3-1~jessie_amd64.deb ...
Unpacking libvarnishapi1 (4.1.3-1~jessie) ...
Setting up libvarnishapi1 (4.1.3-1~jessie) ...
Processing triggers for systemd (215-17+deb8u5) ...
Processing triggers for man-db (2.7.0.2-5) ...
Setting up varnish (4.1.3-1~jessie) ...
Installing new version of config file /etc/init.d/varnishncsa ...
Installing new version of config file /etc/logrotate.d/varnish ...
insserv: warning: script 'K01aeroo-docs' missing LSB tags and overrides
insserv: warning: script 'aeroo-docs' missing LSB tags and overrides
insserv: warning: script 'K01aeroo-docs' missing LSB tags and overrides
insserv: warning: script 'aeroo-docs' missing LSB tags and overrides
insserv: warning: script 'K01aeroo-docs' missing LSB tags and overrides
insserv: warning: script 'aeroo-docs' missing LSB tags and overrides
Processing triggers for systemd (215-17+deb8u5) ...
[ Rootkit Hunter version 1.4.2 ]
File updated: searched for 177 files, found 146
Hope this can be sorted out. Thanks for all you do!
Hello,
Reload script on debian/reload-vcl
has no idea about new flags/parameters added on 4.1/5.0
http://www.varnish-cache.org/docs/4.1/reference/varnishd.html
http://www.varnish-cache.org/docs/5.0/reference/varnishd.html
for example:
-W waiter Specifies the waiter type to use.
it leads to warning report during usage
# /etc/init.d/varnish reload
[....] Reloading HTTP accelerator: varnishdIllegal option -W
. ok
Moved from varnishcache/varnish-cache#2037. CC @bjornbak.
Please create an apt repository for xenial as it is the latest LTS release for Ubuntu.
[13:17:44] < fgs> re #1630 I suspect we have a packaging building issue
[13:18:14] < fgs> and jit was not enabled
[13:18:17] < fgs> on 4.1
[13:20:06] < scn> #1630
[13:20:07] -beb:#varnish-hacking- #1630: Varnish crashes with long regex condition [closed]
https://www.varnish-cache.org/trac/ticket/1630
[13:21:26] < fgs> if the logs are correct we have pcre 8.31 installed
[13:21:44] < fgs> jit is enabled on > 8.31
[13:23:35] < scn> i'll put it on my list of things to check before 4.1.1
Debian source package for 4.1.1 was missing from repo.varnish-cache.org.
Reported on irc. I've fixed this now but we should ensure that future source packages are available.
It seems like the following commit removed the support for having ":" in VCL-names.
varnishcache/varnish-cache@169e162
This causes reload-vcl to fail when building against trunk.
Hello,
as per discussion in varnishcache/varnish-cache#2195, I suggest to put sleep between vcl.load
and vcl.use
in reload-script. Ideally, this should be configurable via ENV variable with some default. 5s worked for me, but I think value will depend on health probe configuration.
Example of suggested patch:
--- a/debian/reload-vcl 2016-10-13 04:21:04.000000000 +0000
+++ b/debian/reload-vcl 2016-12-07 20:12:13.465508289 +0000
@@ -149,6 +149,7 @@
then
printf "$msg_compile_only" $varnishadm $mgmt_interface $vcl_label
else
+ sleep 5
if $varnishadm -T $mgmt_interface -S ${secret} vcl.use $vcl_label
then
printf "$msg_use_ok" $vcl_label
As for how to make it configurable, eg.:
WARMUP_TIME=${WARMUP_TIME:-5}
Reloading the varnishncsa service on Ubuntu 16.04 LTS stops the varnishncsa daemon. This is particularly an issue because the postrotate
script in /etc/logrotate.d/varnish
calls /usr/sbin/invoke-rc.d varnishncsa reload
. To wit:
# systemctl is-active varnishncsa
inactive
# systemctl start varnishncsa
# systemctl is-active varnishncsa
active
# systemctl reload varnishncsa
# systemctl is-active varnishncsa
inactive
I'm using the varnish package that comes with 16.04 because the "official" repository is not yet available for Xenial. I can confirm that the typo described in #28 is not present (i.e. the fix is applied).
Possibly related to/regression of/duplicate of:
I am getting an error when trying to install varnish
on an Ubuntu 14.04 host:
Error: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install varnish' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
libjemalloc1 libvarnishapi1
Suggested packages:
varnish-doc
The following NEW packages will be installed:
libjemalloc1 libvarnishapi1 varnish
0 upgraded, 3 newly installed, 0 to remove and 182 not upgraded.
Need to get 674 kB of archives.
After this operation, 2120 kB of additional disk space will be used.
Get:1 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/ trusty/universe libjemalloc1 amd64 3.5.1-2 [76.8 kB]
Get:2 http://repo.varnish-cache.org/ubuntu/ trusty/varnish-4.1 libvarnishapi1 amd64 4.1.3-1~trusty [58.8 kB]
Get:3 http://repo.varnish-cache.org/ubuntu/ trusty/varnish-4.1 varnish amd64 4.1.3-1~trusty [538 kB]
Fetched 674 kB in 6s (97.4 kB/s)
Selecting previously unselected package libvarnishapi1.
(Reading database ... 75032 files and directories currently installed.)
Preparing to unpack .../libvarnishapi1_4.1.3-1~trusty_amd64.deb ...
Unpacking libvarnishapi1 (4.1.3-1~trusty) ...
Selecting previously unselected package libjemalloc1.
Preparing to unpack .../libjemalloc1_3.5.1-2_amd64.deb ...
Unpacking libjemalloc1 (3.5.1-2) ...
Selecting previously unselected package varnish.
Preparing to unpack .../varnish_4.1.3-1~trusty_amd64.deb ...
Unpacking varnish (4.1.3-1~trusty) ...
Processing triggers for ureadahead (0.100.0-16) ...
Processing triggers for man-db (2.6.7.1-1ubuntu1) ...
Setting up libvarnishapi1 (4.1.3-1~trusty) ...
Setting up libjemalloc1 (3.5.1-2) ...
Setting up varnish (4.1.3-1~trusty) ...
Configuration file '/etc/default/varnishncsa'
==> File on system created by you or by a script.
==> File also in package provided by package maintainer.
==> Using current old file as you requested.
* Starting HTTP accelerator varnishd
...done.
* Starting HTTP accelerator log deamon
install: invalid group 'varnishlog'
...fail!
Can't open format file (No such file or directory)
invoke-rc.d: initscript varnishncsa, action "start" failed.
dpkg: error processing package varnish (--configure):
subprocess installed post-installation script returned error exit status 1
Processing triggers for libc-bin (2.19-0ubuntu6.6) ...
Processing triggers for ureadahead (0.100.0-16) ...
Errors were encountered while processing:
varnish
E: Sub-process /usr/bin/dpkg returned an error code (1)
The varnish
package is 4.1.3-1~trusty
from http://repo.varnish-cache.org/ubuntu/.
The Debian package doesn't seem to be using dh-sytemd, so the systemd service for varnish is not automatically enabled.
According to https://wiki.debian.org/Teams/pkg-systemd/Packaging this will be handled by adding --with systemd
to the dh
call in the debian/rules
file
If i run service varnish restart, it should restart usually
I try to restart varnish and I get this error:
$ sudo service varnish restart
* Checking syntax varnishd
...done.
* Stopping HTTP accelerator varnishd
No /usr/sbin/varnishd found running; none killed.
...fail!
* Starting HTTP accelerator varnishd
...fail!
Error: Cannot open socket: :80: Address already in use
sudo service varnish start
works fine, same with
sudo service varnish stop
It is just the restart that does not work.
Version used: varnish-4.1.5 revision 2c82b1c
Operating System and version: Ubuntu 14.04
I used this package: https://packagecloud.io/varnishcache/varnish41/packages/ubuntu/trusty/varnish_4.1.5-1~trusty_amd64.deb
Running debian the postinst tries to start
All service are installed but disabled by default. (in /etc/default/*)
varnish will "start" just fine because of
case "${START:-}" in
[Yy]es|[Yy]|1|[Tt]|[Tt]rue)
start_varnishd
;;
*)
disabled_varnishd
;;
esac
in the corresponding init.d file, but varnishncsa and varnishlog respectively do not have this check. This leads to a failed upgrade because varnishncsa and varnishlog expect varnish running.
Both service will have to have a check too, or the postinst shouldn't be trying to start them, if varnish isn't running.
I'm curious why the service files for rpm vs deb packages are so different? The one for RedHat/etc has a dedicated params file placed in /etc, whereas with the .deb it's all hard-coded in the .service file.
I upgraded to varnish_5.0.0-1_amd64.deb from https://repo.varnish-cache.org/pkg/5.0.0/ on Ubuntu 16.04.1 LTS which was previously running official xenial varnish 4.1.1-1 and I was only able to run varnishd and varnishtest.
I removed varnish 4.1.1-1 with apt-get remove --purge but libvarnishapi1 stayed causing errors.
apt remove libvarnishapi solved that, but why did it stay on system in first place?
dpkg-query -W varnish
varnish 5.0.0-1
varnishd -V
varnishd (varnish-5.0.0 revision 99d036f)
Copyright (c) 2006 Verdens Gang AS
Copyright (c) 2006-2015 Varnish Software AS
Other binaries complain about missing LIBVARNISHAPI_1.5.
varnishadm: /usr/lib/x86_64-linux-gnu/libvarnishapi.so.1: version `LIBVARNISHAPI_1.5' not found (required by varnishadm)
varnishtop: /usr/lib/x86_64-linux-gnu/libvarnishapi.so.1: version `LIBVARNISHAPI_1.5' not found (required by varnishtop)
varnishstat: /usr/lib/x86_64-linux-gnu/libvarnishapi.so.1: version `LIBVARNISHAPI_1.5' not found (required by varnishstat)
varnishncsa: /usr/lib/x86_64-linux-gnu/libvarnishapi.so.1: version `LIBVARNISHAPI_1.5' not found (required by varnishncsa)
varnishlog: /usr/lib/x86_64-linux-gnu/libvarnishapi.so.1: version `LIBVARNISHAPI_1.5' not found (required by varnishlog)
varnishhist: /usr/lib/x86_64-linux-gnu/libvarnishapi.so.1: version `LIBVARNISHAPI_1.5' not found (required by varnishhist)
edit: versions
It works like this repository needs to be updated after varnishcache/varnish-cache@dd5d189. Specifically, I am trying to build Varnish from source and the service is failing to start with the following error:
CLI result = 300
Error: Child (10967) Pushing vcls failed:
VCL "boot" Failed initialization
Message:
Creating copy of vmod directors: Permission denied
I think that the issue is that /var/lib/varnish/${NAME}/vmod_cache
is owned by varnish:varnish
, and is not writable by the vcache
user.
In a pull-request to the old Varnish-Cache project github user @BMDan suggest that we change our systemd service files::
"After" only ensures ordering if the targets are part of the same transaction. "Requires" only ensures that the targets are part of the same transaction, not ordering. Thus, both are necessary to ensure reliable and deterministic startup. Credit to @davidstrauss at DrupalCon LA 2015 for spotting the issue! He also suggests dropping syslog.target, as it's deprecated. Quoth https://wiki.freedesktop.org/www/Software/systemd/syslog/:
"Newer systemd versions (v35+) do not support non-socket-activated syslog daemons anymore and we do no longer recommend people to order their units after syslog.target. That means that unless your syslog implementation is socket activatable many services will not be able to log to your syslog implementation and early boot messages are lost entirely to your implementation."
Suggested patch:
(=86628 vim:varnish.service) lkarsten@IMMER ~/work/pkg-varnish-cache/redhat> git df --cached
diff --git a/redhat/varnish.service b/redhat/varnish.service
index e8d442f..aec3f07 100644
--- a/redhat/varnish.service
+++ b/redhat/varnish.service
@@ -1,6 +1,6 @@
[Unit]
Description=Varnish Cache, a high-performance HTTP accelerator
-After=syslog.target network.target
+After=network.target
[Service]
# If you want to make changes to this file, please copy it to
After discussing this a bit on IRC, there doesn't seem to be a consensus on how this should be done.
Wants=network.target
?Let's take the remaining discussion here, so we can complete the gitrepo move tasks.
Hey,
I get a "106 Illegal character in VCL name" whenever uuid generates a string beginning with a number and calling reload-vcl via init script or systemctrl
OS: Debian (jessie)
Varnish: 5.0.0-1
See
https://github.com/varnishcache/varnish-cache/blob/master/bin/varnishd/mgt/mgt_vcl.c
pkg-varnish-cache/debian/reload-vcl
Lines 42 to 48 in 1d5b56f
sh is a symlink to dash, but i've also tried bash so that's not it.
$(logname) is also empty, but only when calling via init.
Cheers
I've mentioned on IRC that we are unable to run systemctl reload varnish
as reload-vcl fails with reload-vcl[28101]: /usr/share/varnish/reload-vcl: 44: /usr/share/varnish/reload-vcl: Cannot fork
@daghf Instructed me to add TasksMax=infinity
to the unit file, and that then allowed reload-vcl to run successfully. He also suspected that "One could configure varnish with a min threads value that exceeds the default TasksMax"
I guess in some circumstances, the systemd slice can reach TasksMax and thus prevent reload-vcl from being forked. -xavierg.
Hopefully this is useful to you, I can provide more information if you need it.
On Debian stretch, with VCP4.1 trusty repo in sources.list:
# apt-get install varnish-plus varnish-plus-vmods-extra
The following packages have unmet dependencies:
varnish-plus-vmods-extra : Depends: libmemcached10 but it is not installable
E: Unable to correct problems, you have held broken packages.
# apt-cache search libmemcached
[..]
libmemcached-dbg - Debug Symbols for libmemcached
libmemcached-dev - C and C++ client library to the memcached server (development files)
libmemcached-tools - Commandline tools for talking to memcached via libmemcached
libmemcached11 - C and C++ client library to the memcached server
replicating varnishcache/varnish-cache#1947
I´m having trouble upgrading Varnish 4.0 to 4.1 on Ubuntu 12.04 and 14.04. So, I tried a clean install of 4.1 on a new VM and it fails too.
The varnish
daemon correctly starts out-of-the-box. Then the optional varnishncsa
service is enabled (was already enabled when upgrading or is enabled on a clean install) via the variable VARNISHNCSA_ENABLED=1
in file /etc/default/varnishncsa
.
When I try to start the service via service varnishncsa start
the varnishncsa
daemon should start running in the background.
When I try to start the service via service varnishncsa start
it fails to start.
root@test01:~# service varnishncsa start
* Starting HTTP accelerator log deamon [fail]
Can't open log - retrying for 5 seconds
Can't open VSM file (Cannot open /var/lib/varnish/test01/_.vsm: Permission denied
)
The VSM-file´s permissions are:
-rw-r----- 1 root varnish 84934656 May 11 14:58 /var/lib/varnish/test01/_.vsm
when they used to be root:root
in Varnish 4.0.
According to https://www.varnish-cache.org/docs/trunk/whats-new/changes.html:
varnishlog, varnishncsa and other Varnish shared log utilities now must be run in a context with varnish group membership.
So, the package should fix group membership on upgrade and installation by adding the user varnishlog
to the group varnish
, before attempting to start the varnishlog and varnishncsa daemons.
apt-get install apt-transport-https
curl https://repo.varnish-cache.org/GPG-key.txt | apt-key add -
echo "deb https://repo.varnish-cache.org/ubuntu/ trusty varnish-4.1" >> /etc/apt/sources.list.d/varnish-cache.list
apt-get update && apt-get install varnish
echo "VARNISHNCSA_ENABLED=1" > /etc/default/varnishncsa
service varnishncsa start
Upgrading from Varnish 4.0 to 4.1 failed because we had enabled the varnishncsa service.
root@test01:/var/lib/varnish/test01# apt full-upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
libvarnishapi1 varnish
2 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/588 kB of archives.
After this operation, 113 kB of additional disk space will be used.
Do you want to continue? [Y/n]
(Reading database ... 95839 files and directories currently installed.)
Preparing to unpack .../libvarnishapi1_4.1.2-2~trusty_amd64.deb ...
Unpacking libvarnishapi1 (4.1.2-2~trusty) over (4.0.3-5~trusty) ...
Preparing to unpack .../varnish_4.1.2-2~trusty_amd64.deb ...
* Stopping HTTP accelerator log deamon varnishncsa [ OK ]
* Stopping HTTP accelerator
varnishd [ OK ]
Unpacking varnish (4.1.2-2~trusty) over (4.0.3-5~trusty) ...
Processing triggers for ureadahead (0.100.0-16) ...
Processing triggers for man-db (2.6.7.1-1ubuntu1) ...
Setting up libvarnishapi1 (4.1.2-2~trusty) ...
Setting up varnish (4.1.2-2~trusty) ...
Installing new version of config file /etc/init.d/varnish ...
Configuration file '/etc/varnish/default.vcl'
==> Modified (by you or by a script) since installation.
==> Package distributor has shipped an updated version.
==> Keeping old config file as default.
* Starting HTTP accelerator
varnish [ OK ]
* Starting HTTP accelerator log deamon
varnishncsa [fail]
Can't open log - retrying for 5 seconds
Can't open VSM file (Cannot open /var/lib/varnish/test01/_.vsm: Permission denied
)
invoke-rc.d: initscript varnishncsa, action "start" failed.
dpkg: error processing package varnish (--configure):
subprocess installed post-installation script returned error exit status 1
Processing triggers for libc-bin (2.19-0ubuntu6.7) ...
Processing triggers for ureadahead (0.100.0-16) ...
Errors were encountered while processing:
varnish
E: Sub-process /usr/bin/dpkg returned an error code (1)
I am trying to build Varnish from source using dpkg-buildpackage
, but it is currently failing with the following error:
dh_install -a --fail-missing
install -o root -g root -m 755 \
/home/jenkins/workspace/Infrastructure/varnish/debian/reload-vcl \
/home/jenkins/workspace/Infrastructure/varnish/debian/varnish/usr/share/varnish/reload-vcl
make[1]: Leaving directory '/home/jenkins/workspace/Infrastructure/varnish'
debian/rules override_dh_installdocs
make[1]: Entering directory '/home/jenkins/workspace/Infrastructure/varnish'
test -d doc/html || cp -pr doc/sphinx/build/html doc/html
cp: cannot stat 'doc/sphinx/build/html': No such file or directory
debian/rules:59: recipe for target 'override_dh_installdocs' failed
make[1]: *** [override_dh_installdocs] Error 1
make[1]: Leaving directory '/home/jenkins/workspace/Infrastructure/varnish'
debian/rules:29: recipe for target 'binary-arch' failed
make: *** [binary-arch] Error 2
dpkg-buildpackage: error: debian/rules binary-arch gave error exit status 2
I noticed that sbuild
is used rather than dpkg-buildpackage
in the ./package-deb
, so I don't know if the upstream is interested in the fix here, but I was able to successfully build the varnish
package by adding the following target to debian/rules
:
override_dh_auto_build:
$(MAKE) dist html
I will submit a pull request.
When I tried to install varnish 4.1.4 and following that run apt-get install libvarnishapi-dev
apt finds 4.1.3 and give following error:
root@vagrant-ubuntu-trusty-64:/home/vagrant# apt-get install libvarnishapi-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
libvarnishapi-dev
0 upgraded, 1 newly installed, 0 to remove and 24 not upgraded.
2 not fully installed or removed.
Need to get 0 B/66.7 kB of archives.
After this operation, 400 kB of additional disk space will be used.
(Reading database ... 60466 files and directories currently installed.)
Preparing to unpack .../libvarnishapi-dev_4.1.3-1~trusty_amd64.deb ...
Unpacking libvarnishapi-dev (4.1.3-1~trusty) ...
dpkg: error processing archive /var/cache/apt/archives/libvarnishapi-dev_4.1.3-1~trusty_amd64.deb (--unpack):
trying to overwrite '/usr/lib/libvarnishapi.so', which is also in package varnish 4.1.4-1~trusty
Errors were encountered while processing:
/var/cache/apt/archives/libvarnishapi-dev_4.1.3-1~trusty_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
So I specified libvarnishapi-dev 4.1.4 to be installed and got following:
root@vagrant-ubuntu-trusty-64:/home/vagrant# varnishd -V
varnishd (varnish-4.1.4 revision 4529ff7)
Copyright (c) 2006 Verdens Gang AS
Copyright (c) 2006-2015 Varnish Software AS
root@vagrant-ubuntu-trusty-64:/home/vagrant# apt-get install libvarnishapi-dev=4.1.4-1~trusty
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Version '4.1.4-1~trusty' for 'libvarnishapi-dev' was not found
When I go to https://repo.varnish-cache.org/ubuntu/pool/varnish-4.1/v/varnish/ there is libvarnishapi-dev upto 4.1.3 and 4.1.4 was not there. Is there any way to install 4.1.4 for libvarnishapi-dev package ?
File /etc/sysconfig/varnish is not updated after upgrade.
There are some incompatible params. User and group param do not work. Also listen address cannot be empty. Use * instead. Admin listen address cannot be 127.0.0.1 too, use * instead.
This fixes are not specified at 4.1 upgrade change log. Checking google is posible to find some other users experiencing the same problem. Some people just downgraded to 4.0. But 4.0 is not compatible with new VMODS
There seems to be an issue with the packaging of varnish 4.1.4 on RHEL systems. After installing, you get the following warnings:
yum check
Loaded plugins: fastestmirror
varnish-4.1.4-1.el7.x86_64 has installed conflicts varnish-libs: varnish-4.1.4-1.el7.x86_64
varnish-4.1.4-1.el7.x86_64 has installed conflicts varnish-docs: varnish-4.1.4-1.el7.x86_64
varnish-4.1.4-1.el7.x86_64 has installed conflicts varnish-debuginfo: varnish-4.1.4-1.el7.x86_64
Error: check all
It's functional other than the warnings being present.
I'm not an expert on the RPM spec, but it looks like it both provides these and declares them as conflicts, which doesn't look correct, but I may be barking up the wrong tree...
This happens when trying to install the trusty release in Xenial. The key needs updating to the new requirements
sudo apt-get update
W: https://repo.varnish-cache.org/ubuntu/dists/trusty/InRelease: Signature by key E98C6BBBA1CBC5C3EB2DF21C60E7C096C4DEFFEB uses weak digest algorithm (SHA1)
Key at:
https://repo.varnish-cache.org/ubuntu/GPG-key.txt
Debian and Ubuntu enforce SHA256 or higher entries in the Release and/or Packages files since March. Repositories missing these need to be fixed by their owners.
If you are hosting a repository which is giving these errors. The solution is to change the default cert-digest-algo to be SHA256. By default gnupg defaults to using SHA1
After you fix this issue the next warning will be that the signature "uses weak digest algorithm (SHA1)" And to fix that you can set digest-algo to SHA256 as well.
These values go on the repository server in the gpg.conf which the repository is using.
The short hand is to append
cert-digest-algo SHA256
digest-algo SHA256
to your gpg.conf file.
https://juliank.wordpress.com/2016/03/14/dropping-sha-1-support-in-apt/
Using version 4.1.3-1~trusty
and I am unable to uninstall the varnish
package. #38 left varnish
in a partially-installed state, after which I unsuccessfully attempted to remove the varnish
package:
> sudo apt-get remove --purge varnish
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following package was automatically installed and is no longer required:
libjemalloc1
Use 'apt-get autoremove' to remove it.
The following packages will be REMOVED:
varnish*
0 upgraded, 0 newly installed, 1 to remove and 176 not upgraded.
1 not fully installed or removed.
After this operation, 1,664 kB disk space will be freed.
Do you want to continue? [Y/n] Y
(Reading database ... 75124 files and directories currently installed.)
Removing varnish (4.1.3-1~trusty) ...
* Stopping HTTP accelerator log deamon No /usr/bin/varnishncsa found running; none killed.
[fail]
invoke-rc.d: initscript varnishncsa, action "stop" failed.
dpkg: error processing package varnish (--purge):
subprocess installed pre-removal script returned error exit status 1
* Starting HTTP accelerator varnishd [fail]
/usr/sbin/varnishd already running.
invoke-rc.d: initscript varnish, action "start" failed.
dpkg: error while cleaning up:
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
varnish
E: Sub-process /usr/bin/dpkg returned an error code (1)
The following message appears in journalctl
when systemd is reading the varnish service definition:
systemd[1]: [/usr/lib/systemd/system/varnish.service:20] Failed to parse resource value, ignoring: 82000
The actual value:
# systemctl show varnish | grep LimitMEMLOCK
LimitMEMLOCK=65536
I thought the problem was the integer value and that it could be fixed by setting the value to 82M
. However, contrary to the documentation systemd is unable to parse any values other than infinity
for this field. I even tried inserting the value LimitMEMLOCK=65536
printed by the command above, and it still fails. So it's possible that this is actually a systemd issue, but I couldn't find any relevant bug reports.
yum install varnish
Optionally run systemctl daemon-reload
to see the message again.
The relevant configuration line: LimitMEMLOCK=82000
ulimit -l 82000
works fine on the relevant machine.
There are some additional Provides added by the find-provides script during package builds for repo.varnish-cache.org. These add the VRT (Varnish RunTime) markers to the package, for vmod packages to depend on.
For 4.1.1-2 this set of Provides exist:
[lkarsten@el7 el7]$ rpm -qp --provides varnish-4.1.1-2.el7.x86_64.rpm
varnish = 4.1.1-2.el7
varnish(x86-64) = 4.1.1-2.el7
varnishabi-strict-66bb824
varnishabi-3.1
Originally raised on the EPEL package in https://bugzilla.redhat.com/show_bug.cgi?id=1303543#c6 , Dridi asks if shouldn't instead be:
varnish(abi) = 3.1
varnish(abi-strict) = aabbccd
because it is more redhat-y and possibly so that it is possible to do a numerical compare on it. (so a vmod built against 3.1 and install on 3.2, as planned)
The current Provide has been there for quite a while. I believe I looked at this before extending it with abi-strict a few years back, but I don't remember any details any more.
After the 4.1.1 update I now get this error on a fresh install
The following packages have unmet dependencies:
libvarnishapi-dev : Depends: libvarnishapi1 (= 4.1.1-1~jessie) but 4.1.1-4.1.1~jessie is to be installed
E: Unable to correct problems, you have held broken packages.
Currently, when attempting to install Varnish on Ubuntu 16.04 LTS, I get:
The repository 'http://repo.varnish-cache.org/ubuntu xenial Release' does not have a Release file.
Failed to fetch http://repo.varnish-cache.org/ubuntu/dists/xenial/varnish-4.0/binary-amd64/Packages 404 Not Found [IP: 194.31.39.155 80]
Failed to fetch http://repo.varnish-cache.org/ubuntu/dists/xenial/varnish-4.0/binary-i386/Packages 404 Not Found [IP: 194.31.39.155 80]
Is a Xenial release channel in the works? I know it's only been a day... :)
If i run service varnish restart, it should restart usually
I try to restart varnish and I get this error:
root@localhost:~# sudo service varnish restart
- Stopping HTTP accelerator varnish No /usr/sbin/varnishd found running; none killed.
[fail]- Starting HTTP accelerator varnish
After this varnish no longer works, it totally stops and I need t reboot my ubuntu server in order for it to work again
How can I fix this?
I pretty much cant restart varnish and in order to restart it I need to reboot my system
While readying my Ansible role for Varnish for 5.x (geerlingguy/ansible-role-varnish#39), I noticed that the 5.0 packages aren't built yet—is this the right place to stick an issue tracking 5.0 package build progress?
With a recent upgrade, Debian stretch now has apt 1.4 which disallows SHA1 signatures for repositories. This means the jessie repo can not be used any longer.
root@lima:/home/lkarsten# apt-get update
Hit:1 http://repo.varnish-cache.org/debian jessie InRelease
Hit:2 http://ftp.no.debian.org/debian stretch InRelease
Hit:3 http://ftp.no.debian.org/debian stretch-updates InRelease
Err:1 http://repo.varnish-cache.org/debian jessie InRelease
The following signatures were invalid: E98C6BBBA1CBC5C3EB2DF21C60E7C096C4DEFFEB
[..]
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://repo.varnish-cache.org/debian jessie InRelease: The following signatures were invalid: E98C6BBBA1CBC5C3EB2DF21C60E7C096C4DEFFEB
W: Failed to fetch http://repo.varnish-cache.org/debian/dists/jessie/InRelease The following signatures were invalid: E98C6BBBA1CBC5C3EB2DF21C60E7C096C4DEFFEB
W: Some index files failed to download. They have been ignored, or old ones used instead.
Expected: jessie packages easily installable on stretch, like they've been up until now.
Before the upgrade, this was only a warning:
W: http://repo.varnish-cache.org/debian/dists/jessie/InRelease: Signature by key E98C6BBBA1CBC5C3EB2DF21C60E7C096C4DEFFEB uses weak digest algorithm (SHA1)
From the update note:
apt (1.4~beta1) unstable; urgency=medium
Support for GPG signatures using the SHA1 or RIPE-MD/160 hash
algorithms has been disabled. Repositories using Release files
signed in such a way will stop working. This change has been made
due to security considerations, especially with regards to possible
further breakthroughs in SHA1 breaking during the lifetime
of this APT release series.
It is possible (but STRONGLY ADVISED AGAINST) to revert to the previous
behaviour by setting the options
APT::Hashes::SHA1::Weak "yes";
APT::Hashes::RIPE-MD/160::Weak "yes";
Note that setting these options only affects the verification of the overall
repository signature.
-- Julian Andres Klode <[email protected]> Fri, 25 Nov 2016 13:19:32 +0100
Hello,
I'm not sure how to formulate request politely enough and correctly, but I want to ask for a new Debian release or build. Outstanding bugs which make Varnish 5.0.0 unusable/unreliable for production and are worth of new release(in my opinion and the ones we've encountered so far ... which doesn't mean I'm right):
Thank you and sorry for being pesky :)
Newest varnish 4.1.4 release for Debian tries to overwrite libvarnishapi.so from libvarnishapi-dev 4.1.3
Installing varnish and libvarnishapi-dev via repo should not result in an error.
When installing varnish and libvarnishapi-dev at the same time (via apt from the repo), a dpkg error is generated and installation of varnish fails.
Error log:
root@jibakurei:/home/ubuntu# apt install varnish
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
varnish-dev
The following NEW packages will be installed:
varnish
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/2,415 kB of archives.
After this operation, 5,997 kB of additional disk space will be used.
(Reading database ... 109777 files and directories currently installed.)
Preparing to unpack .../varnish_4.1.4-1~trusty_amd64.deb ...
Unpacking varnish (4.1.4-1~trusty) ...
Replacing files in old package libvarnishapi1 (4.1.3-1~trusty) ...
dpkg: error processing archive /var/cache/apt/archives/varnish_4.1.4-1~trusty_amd64.deb (--unpack):
trying to overwrite '/usr/lib/libvarnishapi.so', which is also in package libvarnishapi-dev 4.1.3-1~trusty
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for systemd (229-4ubuntu12) ...
Processing triggers for ureadahead (0.100.0-19) ...
Errors were encountered while processing:
/var/cache/apt/archives/varnish_4.1.4-1~trusty_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
Did varnish package install libvarnishapi.so earlier? Specifying varnish 4.1.3 works.
Install varnish and libvarnishapi-dev from via apt from the repo, without specifying a version.
I'm installing varnish and the api/dev packages (for vmods) at the same time using chef, which pulls down the latest version of each from the repo, and now the build fails.
Ubuntu 16.04 LTS
With 4.0.3-3.el7 from EPEL with selinux-policy.noarch 3.13.1-60.el7 prohibits backend health polling.
Reported on IRC. Reporter got it working after disabling selinux.
Expected: backend health polling should work by default.
Most likely this applies to all packaging, and needs a policy update.
Have not attempted to reproduce.
Fire up a new Ubuntu 14.04 instance (I used an AWS Ubuntu image):
add-apt-repository "deb https://repo.varnish-cache.org/ubuntu/ trusty varnish-4.1"
apt-get update
apt-get install varnish --force-yes -y
sed -i 's#-s malloc,256m"#-P /run/varnishd.pid \\\n -s malloc,256m"#' /etc/default/varnish
service varnish restart
Clean restart
* Stopping HTTP accelerator varnishd
No /usr/sbin/varnishd found running; none killed.
...fail!
* Starting HTTP accelerator varnishd
...fail!
Error: Cannot open socket: :6081: Address already in use
The sed just adds the -P parameter to /etc/default/varnish:
DAEMON_OPTS="-a :6081 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-P /run/varnishd.pid \
-s malloc,256m"
Took me a while to track it down. I recently opened this report: #71 but was wrong in my assessment.
I found files in repo.varnish-cache.org with version-version on top of the regular version, e.g. varnish-doc_4.1.1-4.1.1jessie_all besides varnish-doc_4.1.1-1jessie_all.
This needs to be checked before 4.1.2.
It would be grate to have packages for varnish-modules in https://repo.varnish-cache.org/ubuntu/
[14:43:55] < ssm> scn: the reload script in the debian package in Debian does not work well anymore, probably better to drop the systemd "reload" action, and let people use varnishadm explicitly, unless someone has a script that works.
[14:44:57] < scn> ssm: i think having reload in place has value. shall we clean up the script?
[14:45:39] < ssm> probably.
Hi,
I have discovered another issue with the Debian init script PIDFILE generation, this time on system boot. Because the PIDFILE prefix is taken from $SERVICE
, which is geneated like so:
SERVICE="$(basename ${0})"
When the system boots, basename
returns the name of the startup symlink S20varnish
, which means the resulting PIDFILE is incorrectly set as /run/S20varnish.pid
.
This then causes the same problems described by my original bug report here varnishcache/varnish-cache#1903.
I'm running Ubuntu 14.04 Trusty.
Thanks,
Ryan
Any reason for this?
After having built from 3e3dbbe:
[lkarsten@el7 el7]$ sudo rpm -Uvh varnish-4.1.0-1.el7.x86_64.rpm \
varnish-libs-4.1.0-1.el7.x86_64.rpm varnish-libs-devel-4.1.0-1.el7.x86_64.rpm \
varnish-debuginfo-4.1.0-1.el7.x86_64.rpm
[lkarsten@el7 el7]$ rpm -ql varnish
[..]
/usr/sbin/varnishd
/usr/share/doc/varnish
/usr/share/doc/varnish-4.1.0
/usr/share/doc/varnish-4.1.0/LICENSE
/usr/share/doc/varnish-4.1.0/README
/usr/share/doc/varnish/builtin.vcl
/usr/share/doc/varnish/example.vcl
/usr/share/man/man1/varnishadm.1.gz
Should all files be in /usr/share/doc/varnish-4.1.0/ or in /usr/share/doc/varnish/ ?
When using systemd, editing /etc/default/varnish and restarting varnish, changement must be applied.
When starting varnish only the default configuration present in /lib/systemd/system/varnish.service is used.
We could use the next code for the systemd unit.
Explanation :
[Unit]
Description=Varnish HTTP accelerator
[Service]
Type=forking
LimitNOFILE=131072
LimitMEMLOCK=82000
Environment="DAEMON_OPTS=-a :6081 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m"
EnvironmentFile=-/etc/default/varnish
ExecStartPre=/usr/sbin/varnishd -C -f /etc/varnish/default.vcl
ExecStart=/usr/sbin/varnishd $DAEMON_OPTS
ExecReload=/usr/share/varnish/reload-vcl
[Install]
WantedBy=multi-user.target
Hi,
I've just installed varnish 4.1 on a ubuntu trusty server and noticed the following issue with the deb package.
I tried using the new jail argument -j
. I should be able to
configure this in /etc/default/varnish
in the DAEMON_OPTS
variable. But
when I start the daemon, it fails with the following message:
* Starting HTTP accelerator varnish
...fail!
Error: -j must be the first argument
When I put the -j
argument as the first argument in the DAEMON_OPTS
, it
still fails. This is because in /etc/init.d/varnish
-P ${PIDFILE}
comes
before ${DAEMON_OPTS}
.
The only way to configure it is to edit /etc/init.d/varnish
, but that doesn't seem like a good idea.
Cheers,
Bert
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.