Code Monkey home page Code Monkey logo

redis's Introduction

Redis backends

This package provides two different Redis backends. If you want to use Redis as cache backend, you have to choose one of the two, but you cannot use both at the same time.

PhpRedis

This implementation uses the PhpRedis PHP extension. In order to use it, you will need to compile the extension yourself.

Predis

Support for the Predis PHP library is experimental, but feel free to try it out. You can install the required library using composer. Check out the README.Predis.txt file for more information.

Important notice

This module requires at least Redis 2.4, additionally, the lock backend requires Redis 2.6 to support millisecond timeouts and atomic lock operations.

Getting started

Quick setup

Here is a simple yet working easy way to setup the module.

This method will allow Drupal to use Redis for all caches.

$settings['redis.connection']['interface'] = 'PhpRedis'; // Can be "Predis".
$settings['redis.connection']['host']      = '1.2.3.4';  // Your Redis instance hostname.
$settings['cache']['default'] = 'cache.backend.redis';

To use some Predis goodness, including a redis primary/replica setup you can use a config like this.

$settings['redis.connection']['interface'] = 'Predis'; // Use predis library.
$settings['redis.connection']['replication'] = TRUE; // Turns on replication.
$settings['redis.connection']['replication.host'][1]['host'] = '1.2.3.4';  // Your Redis instance hostname.
$settings['redis.connection']['replication.host'][1]['port'] = '6379'; // Only required if using non-standard port.
$settings['redis.connection']['replication.host'][1]['role'] = 'primary'; // The redis instance role.
$settings['redis.connection']['replication.host'][2]['host'] = '1.2.3.5';
$settings['redis.connection']['replication.host'][2]['port'] = '6379';
$settings['redis.connection']['replication.host'][2]['role'] = 'replica';
$settings['redis.connection']['replication.host'][3]['host'] = '1.2.3.6';
$settings['redis.connection']['replication.host'][3]['port'] = '6379';
$settings['redis.connection']['replication.host'][3]['role'] = 'replica';
$settings['cache']['default'] = 'cache.backend.redis';

Either include the default example.services.yml from the module, which will replace all supported backend services (that currently includes the cache tags checksum service and the lock backends, check the file for the current list) or copy the service definitions into a site specific services.yml.

$settings['container_yamls'][] = 'modules/redis/example.services.yml';

Note that for any of this, the redis module must be enabled. See next chapters for more information.

Is there any cache bins that should never go into Redis?

TL;DR: No.

Redis has been maturing a lot over time, and will apply different sensible settings for different bins; It's today very stable.

Advanced configuration

Choose the Redis client library to use

Note: This is not yet supported, only the PhpRedis interface is available.

Add into your settings.php file:

$settings['redis.connection']['interface'] = 'PhpRedis';

You can replace 'PhpRedis' with 'Predis', depending on the library you chose.

Tell Drupal to use the cache backend

Usual cache backend configuration, as follows, to add into your settings.php file like any other backend:

# Use for all bins otherwise specified.
$settings['cache']['default'] = 'cache.backend.redis';

# Use this to only use it for specific cache bins.
$settings['cache']['bins']['render'] = 'cache.backend.redis';

Tell Drupal to use the lock backend

See the provided example.services.yml file on how to override the lock services.

Tell Drupal to use the queue backend

This module provides reliable and non-reliable queue implementations. Depending on which is to be use you need to choose "queue.redis" or "queue.redis_reliable" as a service name.

When you have configured basic information (host, library, ... - see Quick setup) add this to your settings.php file:

# Use for all queues unless otherwise specified for a specific queue.
$settings['queue_default'] = 'queue.redis';

# Or if you want to use reliable queue implementation.
$settings['queue_default'] = 'queue.redis_reliable';


# Use this to only use Redis for a specific queue (aggregator_feeds in this case).
$settings['queue_service_aggregator_feeds'] = 'queue.redis';

# Or if you want to use reliable queue implementation.
$settings['queue_service_aggregator_feeds'] = 'queue.redis_reliable';

Common settings

Connect to a remote host

If your Redis instance is remote, you can use this syntax:

$settings['redis.connection']['interface'] = 'PhpRedis'; // Can be "Predis".
$settings['redis.connection']['host']      = '1.2.3.4';  // Your Redis instance hostname.
$settings['redis.connection']['port']      = '6379';  // Redis port

Port is optional, default is 6379 (default Redis port).

Compression

Compressing the data stored in redis can massively reduce the nedeed storage.

To enable, set the minimal length after which the cached data should be compressed:

$settings['redis_compress_length'] = 100;

By default, compression level 1 is used, which provides considerable storage optimization with minimal CPU overhead, to change:

$settings['redis_compress_level'] = 6;

Using a specific database

Per default, Redis ships the database "0". All default connections will be use this one if nothing is specified.

Depending on you OS or OS distribution, you might have numerous database. To use one in particular, just add to your settings.php file:

$settings['redis.connection']['base']      = 12;

Connection to a password protected instance

If you are using a password protected instance, specify the password this way:

$settings['redis.connection']['password'] = "mypassword";

Depending on the backend, using a wrong auth will behave differently:

  • Predis will throw an exception and make Drupal fail during early boostrap.

  • PhpRedis will make Redis calls silent and creates some PHP warnings, thus Drupal will behave as if it was running with a null cache backend (no cache at all).

Prefixing site cache entries (avoiding sites name collision)

If you need to differentiate multiple sites using the same Redis instance and database, you will need to specify a prefix for your site cache entries.

Cache prefix configuration attempts to use a unified variable across contrib backends that support this feature. This variable name is 'cache_prefix'.

This variable is polymorphic, the simplest version is to provide a raw string that will be the default prefix for all cache bins:

$settings['cache_prefix'] = 'mysite_';

Alternatively, to provide the same functionality, you can provide the variable as an array:

$settings['cache_prefix']['default'] = 'mysite_';

This allows you to provide different prefix depending on the bin name. Common usage is that each key inside the 'cache_prefix' array is a bin name, the value the associated prefix. If the value is FALSE, then no prefix is used for this bin.

The 'default' meta bin name is provided to define the default prefix for non specified bins. It behaves like the other names, which means that an explicit FALSE will order the backend not to provide any prefix for any non specified bin.

Here is a complex sample:

// Default behavior for all bins, prefix is 'mysite_'.
$settings['cache_prefix']['default'] = 'mysite_';

// Set no prefix explicitely for 'cache' and 'cache_bootstrap' bins.
$settings['cache_prefix']['cache'] = FALSE;
$settings['cache_prefix']['cache_bootstrap'] = FALSE;

// Set another prefix for 'cache_menu' bin.
$settings['cache_prefix']['cache_menu'] = 'menumysite_';

Note that if you don't specify the default behavior, the Redis module will attempt to use the HTTP_HOST variable in order to provide a multisite safe default behavior. Notice that this is not failsafe, in such environment you are strongly advised to set at least an explicit default prefix.

Note that this last notice is Redis only specific, because per default Redis server will not namespace data, thus sharing an instance for multiple sites will create conflicts. This is not true for every contributed backends.

Flush mode

@todo: Update for Drupal 8

Redis allows to set a time-to-live at the key level, which frees us from handling the garbage collection at clear() calls; Unfortunately Drupal never explicitely clears single cached pages or blocks. If you didn't configure the "cache_lifetime" core variable, its value is "0" which means that temporary items never expire: in this specific case, we need to adopt a different behavior than leting Redis handling the TTL by itself; This is why we have three different implementations of the flush algorithm you can use:

  • 0: Never flush temporary: leave Redis handling the TTL; This mode is not compatible for the "page" and "block" bins but is the default for all others.

  • 1: Keep a copy of temporary items identifiers in a SET and flush them accordingly to spec (DatabaseCache default backend mimic behavior): this is the default for "page" and "block" bin if you don't change the configuration.

  • 2: Flush everything including permanent or valid items on clear() calls: this behavior mimics the pre-1.0 releases of this module. Use it only if you experience backward compatibility problems on a production environement - at the cost of potential performance issues; All other users should ignore this parameter.

You can configure a default flush mode which will override the sensible provided defaults by setting the 'redis_flush_mode' variable.

// For example this is the safer mode. $conf['redis_flush_mode'] = 1;

But you may also want to change the behavior for only a few bins.

// This will put mode 0 on "bootstrap" bin. $conf['redis_flush_mode_cache_bootstrap'] = 0;

// And mode 2 to "page" bin. $conf['redis_flush_mode_cache_page'] = 2;

Note that you must prefix your bins with "cache" as the Drupal 7 bin naming convention requires it.

Keep in mind that defaults will provide the best balance between performance and safety for most sites; Non advanced users should ever change them.

Default lifetime for permanent items

@todo: Update for Drupal 8

Redis when reaching its maximum memory limit will stop writing data in its storage engine: this is a feature that avoid the Redis server crashing when there is no memory left on the machine.

As a workaround, Redis can be configured as a LRU cache for both volatile or permanent items, which means it can behave like Memcache; Problem is that if you use Redis as a permanent storage for other business matters than this module you cannot possibly configure it to drop permanent items or you'll loose data.

This workaround allows you to explicity set a very long or configured default lifetime for CACHE_PERMANENT items (that would normally be permanent) which will mark them as being volatile in Redis storage engine: this then allows you to configure a LRU behavior for volatile keys without engaging the permenent business stuff in a dangerous LRU mechanism; Cache items even if permament will be dropped when unused using this.

Per default the TTL for permanent items will set to safe-enough value which is one year; No matter how Redis will be configured default configuration or lazy admin will inherit from a safe module behavior with zero-conf.

For advanturous people, you can manage the TTL on a per bin basis and change the default one:

// Make CACHE_PERMANENT items being permanent once again
// 0 is a special value usable for all bins to explicitely tell the
// cache items will not be volatile in Redis.
$conf['redis_perm_ttl'] = 0;

// Make them being volatile with a default lifetime of 1 year.
$conf['redis_perm_ttl'] = "1 year";

// You can override on a per-bin basis;
// For example make cached field values live only 3 monthes:
$conf['redis_perm_ttl_cache_field'] = "3 months";

// But you can also put a timestamp in there; In this case the
// value must be a STRICTLY TYPED integer:
$conf['redis_perm_ttl_cache_field'] = 2592000; // 30 days.

Time interval string will be parsed using DateInterval::createFromDateString please refer to its documentation:

http://www.php.net/manual/en/dateinterval.createfromdatestring.php

Last but not least please be aware that this setting affects the CACHE_PERMANENT ONLY; All other use cases (CACHE_TEMPORARY or user set TTL on single cache entries) will continue to behave as documented in Drupal core cache backend documentation.

Lock backends

@todo: Update for Drupal 8

Both implementations provides a Redis lock backend. Redis lock backend proved to be faster than the default SQL based one when using both servers on the same box.

Both backends, thanks to the Redis WATCH, MULTI and EXEC commands provides a real race condition free mutexes if you use Redis >= 2.1.0.

Testing

I did not find any hint about making tests being configurable, so per default the tested Redis server must always be on localhost with default configuration.

redis's People

Contributors

berdir avatar csandanov avatar davereid avatar guiajlopes avatar lks90 avatar mkalkbrenner avatar pounard avatar slashrsm avatar znerol avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

redis's Issues

Default prefix $_SERVER['HTTP_HOST'] breaks drush + chained fast

Chained fast needs a consistent backend that is always the same.

When using the redis cache backend and e.g. doing changes in drush, then it writes into the default_ which means that it only invalidates that. When you then access it on the website again, chained fast will check against the different prefix and consider the data in apcu to be still valid.

The only way out that I can see is to drop the default fallback, likely combined with a warning/info on the requirements page about it. warning isn't really good because in managed environments with containers and so on, you are the only one accessing it, so not using a prefix is perfectly fine.

drupal cache not cleared properly with drush cr

Hello,

we use redis as caching backend and i found that the cache is not cleared fully while running the drush cr command. To be specific, the field definitions for entities were still being pulled from the cache which should have been cleared with my previous drush cr command.

Fatal error when redis is loading a dataset

I am restarting redis as part of a deploy and the dataset is around 800MB.
If I try to run a drush command just after the redis restart I get the following:

Fatal error: Uncaught RedisException: LOADING Redis 
is loading the dataset in memory in /var/www/html/web/modules/contrib/redis/src/Cache/PhpRedis.php:99

Is there a way to handle the exception more gracefully? Like waiting for the redis server to be ready

I saw this similar issue but it is closed without any resolution
#10

Exception when add redis settings without enabling the module

When I add redis module and its settings to settings.php I get an exception when I try to enable the module:

$ drush en redis -y
Symfony\Component\DependencyInjection\Exception\LogicException: Service 'cache_tags.invalidator.checksum' for consumer 'cache_tags.invalidator' does not implement Drupal\Core\Cache\CacheTagsInvalidatorInterface. in /srv/app/core/lib/Drupal/Core/DependencyInjection/Compiler/TaggedHandlersPass.php on line 125
 [error]  Drush command terminated abnormally due to an unrecoverable error.
Symfony\Component\DependencyInjection\Exception\LogicException: Service 'cache_tags.invalidator.checksum' for consumer 'cache_tags.invalidator' does not implement Drupal\Core\Cache\CacheTagsInvalidatorInterface. in Drupal\Core\DependencyInjection\Compiler\TaggedHandlersPass->process() (line 125 of /srv/app/core/lib/Drupal/Core/DependencyInjection/Compiler/TaggedHandlersPass.php).

The similar problem when I try to disable redis

composer-install redis/git ?

In the current state, prior to installable release on drupal.org/packagist, what's correct D8-friendly usage for composer-install from this git repo?

I've a composer-installed Drupal 8.0.3 & a variety of modules.

Now, i'd like to install md-systems/redis to starting working on.

1st

cd <drupal root>

then add this git repo locally

composer config repositories.md-systems git https://github.com/md-systems/redis.git

so that

cat <druapl root>/composer.json
    ...
      "repositories": {
        "md-systems": {
          "type": "git",
          "url": "https://github.com/md-systems/redis.git"
        }
      }
    ...

But attempt to

composer require md-systems/redis:dev-8.x-1.x

fails

Loading composer repositories with package information
Updating dependencies (including require-dev)                                         
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - The requested package md-systems/redis could not be found in any version, there may be a typo in the package name.

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your minimum-stability setting
   see <https://getcomposer.org/doc/04-schema.md#minimum-stability> for more details.

Read <https://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.

Installation failed, reverting ./composer.json to its original content.

Is something missing from my method, or the pacakge?

Running drush cr does not clear Redis cache.

Hello,

I noticed that running drush cr does not clear or remove any keys from the Redis cache.

In the deleteAll function, located in src/Cache/PhpRedis.php, if I add in the line $this->client->flushDB();, it will then clear the Redis cache.

Typo in README.md

It seems you have a typo when you describe how to set password, probably should be $settings['redis.connection']['password'] = "mypassword"; instead of $settings['redis.connection']['base'] = "mypassword";

Fatal error when Redis server is down

Using Redis as Drupal Cache. Should throw the module a fatal error (site down) when Redis server is down? There is a way to avoid that?. Maybe having a warning on Status page

Dependency issue with installing redis

Hi,

Currently we trying to use redis cache (Drupal 8) while doing that we run into this installation issue.

We work we a team and multiple environments. We're doing a 'code driven' approach and deploy from git.

In order to enable the redis cache the module needs to be enabled first. After that you put the config in settings.php in order to use redis.
While deploying instances and/or doing a site install this is an issue. It fails since settings.php already contains the redis config before the module is enabled.

What is the preferred way in enabling redis cache when working in teams with multiple environments?

Document how use redis for cache_container

That bin is special because it's used before the actual container is used, as it contains the actual container.

To use it, there are two requirements.

  1. redis must be in the hardcoded composer autoload namespaces or it won't find the classes. For that to work, the top-level composer.json must contain redis as a module. and composer dump-autoloader needs to be called (or anything that calls it). This needs to be better documented, maybe an alternative would be to manually load the required fields through settings.php. clarify and properly document this.
  2. You need to put this into your settings.php:
  // Use redis for container cache.
  $settings['bootstrap_container_definition'] = [
    'parameters' => [],
    'services' => [
      'redis.factory' => [
        'class' => 'Drupal\redis\ClientFactory',
      ],
      'cache.backend.redis' => [
        'class' => 'Drupal\redis\Cache\CacheBackendFactory',
        'arguments' => ['@redis.factory', '@cache_tags_provider.container'],
      ],
      'cache.container' => [
        'class' => '\Drupal\redis\Cache\PhpRedis',
        'factory' => ['@cache.backend.redis', 'get'],
        'arguments' => ['container'],
      ],
      'cache_tags_provider.container' => [
        'class' => 'Drupal\redis\Cache\RedisCacheTagsChecksum',
        'arguments' => ['@redis.factory'],
      ],
    ],
  ];

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.