Code Monkey home page Code Monkey logo

dio-http-cache's Introduction

dio-http-cache

Pub

中文介绍

Dio-http-cache is a cache library for Dio ( http client for flutter ), like Rxcache in Android.

Dio-http-cache uses sqflite as disk cache, and LRU strategy as memory cache.

Inspired by flutter_cache_manager.

Add Dependency

dependencies:
  dio_http_cache: ^0.3.x #latest version

QuickStart

  1. Add a dio-http-cache interceptor in Dio :

    dio.interceptors.add(DioCacheManager(CacheConfig(baseUrl: "http://www.google.com")).interceptor);
  2. Set maxAge for a request :

    Dio().get(
      "http://www.google.com",
      options: buildCacheOptions(Duration(days: 7)),
    );

The advanced

  1. Custom your config by buildCacheOptions :

    1. primaryKey: By default, host + path is used as the primaryKey, and you can also customize it.

    2. subKey: By default, query ( data or queryParameters) is used as subKey, and you can specify the subKey when it's necessary, for example:

      buildCacheOptions(Duration(days: 7), subKey: "page=1")
    3. maxAge: set the cache time. If the value is null or not setted, it will try to get maxAge and maxStale from response headers.

    4. maxStale: set stale time. When an error (like 500,404) occurs before maxStale, try to return cache.

      buildCacheOptions(Duration(days: 7), maxStale: Duration(days: 10))
    5. forceRefresh: false default.

      buildCacheOptions(Duration(days: 7), forceRefresh: true)
      1. Get data from network first.
      2. If getting data from network succeeds, store or refresh cache.
      3. If getting data from network fails or no network avaliable, try get data from cache instead of an error.
  2. Use "CacheConfig" to config default params

    1. baseUrl: it’s optional; If you don't have set baseUrl in CacheConfig, when you call deleteCache, you need provide full path like "https://www.google.com/search?q=hello", but not just "search?q=hello".
    2. encrypt / decrypt: these two must be used together to encrypt the disk cache data, you can also zip data here.
    3. defaultMaxAge: use Duration(day:7) as default.
    4. defaultaMaxStale: similar with DefaultMaxAge.
    5. databasePath: database path.
    6. databaseName: database name.
    7. skipMemoryCache: false defalut.
    8. skipDiskCache: false default.
    9. maxMemoryCacheCount: 100 defalut.
    10. defaultRequestMethod: use "POST" as default, it will be used in delete caches.
    11. diskStore: custom disk storage.
  3. How to clear expired cache

    • Just ignore it, that is automatic.

    • But if you insist : DioCacheManager.clearExpired();

  4. How to delete caches

    1. No matter what subKey is, delete local cache if primary matched.

      // Automatically parses primarykey from path
      _dioCacheManager.deleteByPrimaryKey(path, requestMethod: "POST"); 
    2. Delete local cache when both primaryKey and subKey matched.

      // delete local cache when both primaryKey and subKey matched.
      _dioCacheManager.deleteByPrimaryKeyAndSubKey(path, requestMethod: "GET"); 

      INPORTANT: If you have additional parameters when requesting the http interface, you must take them with it, for example:

      _dio.get(_url, queryParameters: {'k': keyword}, 
      	options: buildCacheOptions(Duration(hours: 1)))
      //delete the cache:
      _dioCacheManager.deleteByPrimaryKeyAndSubKey(_url, requestMethod: "GET", queryParameters:{'k': keyword}); 
      _dio.post(_url, data: {'k': keyword}, 
      	options: buildCacheOptions(Duration(hours: 1)))
      //delete the cache:
      _dioCacheManager.deleteByPrimaryKeyAndSubKey(_url, requestMethod: "POST", data:{'k': keyword}); 
    3. Delete local cache by primaryKey and optional subKey if you know your primarykey and subkey exactly.

      // delete local cache by primaryKey and optional subKey
      _dioCacheManager.delete(primaryKey,{subKey,requestMethod});
  5. How to clear All caches (expired or not)

    _dioCacheManager.clearAll();
  6. How to know if the data come from Cache

    if (null != response.headers.value(DIO_CACHE_HEADER_KEY_DATA_SOURCE)) {
     	// data come from cache
    } else {
     	// data come from net
    }

Example for maxAge and maxStale

_dio.post(
	"https://www.exmaple.com",
	data: {'k': "keyword"},
	options:buildCacheOptions(
  		Duration(days: 3), 
  		maxStale: Duration(days: 7), 
	)
)
  1. 0 ~ 3 days : Return data from cache directly (irrelevant with network).
  2. 3 ~ 7 days:
    1. Get data from network first.
    2. If getting data from network succeeds, refresh cache.
    3. If getting data from network fails or no network avaliable, try get data from cache instead of an error.
  3. 7 ~ ∞ days: It won't use cache anymore, and the cache will be deleted at the right time.

License

Copyright 2019 Hurshi

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

 http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

dio-http-cache's People

Contributors

erickok avatar franticn avatar hurshi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dio-http-cache's Issues

Make all async method await-able

Currently, almost all methods inside cache manager are asynchronous by nature but they are not marked as async, so you cannot await them to make a delete operation before retrieving from the cache.

delete(String key, {String subKey}) {
    key = _convertMd5(key);
    if (null != subKey) subKey = _convertMd5(subKey);

    _memoryCacheStore?.delete(key, subKey: subKey);
    _diskCacheStore?.delete(key, subKey: subKey);
  }

How to use normal options and cached options together?

Hi,
When we use dio , we can set options as :

Dio dio = Dio();
Options options = Options(
  headers: {
    'Accept-Language': 'en',
    'Authorization': 'Token 12345',
  },
);

Response response = await Dio().get("http://www.google.com", options: options);

And when use dio-http-cache. we set options as:

DioCacheManager dioCacheManager = DioCacheManager(CacheConfig());
Options cacheOptions = buildCacheOptions(Duration(days: 7));
Dio dio = Dio();
dio.interceptors.add(dioCacheManager.interceptor);

Response response = await Dio().get("http://www.google.com", options: cacheOptions);

Now how we can use theses options together?

Support for ETag and If-None-Match headers

Hello,

I love your library, it's a really nice plug-and-play addon.

I could not find anything in the docs regarding ETag and If-None-Match HTTP headers. I am using those to diminish network traffic between my app and my back-end.

Are these supported out of the box? If not, I'd be glad to try and implement this feature.

Custom key

DOC: dio-http-cache uses url as key.
Letting the user define custom keys for caching would enable users to share cache between requests.

Such a feature is interesting in the following example:
http://example.com/a/b
http://example.com/a/b?timestamp=1566563506431
http://example.com/a/b?timestamp=1566563506433
The API needs a timestamp GET parameter that cannot be shared between requests, be we would like to save them as ONE request in sqflite. (for MaxStale fallback)

The above case can also be solved by providing queryParameters at the buildCacheOptions level, where they could be ignore by the hash method.

Cache location

Where is the disk cache saved? If it's saved to a temp directory how can one change that to a permanent directory?

从缓存读取状态码的bug

一个搜索功能的接口,通过http状态码判断请求状态,当状态码为200代表正常数据返回,204的时候代表没有数据,缓存之后原来是204的数据再次请求命中缓存却变成了200

Cache doesn't return any data if the duration of max stale has passed, even if the cache Duration itself hasn't passed.

According to the documentation:

Example for maxAge and maxStale #


 _dio.post(
	"https://www.exmaple.com",
	data: {'k': "keyword"},
	options:buildCacheOptions(
 		Duration(days: 3), 
  		maxStale: Duration(days: 7), 
 	)
 )

0 ~ 3 days : Return data from cache directly (irrelevant with network).
3 ~ 7 days:
Get data from network first.
If getting data from network succeeds, refresh cache.
If getting data from network fails or no network avaliable, try get data from cache instead of an error.
7 ~ ∞ days: It won't use cache anymore, and the cache will be deleted at the right time.

Yet it is not getting the data from the cache in the 3-7 days range, instead it just throws an error of failed host lookup (Airplane mode was turned on to test this). Any solutions to this?

can I use dio http cache with provider?

`Future<List<People>> getDataName() async {
    DioCacheManager _dioCacheManager;
  _dioCacheManager = DioCacheManager(CacheConfig());

  Options _cacheOptions =
  buildCacheOptions(Duration(days: 7), forceRefresh: true,);
  Dio _dio = Dio();
  _dio.interceptors.add(_dioCacheManager.interceptor);
  Response response = await _dio.get(
  'https://firebasestorage.googleapis.com/v0/b/sample-awka.appspot.com/o/people.json?alt=media',
  options: _cacheOptions);

   List<People> _persons =
       (response.data as List).map((pr) {
      print('$pr');
      
      People prt = People.fromJson(pr);
    }).toList();
return _persons;
}`

`Provider.of<People>(context).getDataName();`

Make CacheStore configurable

It would be nice to be able to pass in your own version of CacheStore so that users are able to implement using different databases, such as Hive or Moor.

关于通过拦截器设置缓存策略的几点疑问?

@hurshi 您好
首先很感谢您开源的网络框架 我看了您的文档 现在有如下疑问:
1、通过拦截器设置的缓存策略是否可以称之为全局缓存策略?也就是设置全局缓存策略后,同一个baseUrl下的所有请求都会采用这一策略进行缓存,所以只有当个别请求需要采用和全局缓存不一致的策略的时候 才需要在请求的时候通过option进行针对性设置;
2、通过拦截器设置缓存策略的baseUrl参数如果为空 是否就代表针对后续的所有请求(不管其baseUrl是什么),该缓存策略都会生效?或者baseUrl 是否支持多个取值?
3、目前通过拦截器设置的缓存策略(全局策略 如果我理解无误的话),是否可以支持指定接口请求类型的缓存过滤(比如,只针对GET进行缓存 排除POST PUT等)

Usage questions

First of all thanks for your amazing package!

I see in the README that you need to enable and interceptor with a BaseUrl.
How this exactly works? is mandatory? I mean i can only set the CacheManager with no baseUrl?
All these questions arose because i have a case when I want to cache some requests but others don't, how do i should do that? set in the request maxAge to 0?

CacheConfig.defaultMaxAge is ignored by DioCacheManager

Example:

rest.interceptors.add(
      DioCacheManager(
        CacheConfig(
          baseUrl: rest.options.baseUrl,
          defaultMaxAge: Duration(minutes: 5),
        ),
      ).interceptor,
    );

defaultMaxAge is NOT used in case server does not return any max-age in response.
The bug is probably in:

Future<bool> _pushToCache(Response response) {
    RequestOptions options = response.request;
    Duration maxAge = options.extra[DIO_CACHE_KEY_MAX_AGE];
    Duration maxStale = options.extra[DIO_CACHE_KEY_MAX_STALE];
    if (null == maxAge) {
      _tryParseHead(response, maxStale, (_maxAge, _maxStale) {
        maxAge = _maxAge;
        maxStale = _maxStale;
      });
    }
    
    // FIX
    if (null == maxAge) {
        maxAge = _manager?._config.defaultMaxAge;
        maxStale = _manager?._config.defaultMaxStale;
    }
    // End of FIX

    if (null == maxAge) return Future.value(false);

See also comments in #12

Error while using

[VERBOSE-2:ui_dart_state.cc(148)] Unhandled Exception: DioError [DioErrorType.DEFAULT]: NoSuchMethodError: The method 'trim' was called on null.
Receiver: null
Tried calling: trim()
#0 Object.noSuchMethod (dart:core-patch/object_patch.dart:51:5)
#1 Headers.set (package:dio/src/headers.dart:61:27)
#2 Headers.add (package:dio/src/headers.dart:49:29)
#3 DioCacheManager._buildResponse. (package:dio_http_cache/src/manager_dio.dart:65:47)
#4 _LinkedHashMapMixin.forEach (dart:collection-patch/compact_hash.dart:377:8)
#5 DioCacheManager._buildResponse (package:dio_http_cache/src/manager_dio.dart:65:21)
#6 DioCacheManager._onError (package:dio_http_cache/src/manager_dio.dart:58:16)

#7 InterceptorsWrapper.onError (package:dio/src/interceptor.dart:125:14)

#8 DioMixin._request._errorInterceptorWrapper. (package:dio/src/dio.dart:846:40)

#9 _rootRunUn<…>

请支持dio 3.0

这个插件挺好,目前dio 3.0已发布,希望能尽快支持,当支持3.0后,我们会将此插件归档到dio文档中。

MemoryCache is not invalidated when using deleteByPrimaryKey

Hello Hurshi. The memory is not invalidated because it should support "where" clause. Let me elaborate
mapCache.invalidate("${key}${subKey ?? ""}");
this line deletes only one record with the specified key and subkey, but what we actually want to achieve is to delete all the occurrences where key is present. So, it clears the disk cache, but the memory still stays the same.

切换网络会产生bug

切换网络以后,请求接口数据变化,但是只能请求到旧的数据,而且response的code和status都是null

Feature to check if data is from cache

Hi , thanks for the library.I wanted to ask if its possible to know if the response is cached data or not.This way i will be able to notify UI that the data is cached and may not be updated.Thanks

Response data type is String when setting ResponseType.bytes

Wrapping the options property of a get request with buildCacheOptions in this way:
dio.get( url, options: buildCacheOptions(Duration(days: 7), options: Options(responseType: ResponseType.bytes)) );

The response data property is an String that contains correct data ("[1,23,232...]") but it's not a List of ints ([1,23,232]).
Is that the expected behaviour , or it's a bug?

[Question] forceRefresh on config object ?

I would like to use this option as "defaultConfig"
buildCacheOptions(Duration(days: 7), forceRefresh: true)

I do not see the forceRefresh property in the CacheConfig object ?
Is it possible or do we need to buildCacheOption for each request requiring forceRefresh to true ?

没有网络时依旧使用了缓存

没有网络的情况下,默认强制刷新的数据也使用了缓存,我希望没有网络的时候正常调用,不使用缓存; 使用clearAll();清掉之后依旧还是使用了缓存,
image
设置maxStale天数也没有生效,
dio版本dio: 3.0.4
diohttp_cache:0.2.0
flutter:1.9.1+hotfix.6

Why when closing an app cache cleared

Hello i'm beginner

Why when closing an app cache cleared.

var _dio = Dio();
var _dioCacheManager = new DioCacheManager(CacheConfig(baseUrl: url));
_dio.interceptors.add(_dioCacheManager.interceptor);

var response = await _dio.post("${url}",
    data: params,
    options: buildCacheOptions(
      Duration(days: 3),
      maxStale: Duration(days: 7),
    ));

Usage questions

Thank you @hurshi for the awesome package! I got a few questions, maybe you can add the answers in the future to README?

  • how to invalidate manually the cache? should I set max-age to 0? (when user makes pull to refresh for example)
  • how to delete ALL items from cache (when the user logs in with another account)

缓存primaryKey 能将请求类型也加入么? 避免同一个path 不同请求类型 缓存出现错乱

1、目前我同一个path 有一个GET请求 一个是PUT请求 发现两者缓存会出现混乱 GET请求的时候会加载到PUT请求的缓存 虽然目前通过认为设置subKey避免了 但觉得可以优化一下
2、希望可以设置可以缓存的请求类型 ,或者缓存只针对GET请求 对于非GET请求 比如POST/PUT等操作类请求默认不缓存 或者可以设置为不缓存

`cache-control` not properly parsed

When relying on response headers for max cache time, i.e. no manually set max age, the cache-control header is not properly parsed, causing the whole caching to fail.

To reproduce, set up a dio get with options: buildServiceCacheOptions(). The request will never be served from cache.

I tried to debug the library and found that the cache-control header value is noty properly parsed in_tryParseHead. Currently this reads:

parameters = HeaderValue.parse(cacheControl,
                parameterSeparator: ",", valueSeparator: "=")

But the HeaderValue.parse method expects the header name to be present in the given String, so it should be:

parameters = HeaderValue.parse(HttpHeaders.cacheControlHeader + ": " + cacheControl,
                parameterSeparator: ",", valueSeparator: "=")

Cache is not working on mobile device, but works perfectly on Emulators

Hi,

Just implemented the cache plugin and it works perfectly on Android Emulator, returns data from cache when Airplane mode is on. But when the app is deployed on a physical device, either in debug or release mode, the cache doesn't work at all. No error messages can be seen in the debug log as well. What am I missing here?

Usage questions offline mode

Can you please detail in the documentation how is managed the case when no network is available ? Is there an error or the data is returned by the cache ?

从缓存获取数据时,baseURL为空,导致报错

flutter:
flutter: ╔╣ DioError ║ DioErrorType.DEFAULT
flutter: ║  RangeError (index): Invalid value: Only valid value is 0: 1
flutter: ╚══════════════════════════════════════════════════════════════════════════════════════════
flutter: DioError [DioErrorType.DEFAULT]: RangeError (index): Invalid value: Only valid value is 0: 1
#0      List.[] (dart:core-patch/growable_array.dart:147:60)
#1      RequestOptions.uri (package:dio/src/options.dart:299:29)
#2      DioCacheManager._getPrimaryKeyFromOptions (package:dio_http_cache/src/manager_dio.dart:170:41)
#3      DioCacheManager._pullFromCacheBeforeMaxStale (package:dio_http_cache/src/manager_dio.dart:92:9)
#4      DioCacheManager._onError (package:dio_http_cache/src/manager_dio.dart:64:41)
<asynchronous suspension>
#5      InterceptorsWrapper.onError (package:dio/src/interceptor.dart:125:14)
<asynchronous suspension>
#6      DioMixin._request._errorInterceptorWrapper.<anonymous closure> (package:dio/src/dio.dart:846:40)
<asynchronous suspension>
#7      _rootRunUnary (dart:async/zone.dart:1132:38)
#8      _CustomZone.runUnary (dart:async/zone.dart:1029:19)
#9      _FutureListener.handleError (dart:async/future_impl.dart:155:20)
#10     Future._propagateToListeners.handl<…>
flutter: NoSuchMethodError: The getter 'login' was called on null.
Receiver: null

http.get()参数设置疑问

在编码调试中发现,网络断了,未使用缓存中的数据。

查看文档发现,有两种方式设置:

  1. 在http.get()中设置options参数
  2. 在拦截器中设置,使用CacheConfig,有默认值

如果这两种方式同时设置,参数如何生效?

// 1. 在请求配置了参数options
Response response = await authHttp.get('/user/followers',
          queryParameters: {'page': page, 'per_page': pageSize},
          options: buildOptions(forceRefresh))

// 2. method
Options buildOptions(bool forceRefresh) {
  return buildCacheOptions(Duration(days: 7), forceRefresh: forceRefresh);
}

// 3. 在拦截器中配置了CacheConfig
DioCacheManager(CacheConfig(
                baseUrl: Constants.GITHUB_API_PREFIX,
                databaseName: 'Cache',
                defaultMaxStale: Duration(days: 10)))
            .interceptor,

Flutter Web support

Flutter Web will be stable in the coming month.
It would be very nice to have WEB support for our current app with minimum changes.
My current application is using http-cache and due to sqlite dependency, the app can not run on WEB.
A solution will be to use sembast plugin (WEB ready).

How to limit disk cache size?

I've been diggling through the codebase, and while I see MaxMemoryCacheCount option, there is nothing I'm aware of which would limit the on-disk cache.

How do we ensure the SQLite cache doesn't eat up too much space on client device ? Is there any way of approaching this, currently ?

If not, I'd be happy to implement this!

能加个按照最大缓存数量吗

作者你好:
在我们项目中遇到了需要按照缓存文件数量来清理缓存的需求,或者说按照缓存磁盘大小来清理。当达到文件数量或者磁盘空间时,把老的数据清理掉~肯请作者支持下。 mua.. 😄

Clear cache based on resource not key

Hello, @hurshi I'm using this package and it's awesome! What do you think, right now we can delete the resource via key and subkey but if we want to delete an entire resource what to do?
E.g, I have a https://resource.com?page=1 and 100 pages of it but I need to refresh the whole resource, not only a particular page. From my perspective, I see two solutions: add key as the path and subkey as the query by default or add a wildcard delete that will take only the URL path into consideration.
Have a nice day ✨

Enhancement: Make cache respect HTTP cache headers

There are multiple cache control headers sent as part of an HTTP response. It would be nice if cache expiration was controlled by those headers by default, and then override using the existing mechanisms.

无法清理单个缓存

你在插入缓存的时候key值使用了_getPrimaryKeyFromOptions转换,清除的时候并没有使用_getPrimaryKeyFromOptions转换,导致key值无法匹配,最好在清除的时候也用_getPrimaryKeyFromOptions转换,并且传递options.method。

Preserve response headers

It seems that the headers of network responses are discarded, and when a response is built from the cache it uses the request headers instead.

In my use case the response headers for a certain request contain crucial link information for paginated data. For example, I make a request to an API that returns 100 items in the first 'page', and the headers contain a link to the next 'page' that also returns 100 items and also has link headers for the subsequent page, and so forth. However, if the response is a cached response then these headers will be missing, which means that I cannot access more than a single page of data even if the subsequent pages have been previously cached.

Preserving the original headers and restoring them in the cached response would solve this issue.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.