victoralbertos / rxcache Goto Github PK

Say I have an api that gets entities by id and a list of all entities.

I cache each entity by id, but when the application starts I would like to get all entities and cache each one by their id, thus when a get by id is called if the item has been pre cached then it is returned from cache, else it is loaded from the api by id.

Thanks!

Hello,
I'm just starting playing around with RxCache. Can it handle somehow configuration changes? What I mean by that is a typical use case:

1. User starts an expensive network call.
2. User rotates screen - request is still running in the background.
3. After activity recreation it subscribes to ongoing call.

I've done some simple tests and it looks like on activity recreation the call is being reset.

what happens when I provide @LifeCache(duration = 0, timeUnit = TimeUnit.MINUTES) ? It seems like it is caching for ever ?

io.rx_cache.internal.ProxyProviders$RxCacheException: The Loader provided did not return any data and there is not data to load from the Cache getNewsList

Currently the library depends on Gson to serialise and deserialise objects. It is needed to abstract this feature into some interface and let the client to be who provides its custom vendor. For that matter, at least one built-in json convertor based on Gson it is needed to be implemented too.

Hey,

When updating to RxJava 2, I am getting this error :

Caused by: java.lang.ClassCastException: Couldn't convert result of type rx.Observable to io.reactivex.Observable

at this line of code (invoking the dataCached function), when retrieving data from the cache :

Observable apiObservable = ...
Observable observableCache = providers.getArticles(apiObservable, new EvictDynamicKey(isload)).map(
            new dataCached<List<Article>>());

Here is dataCached function :

    private  class dataCached<T> implements Function<Reply<T>, T> {
            @Override public T apply(Reply<T> httpResult) throws Exception  {
            return httpResult.getData();
        }

Anyway to solve this ? Thanks :)

First: Thx for this awesome library. 😄

I'm using a simple RXCache implementation with no encryption on Android with Retrofit2. Everything works fine. But on every start I get a NullPointerException, because the BuiltInEncryptor tries to generate a secret Key in generateSecretKey(String key). Due to no encryption the string key is null. Here is the head of the stackTrace:

06-29 14:04:11.647 7209-7235/? W/System.err: java.lang.NullPointerException: Attempt to invoke virtual method 'byte[] java.lang.String.getBytes(java.lang.String)' on a null object reference
06-29 14:04:11.647 7209-7235/? W/System.err:     at io.rx_cache.internal.encrypt.BuiltInEncryptor.generateSecretKey(BuiltInEncryptor.java:81)
06-29 14:04:11.647 7209-7235/? W/System.err:     at io.rx_cache.internal.encrypt.BuiltInEncryptor.initCiphers(BuiltInEncryptor.java:67)
06-29 14:04:11.647 7209-7235/? W/System.err:     at io.rx_cache.internal.encrypt.BuiltInEncryptor.decrypt(BuiltInEncryptor.java:55)
06-29 14:04:11.647 7209-7235/? W/System.err:     at io.rx_cache.internal.encrypt.FileEncryptor.decrypt(FileEncryptor.java:53)
06-29 14:04:11.647 7209-7235/? W/System.err:     at io.rx_cache.internal.Disk.retrieveRecord(Disk.java:187)
06-29 14:04:11.647 7209-7235/? W/System.err:     at io.rx_cache.internal.cache.EvictExpiredRecordsPersistence.startEvictingExpiredRecords(EvictExpiredRecordsPersistence.java:48)
06-29 14:04:11.647 7209-7235/? W/System.err:     at io.rx_cache.internal.ProxyProviders$1.call(ProxyProviders.java:59)
06-29 14:04:11.647 7209-7235/? W/System.err:     at io.rx_cache.internal.ProxyProviders$1.call(ProxyProviders.java:57)

The root cause seems to be in EvictExpiredRecordsPersistence. The Method persistence.retrieveRecord(key, false, getEncryptKey.getKey()); returns null for a key and thats why he tries to get the record with encryption enabled.

Is this a Problem in RXCache or am I missing a configuration?

Thx for the help.

My Config:

public interface UserCache {

  Observable<Reply<UserDto>> usersFindOne(
      Observable<UserDto> oUser, DynamicKey userId, EvictDynamicKey update);

  Observable<Reply<List<UserDto>>> usersFindAll(
      Observable<List<UserDto>> oUsers, EvictProvider update);

.... more methods
}

public interface RouteCache {

  Observable<Reply<RouteDto>> routesFindOne(
      Observable<RouteDto> oRoute, DynamicKey id, EvictDynamicKey update);

  Observable<Reply<List<RouteDto>>> routesFindAll(
      Observable<List<RouteDto>> oRoutes, EvictProvider evictProvider);

...more methods

public class CacheFactory {

  private static <S> S createCache(Class<S> serviceClass, File cacheDir){
    return new RxCache.Builder()
        .useExpiredDataIfLoaderNotAvailable(true)
        .persistence(cacheDir, new JacksonSpeaker())
        .using(serviceClass);
  }

  public static UserCache getUserCache(File cacheDir){
    return createCache(UserCache.class, cacheDir);
  }

  public static RouteCache getRouteCache(File cacheDir){
    return createCache(RouteCache.class, cacheDir);
  }
}

Hi, this is a minor thing, but I recently discovered that if you configure RxCache with a directory that does not exist (but can be created - so there are no access rights issues, someone just needs to call mkdirs on it), you are going to get the following exception:

io.rx_cache.RxCacheException: The Loader provided did not return any data and there is not data to load from the Cache getEpisodes
(Plus a very very long stack trace of course)

This I find a little dangerous, because it makes you start debugging the wrong parts of your code. :D Only if you go through the whole stack trace, on the bottom you can see that that it was caused by a file not found error. And even that isn't a very good hint, because if you don't know how the persistent cache works internally, you are going to assume that this just means there indeed isn't any data cached.

So I would propose to either:

• Make sure the directory exists and is writeable right during the configuration phase
• Or make the error handling a bit more flexible so that not everything looks like it was caused by an empty loader

(And I'll gladly make a pull request, just tell me which one seems better to you)

i use retrofit + rxjava + rxcache in my demo,and the response json data format is

{
 "code": 200,
 "data": {}
}

so， i define a BaseResponse.class

public class BaseResponse<T> {
    public int code;
    public T datas;
}

Observable<BaseResponse<HomeDatasObj>> getHomeData();

but an error has occurred，java.lang.ClassCastException: com.google.gson.internal.LinkedTreeMap cannot be cast to HomeDatasObj，I don't know how to solve this question.

Hi,

So I'm about to use your framework but I see that using a Disk File for caching is mandatory. Can you make it NOT mandatory since I'm interested in your memory caching only.

This is useful with applications where we don't want to store cached data on the filesystem for security reasons.

Thanks!

Hi. When I try to add your library into my project, another library stops working!
emilsjolander/IntentBuilder#22

Not sure what is wrong

This doesn't seems to be working for me with retrofit

fot inserting :-
MyApplication.cacheProviders.getSignInToken(Observable.just(signInToken));

while fetching :-

 MyApplication.cacheProviders.getSignInToken(Observable.<SignInToken>just(null)).map(new Func1<SignInToken, String>() {
            @Override
            public String call(SignInToken signInToken) {
                Log.v(MyWorkoutsFragment.class.getName(),signInToken.getToken());
                return signInToken.getToken();
            }
        }).subscribe(new Subscriber<String>() {
            @Override
            public void onCompleted() {

            }

            @Override
            public void onError(Throwable e) {

            }

            @Override
            public void onNext(String s) {
                Toast.makeText(getActivity().getApplicationContext(),"Got Token",Toast.LENGTH_LONG).show();
            }
        });

What am i doing wrong ?

i have activate proguard
and my project cant be build

i have added
-dontwarn io.rx_cache.** to my proguard-rules.pro but it seems still failed

here are log error from your project link
i have actived minifyEnabled:true and build using ./gradlew clean assembleRelease

Note: there were 9 unresolved dynamic references to classes or interfaces.
      You should check if you need to specify additional program jars.
      (http://proguard.sourceforge.net/manual/troubleshooting.html#dynamicalclass)
Note: there were 11 accesses to class members by means of introspection.
      You should consider explicitly keeping the mentioned class members
      (using '-keep' or '-keepclassmembers').
      (http://proguard.sourceforge.net/manual/troubleshooting.html#dynamicalclassmember)
Warning: there were 1615 unresolved references to classes or interfaces.
         You may need to add missing library jars or update their versions.
         If your code works fine without the missing classes, you can suppress
         the warnings with '-dontwarn' options.
         (http://proguard.sourceforge.net/manual/troubleshooting.html#unresolvedclass)
Warning: there were 1 unresolved references to library class members.
         You probably need to update the library versions.
         (http://proguard.sourceforge.net/manual/troubleshooting.html#unresolvedlibraryclassmember)
Warning: Exception while processing task java.io.IOException: Please correct the above warnings first.
:sample_android:transformClassesAndResourcesWithProguardForRelease FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sample_android:transformClassesAndResourcesWithProguardForRelease'.
> java.io.IOException: Please correct the above warnings first.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 8.612 secs

have you trying build using proguard activate..? any suggestion proguard config for rxcache ? thanks

Can you evict a certain observable from the cache without retrieving a new observable? With EvictProvider currently, if I pass in true, I evict and then retrieve a new observable immediately. Is there a way to just evict certain observables?

EDIT: Duplicate

any sample for how to implement custom persistence mechanism using sqlite/orm or Realm ?

the RxCache class is the entry of whole cache system, why make it internal?

I'm

I have few questions about RX-cache.

1. Is it possible using RX-cache to filter data like fetch all live(live = 1) record.
2. How can we sort our object on the basis of some fields.

If this is possible, then how can we achieve this. Any example will be helpful for me.

Thanks in advance.

When used along with Databinding and dagger, App doesn't compile and breaks the Data binding stuff.

First of all, great work! A powerful library to apply together with Retrofit.

However, do you really need so many dependencies? As an Android Developer, I always work on minimizing method count. As far as I'm concerned, it's unnecessary to import Dagger and Guava. These will cause growth of method count.

why don't allow set cache key , currently use method name.

Due to ProviderKey Generation it is not possible to have multiple cache interfaces with the same method names.

It would be great if the interface name gets added to the ProviderKey.

public interface UserCache {

  Observable<Reply<UserDto>> findOne(
      Observable<UserDto> oUser, DynamicKey userId, EvictDynamicKey update);

  Observable<Reply<List<UserDto>>> findAll(
      Observable<List<UserDto>> oUsers, EvictProvider update);

.... more methods
}

public interface RouteCache {

  Observable<Reply<RouteDto>> findOne(
      Observable<RouteDto> oRoute, DynamicKey id, EvictDynamicKey update);

  Observable<Reply<List<RouteDto>>> findAll(
      Observable<List<RouteDto>> oRoutes, EvictProvider evictProvider);

...more methods

i use retrofit + rxjava + rxcache in my project,and the response json data format is:

{"data":{},"status":200}

i define a BaseResponse.class ps : data is a object not list:

public class BaseResponse<T> { public int status; public T data; }

i define a cache interface method :
@LifeCache(duration = 20, timeUnit = TimeUnit.MINUTES) Observable<Reply<BaseResponse<UserBasicInfoResponse>>> loadUserBasicInfo(Observable<BaseResponse<UserBasicInfoResponse>> userBasicInfoResponseObservable , DynamicKey userId , EvictProvider evictProvider);

when racache load from memory it's ok,but raCache load from disk an error has occurred(java.lang.ClassCastException: com.google.gson.internal.LinkedTreeMap ),I don't know how to solve this question.

when i debug I find diskCache lose type "UserBasicInfoResponse", do you have any advice about it ?

thinks!

The first one, throws errors like this one:
EvictDynamicKey was provided but not was provided any DynamicKey

To be more specific, this is the stacktrace:

Fatal Exception: java.lang.IllegalArgumentException
matchesCacheInfo EvictDynamicKey was provided but not was provided any DynamicKey
 Raw
io.rx_cache.internal.ProxyTranslator.checkIntegrityConfiguration (ProxyTranslator.java:133)
io.rx_cache.internal.ProxyTranslator.processMethod (ProxyTranslator.java:45)
io.rx_cache.internal.ProxyProviders.invoke (ProxyProviders.java:78)
$Proxy1.getChatMessages (Unknown Source)
com.wwn.kickoff.data.repository.ChatDataRepository.getMergedMessages (ChatDataRepository.java:176)

It has a reference to matchesCacheInfo, but I'm not really calling this method.
Here's the code inside the method ChatDataRepository.getMergedMessages() with the line referenced by the stacktrace.

    private Observable<CacheData<ChatList>> getMergedMessages(final String username, final long userId, boolean refresh)
    {
        ...

        final Observable<ChatList> localCacheCall = cacheProvider.getChatMessages(Observable.just(new ChatList()), keyForUnsentMessages(username, userId), new EvictDynamicKeyGroup(false));

        ...
    }

And here's the CacheProvider

public interface CacheProvider
{
    ...

    ///////////////////////////////
    /// Matches
    ///////////////////////////////

    Observable<MatchesList> getMatches(Observable<MatchesList> matches, DynamicKey key, EvictDynamicKey evict);
    Observable<CacheData.Info> matchesCacheInfo(Observable<CacheData.Info> cacheInfo, DynamicKey key, EvictDynamicKey evict);

    ...

    ///////////////////////////////
    /// Chat
    ///////////////////////////////

    Observable<ChatList> getChatMessages(Observable<ChatList> messages, DynamicKeyGroup key, EvictDynamicKey evict);
    Observable<CacheData.Info> chatCacheInfo(Observable<CacheData.Info> cacheInfo, DynamicKeyGroup key, EvictDynamicKeyGroup evict);

    @LifeCache(duration = 3, timeUnit = TimeUnit.DAYS)
    Observable<String> getChatMessageDraft(Observable<String> messages, DynamicKeyGroup key, EvictDynamicKeyGroup evict);

    ...
}

As you can see, it refers to the method matchesCacheInfo() in the stack trace, but the call itself has nothing to do with that method.

I've investigated it a bit but still have no clue about what is happening.
Also, it's not a consistent thing, it happens occasionally, not every time.

Hi, i want to catch my throwable error from retrofit, but from rxcache just return something like these

Proguard errors:

Warning: io.rx_cache.internal.GuavaMemory: can't find referenced class com.google.common.cache.CacheBuilder
Warning: io.rx_cache.internal.GuavaMemory: can't find referenced class com.google.common.cache.CacheBuilder
Warning: io.rx_cache.internal.GuavaMemory: can't find referenced class com.google.common.cache.CacheBuilder
Warning: io.rx_cache.internal.GuavaMemory: can't find referenced class com.google.common.cache.Cache
Warning: io.rx_cache.internal.GuavaMemory: can't find referenced class com.google.common.cache.Cache
Warning: io.rx_cache.internal.GuavaMemory: can't find referenced class com.google.common.cache.Cache
Warning: io.rx_cache.internal.GuavaMemory: can't find referenced class com.google.common.cache.Cache
Warning: io.rx_cache.internal.GuavaMemory: can't find referenced class com.google.common.cache.Cache
Warning: io.rx_cache.internal.GuavaMemory: can't find referenced class com.google.common.cache.CacheBuilder
Warning: io.rx_cache.internal.GuavaMemory: can't find referenced class com.google.common.cache.Cache
Warning: io.rx_cache.internal.GuavaMemory: can't find referenced class com.google.common.cache.Cache
Warning: io.rx_cache.internal.GuavaMemory: can't find referenced class com.google.common.cache.Cache
Warning: io.rx_cache.internal.GuavaMemory: can't find referenced class com.google.common.annotations.VisibleForTesting
Warning: io.rx_cache.internal.ProxyProviders: can't find referenced class com.google.common.annotations.VisibleForTesting
Warning: io.rx_cache.internal.Record: can't find referenced class com.google.common.annotations.VisibleForTesting
Warning: io.rx_cache.internal.cache.EvictExpirableRecordsPersistence: can't find referenced class com.google.common.annotations.VisibleForTesting
Warning: io.rx_cache.internal.cache.EvictRecord: can't find referenced class com.google.common.annotations.VisibleForTesting
Warning: io.rx_cache.internal.cache.TwoLayersCache: can't find referenced class com.google.common.annotations.VisibleForTesting
Warning: io.rx_cache.internal.cache.TwoLayersCache: can't find referenced class com.google.common.annotations.VisibleForTesting

Possible working solutions:

-dontwarn io.rx_cache.internal.GuavaMemory
-dontwarn io.rx_cache.internal.ProxyProviders
-dontwarn io.rx_cache.internal.Record
-dontwarn io.rx_cache.internal.cache.**

or

-dontwarn io.rx_cache.internal.**

I like second one more because it's future-proof.

When I try to use RxCache in project that uses the latest version of Dagger (2.4 at this point of time), it crashes with java.lang.ClassNotFoundException: Didn't find class "dagger.internal.ScopedProvider" on path: DexPathList because Google introduced some breaking changes in 2.3 (everything works with 2.2).

It is easily reproducible by including Dagger 2.4 into a project that uses RxCache.

The easy solution is to update your Dagger version to 2.4, but new Dagger releases can potentially break it again. The ideal solution for me as user would be to remove Dagger dependency from the library, but I guess that's not an option.

I'm not too familiar with jitpack's process but the POM downloaded using gradle is giving a syntax issue.

https://jitpack.io/com/github/VictorAlbertos/RxCache/0.4.9/RxCache-0.4.9.pom

Lines 20-22 should be removed.

...
  </dependencies>
</project>
</groupId>
`

If I specify some string with "/" symbol as DynamicKey value then RxCache generates own key based on this string and use it as part of path. But "/" symbol is not escaped or removed and it cause OnError when RxCache trying to create the file which directory doesn't exists.

There should be a way to clear all the cached data, from both disk and memory (with one single call).
For example, imagine a scenario where you have a user logged in with all his information cached either in disk and memory.
This user logs out an another user logs in in the same device, there might be some displayed information that actually belongs to the first user.

Of course you could evit every single cache entry you have, but this is very labourous. What if you have 20K individual cache entries?
So that's where a clearAll() method would come in handy.

Hey, After updating to RxJava 2 and use the appropriate RxCache version, I couldn't anymore make API requests,by using HttpLoggingInterceptor nothing happens when invoking "performGetQuiz" method..just an empty screen and logcat.

.debug D/Interactor$1$override: java.lang.IllegalArgumentException: getCachedQuestions EvictDynamicKey was provided but not was provided any DynamicKey

Code : https://gist.github.com/alouanemed/7aa9c2153fd80a31cfc74654743988a9

The first one, throws errors like this one:
EvictDynamicKey was provided but not was provided any DynamicKey

To be more specific, this is the stacktrace:

Fatal Exception: java.lang.IllegalArgumentException
matchesCacheInfo EvictDynamicKey was provided but not was provided any DynamicKey
 Raw
io.rx_cache.internal.ProxyTranslator.checkIntegrityConfiguration (ProxyTranslator.java:133)
io.rx_cache.internal.ProxyTranslator.processMethod (ProxyTranslator.java:45)
io.rx_cache.internal.ProxyProviders.invoke (ProxyProviders.java:78)
$Proxy1.getChatMessages (Unknown Source)
com.wwn.kickoff.data.repository.ChatDataRepository.getMergedMessages (ChatDataRepository.java:176)

It has a reference to matchesCacheInfo, but I'm not really calling this method.
Here's the code inside the method ChatDataRepository.getMergedMessages() with the line referenced by the stacktrace.

    private Observable<CacheData<ChatList>> getMergedMessages(final String username, final long userId, boolean refresh)
    {
        ...

        final Observable<ChatList> localCacheCall = cacheProvider.getChatMessages(Observable.just(new ChatList()), new DynamicKeyGroup(userId, KEY_CHAT_UNSENT_MESSAGES + "-" + username), new EvictDynamicKeyGroup(false));

        ...
    }

And here's the CacheProvider

public interface CacheProvider
{
    ...

    ///////////////////////////////
    /// Matches
    ///////////////////////////////

    Observable<MatchesList> getMatches(Observable<MatchesList> matches, DynamicKey key, EvictDynamicKey evict);
    Observable<CacheData.Info> matchesCacheInfo(Observable<CacheData.Info> cacheInfo, DynamicKey key, EvictDynamicKey evict);

    ...

    ///////////////////////////////
    /// Chat
    ///////////////////////////////

    Observable<ChatList> getChatMessages(Observable<ChatList> messages, DynamicKeyGroup key, EvictDynamicKey evict);
    Observable<CacheData.Info> chatCacheInfo(Observable<CacheData.Info> cacheInfo, DynamicKeyGroup key, EvictDynamicKeyGroup evict);

    @LifeCache(duration = 3, timeUnit = TimeUnit.DAYS)
    Observable<String> getChatMessageDraft(Observable<String> messages, DynamicKeyGroup key, EvictDynamicKeyGroup evict);

    ...
}

As you can see, it refers to the method matchesCacheInfo() in the stack trace, but the call itself has nothing to do with that method.

I've investigated it a fair bit, but still have no clue about what is happening.
Also, it's not a consistent thing, it happens occasionally, not every time.

When setting @LifeCache with 0 value the provider data needs to be evicted, but now it doesn't, because it is interpreted by RxCache as a provider without LifeCache; and therefore, no matter how many time is elapsed, its data never will be evicted.

Now can set cache life by Anotation @LifeCache, but how to set cache life in statement?

I'm

Title says it. It would be great to have support for singles (as all my api calls are Singles).

Lets say I have 2 endpoints:
TrainingSummary and Trainings

Once I add a new training in the backend I want to evict the TrainingSummary for the day the training was uploaded. But I don't want to fetch new data instantly. Is that possible?

Hi,

So I am using this framework in an app that requires me to flush the cached memory data when a user logs out.

Is there an easy way of doing this?

The only way I have found so far is that I use the annotation Actionable and explicitly evictAll. But this poses a problem since the Actionable annotation can only be applied to results of List<>, leaving my other non-List<> data unable to be evicted.

I.e, you've got smth from GET /api/smth/1 and then successfuly made PUT /api/smth/1 or DELETE /api/smth/1. Is it possible to force invalidate cache from GET method?

Hi Victor,

First let me say, thank you for sharing this library. I'm still just exploring, but so far it looks really cool.

I downloaded your sample project for android and configured it in Android Studio. Just a quick note that I had to change the double-quotes for calling your dependency in sample_date.build.gradle to single quotes or it wouldn't compile:

compile ('com.github.VictorAlbertos.RxCache:android:0.4.9') { exclude module: 'guava' }

Not a big deal at all, maybe that works fine in some environements...it just wouldn't work in mine.

The big issue I'm having right now is that the call for this dependency returns an APK instead of an AAR so I'm getting this error:

Warning:Dependency com.github.VictorAlbertos.RxCache:android:0.4.9 on project sample_android resolves to an APK archive which is not supported as a compilation dependency. File: /Users/foo/.gradle/caches/modules-2/files-2.1/com.github.VictorAlbertos.RxCache/android/0.4.9/a29a405c4d3c542571e9f062b7158916fecb2219/android-0.4.9.apk

Thanks,
Amanda

We need to create a simple mechanism for handle migrations between different deploy versions.

When I call to a provider and I retrieve an object from the cache (it can be a simple object, a list, ...) and I modify it, for instance, remove an item from the list, modify the object's properties, then the next time when I call to that provider, the provided object from the cache is the same object I've modified in my app.

So, I think the right behaviour should be that the providers return copy to objects, instead of return reference to objects. This way RxCache will be able to preserve immutability and stay sync with the persistence layer.

If a retrofit api call returns with an empty list, is it possible to store the empty list in the cache and have RxCache load this empty list instead of making more api calls? Currently, for a given api call, the cache keeps making this call until it has data to store

I use @ Actionable, RxProviders provide the method is this:
@actionable
Observable<List> getChildRecommend(Observable<List> childEntity,
DynamicKey filterPage, EvictDynamicKey evictFilter);

Retrofit method is this:
@FormUrlEncoded
@post("/")
Observable<StatusEntity<List>> getChildRecommend(@fieldmap HashMap<String, String> params);

I call this：
CacheProvidersActionable.getChildRecommend(cacheProviders, new DynamicKey(makeKey(params)))
.addFirst(childEntity)
.toObservable()

Result The service does not see the server returning data?
I use the wrong way?