go-gorm / caches Goto Github PK

only works with fetch rows

Good example:

var list []Item
db.find(&list)

Bad example:

var count int64
db.count(&count)

The reason count doesn't work might be related to this:
https://github.com/go-gorm/gorm/blob/a7f01bd1b22ec7131c420de62abe5f7e85573277/finisher_api.go#L490-L495

Does it support rows cache?

There're mainly 5 kinds of operations in gorm (gorm function names in brackets):

Query (First/Take/Last/Find/FindInBatches/FirstOrInit/FirstOrCreate/Count/Pluck)
Create (Create/CreateInBatches/Save)
Delete (Delete)
Update (Update/Updates/UpdateColumn/UpdateColumns/Save)
Row (Row/Rows/Scan)

The document you expected this should be explained

What operations / methods uses gorm/caches

Expected answer

A list of methods that are cached and some workaround/documentation for methods that aren't by default.

Your Question

How can I reset my cache from the memory example by specified operations like create, update and delete?

The document you expected this should be explained

Expected answer

Hello,

I found an other error with the identifier for the cache with pointer query args.

My Example of query args to build the identifier:

you can see the string is a pointer variable and the problem is that the pointer value will be changed on same request again like this:

and in the next request with the same query args (values), the string pointer variable value shows like this:

Before I can try out the invalidate function, this needs to be fixed.

You can find it here:

https://github.com/go-gorm/caches/blob/master/identifier.go#L22

Solution like this maybe:

https://go.dev/play/p/LFtb1FWfWQn

Originally posted by @dennis-dko in #10 (comment)

Your Question

I wonder what logic is used to determine the cache supported by this component? For example, I do a paging query, and the table is constantly inserted data, this component will not appear inconsistent data?

The document you expected this should be explained

Expected answer

Describe the feature

Hi there,
There are three methods of Cacher interface: Get/Store/Invalidate, that is great. Can you impl a new method "Delete" for it, I think "Delete" is important and the Cacher interface would be completed.

Thank you.

Motivation

In some scenarios, the data in Cacher would be delete.

Related Issues

None

Describe the feature

Hi @ktsivkov
I queried somethings from DB via gorm-caches, found that the prefix is "gorm-caches" and the keys is a long SQL script, sometimes the long keys have lower performance. Can we customize the prefix and keys? Thank you.

Hi @xpume

I noticed that there is a new config "AllowGlobalUpdate=true" in the latest examples, I test it but found no difference with old version. May I know what's the purpose of AllowGlobalUpdate? and when can we use it? Thank you.

Describe the feature

The Cacher interface would have an additional first argument of a context.Context obtained from the DB Statement.

Motivation

• It would be idiomatic to pass a context to a blocking cache adapter such as Redis.
• I wish to explicitly mark queries as being cacheable. For this I am adding a value to the context object; queries without this value will be uncached. I appreciate this is a controversial pattern but it is the simplest way to achieve this without support within Gorm itself. This is perhaps a separate issue, but having the context argument at least provides a workaround.

I appreciate this is a breaking change. For now I have forked this library to add this argument to the Cacher. See https://github.com/mgdigital/gorm-cache/pull/1/files

Related Issues

How to implement custom stores, for example redis?

I've tried to use the package with redis but seems that I can't decode/encode the values properly:

panic: reflect: call of reflect.Value.Elem on map Value, it's occurs because I'm Mashal/Unmarshal values, but idk how can I safely save it on redis. Looks like here I have src as map not as a proper way to reflect back to dst

package cache

import (
	"encoding/json"
	"time"

	"github.com/go-gorm/caches/v2"
	"github.com/gofiber/fiber/v2"
	"github.com/gofiber/storage/redis"
	"gorm.io/gorm"

	"core/env"
)

type customCacher struct {
	storage fiber.Storage
}

func newCustomCacher(config env.Config) caches.Cacher {
	storage := redis.New(redis.Config{
		URL:      config.RedisURL,
		Database: 3,
	})

	return &customCacher{
		storage: storage,
	}

}

func (c customCacher) Get(key string) *caches.Query {
	val, err := c.storage.Get(key)
	if val == nil || err != nil {
		return nil
	}

	var query caches.Query
	if err = json.Unmarshal(val, &query); err != nil {
		return nil
	}

	return &query
}

func (c customCacher) Store(key string, val *caches.Query) error {

	b, err := json.Marshal(val)
	if err != nil {
		return err
	}

	return c.storage.Set(key, b, 6*time.Hour)
}

func NewGormCacherPlugin(config env.Config) gorm.Plugin {

	return &caches.Caches{
		Conf: &caches.Config{
			Easer:  true,
			Cacher: newCustomCacher(config),
		},
	}
}

The document you expected this should be explained

Custom cacher, with different stores than memory.

Expected answer

A consise way to store in another place the informations

Hi there

@mgdigital From issue 9, we know that the client can decide whether to use Redis cache. This is a great idea, just want to know how to use it?

If there are two models, model A wants to use Redis cache, model B do not. Both of them use the same Caches interface, how to use Context to impl this? Can you kindly give an example?

Thank you.

Your Question

When using Gorm Many2Many, the data from the Preload association query is stored in the Reids cache, and the json deserialized data is empty, but after my debugging, it is actually there.

1.This is my database model

type SysUser struct {
	ormx.GORM_MODEL
	Account    string     `gorm:"column:account"`
	Password   string     `gorm:"column:password"`
	AesKey     string     `gorm:"aes_key"`
	NickName   string     `gorm:"column:nick_name"`
	Birthday   utils.Time `gorm:"column:birthday"`
	Sex        *int       `gorm:"column:sex"`
	Avatar     string     `gorm:"column:avatar"`
	Email      string     `gorm:"column:email"`
	Mobile     string     `gorm:"column:mobile"`
	Sign       string     `gorm:"column:sign"`
	RegIp      string     `gorm:"column:reg_ip"`
	Status     *int       `gorm:"column:status"`
	CreateUser uint64     `gorm:"column:create_user"`
	UpdateUser uint64     `gorm:"column:update_user"`
	RoleList   []SysRole  `gorm:"many2many:sys_user_role;joinForeignKey:UserId;joinReferences:RoleId;->"`
}

2.Using the Redis cache as described in the README, I just added console output debugging in the Store method

func (c *redisCacher) Store(ctx context.Context, key string, val *caches.Query[any]) error {
	res, err := val.Marshal()
	if err != nil {
		return err
	}

	fmt.Printf(`
SET-KEY: %s
SET-VALUE:
%s
`, key, string(res))

	c.rdb.Set(ctx, key, res, time.Duration(c.Timeout)*time.Second) // Set proper cache time
	return nil
}

3.Concrete logic code

	userInfo := &model.SysUser{}
	db := l.svcCtx.Db.Debug()

	if err := db.Preload("RoleList").First(userInfo, in.Id).Error; err != nil {
		return nil, errorx.NewFailError(err)
	}

        // For comparison, print the results to the console
	st, _ := json.Marshal(db.Preload("RoleList").First(userInfo, in.Id).Statement.Dest)
	fmt.Printf(`
Query Result: %s
`, string(st))

4.Execution result

• Console output of the Store method with Redis caching enabled

SET-KEY: gorm-caches::SELECT * FROM `sys_user_role` WHERE `sys_user_role`.`user_id` = ?-[55]
SET-VALUE:
{"Dest":[{"SysUserRoleList":{"id":0,"create_time":"","update_time":"","Account":"","Password":"","AesKey":"","NickName":"","Birthday":"","Sex":null,"Avatar":"","Email":"","Mobile":"","Sign":"","RegIp":"","Status":null,"CreateUser":0,"UpdateUser":0,"RoleList":null}},{"SysUserRoleList":{"id":0,"create_time":"","update_time":"","Account":"","Password":"","AesKey":"","NickName":"","Birthday":"","Sex":null,"Avatar":"","Email":"","Mobile":"","Sign":"","RegIp":"","Status":null,"CreateUser":0,"UpdateUser":0,"RoleList":null}}],"RowsAffected":2}

It is clear that the data structures are inconsistent and blank.

• Console output of a usage scenario with Redis caching enabled

Query Result: {"id":55,"create_time":"2022-12-04 14:08:04","update_time":"2023-12-26 22:56:57","Account":"admin","Password":"O5qt3VF0xInprv4h87fAQHjLUPKGuCW7I+jyBca25LQsnyP6NHkq1qR6c+r3chCm","AesKey":"1b214e547672df85","NickName":"admin","Birthday":"","Sex":1,"Avatar":"","Email":"[email protected]","Mobile":"","Sign":"","RegIp":"","Status":1,"Creatser":0,"UpdateUser":55,"RoleList":[]}

As you can see, RoleList is an empty array

• Console output of a usage scenario with Redis caching not enabled

Query Result: {"id":55,"create_time":"2022-12-04 14:08:04","update_time":"2023-12-26 22:56:57","Account":"admin","Password":"O5qt3VF0xInprv4h87fAQHjLUPKGuCW7I+jyBca25LQsnyP6NHkq1qR6c+r3chCm","AesKey":"1b214e547672df85","NickName":"admin","Birthday":"","Sex":1,"Avatar":"","Email":"[email protected]","Mobile":"","Sign":"","RegIp":"","Status":1,"Creatser":0,"UpdateUser":55,"RoleList":[{"id":26,"create_time":"2022-12-22 17:35:16","update_time":"2023-12-29 17:07:35","RoleCode":"7b9edb82-7fcd-4bac-b9f9-3f75c8f6e4d5988","RoleName":"admin","Status":1,"Remark":"admin","CreateUser":5ateUser":55},{"id":28,"create_time":"2023-01-08 22:35:45","update_time":"2023-12-20 19:32:31","RoleCode":"7b9edb82-7fcd-4bac-b9f9-3f75c8f6e4d99","RoleName":"SuperAdmin","Status":1,"Remark":"SuperAdmin","CreateUser":55,"UpdateUser":55}]}

As you can see, the RoleList array has two pieces of data

The document you expected this should be explained

Expected answer

The data for the Many2Many association schema should be parsed correctly and stored in the Redis cache without affecting normal queries