Subscript crash when count changed about pagedarray HOT 3 CLOSED

I've also been thinking about this issue but have been wary of implementing a solution like guarding and returning nil because of data consistency concerns. In my own project, when fetching paginated data from a backend and the total count changes, I prefer to throw away old data and create a new array because you no longer know the old indexes match the updated.

What are your thoughts? In what situations are you running into this problem?

from pagedarray.

I got this error when server response had count field different than before what can happen quite often I think, depending how much queried data set changes by processing for example requests of other clients.
So in this case I've tried to change count. And that error happened.
I haven't thought about your solution to make new PagedArray. It seems quite good choice.

Let's think about the case when server data set is changing relatively often, for example various clients are adding new records to set that is matching our query:

(a) if changes occur very often, we would have to discard PagedArray content at every page fetched to make this consistent with server data.

(b) if changes occur very often, it can happen that count is the same as before but N records were deleted and N records were added

(c) changes occur very often - but new data is appearing at the end of our paged list (because it's sorted by ID, or creation date etc)

So some thoughts:

(a) resetting full array degrades performance a bit, prefetching doesn't make sense probably too and prefetched page can reset our currently visible page, in case (c) resetting array may be not necessary at all

(b) we cannot detect change by looking at count

(c) I would prefer to change count only and grow PagedArray at the end - exactly the same as data in api endpoint does

I think overall drawback of page number pagination is that it can be inconsistent with server. It's not the same as cursor pagination or other solutions. I think that depending on above (a), (b), (c) cases developers may want different solutions specific to their needs.

My current fix returns nil. I am not sure what would I choose for final solution: to return nil, make count immutable or to allow growing.

from pagedarray.

I agree with you that cursor pagination is a better approach to use for when data changes rapidly and making sure the client is absolutely consistent with the server is crucial.

The drawback is that you cannot fetch something in the middle of a list when scrolling fast and thus have to fetch all pages leading up to the one the user is currently looking at.

Ultimately, this is a performance/UX vs data integrity tradeoff. Depending on your application, you favor the one over the other. PagedArray is aimed at solving the case where data integrity isn't the absolute highest priority. Originally it was created for an app where content isn't user-driven and it's more important to get content fast than making sure users get exactly what's on the server. Although you can certainly mitigate some data integrity risks with the techniques above, it's not 100% fool proof without another way for the server to signal that data has changed.

from pagedarray.