Support manual pagination keys about scanamo HOT 6 CLOSED

this was a long time ago so I'm not sure if the API has changed since then, but you'd use it the same way as you'd use query.

If you modify the function to accept table as a param, it would look like:

val operations = for {
  _ <- table.putAll(...)
  result <- queryPaginated(table, 'something -> 123, limit, Some(startKey))
} yield result

ScanamoAsync.exec(client)(operations)

or for nicer syntax you could define an implicit class that wraps table

implicit class TableOps(val table: Table[_]) extends AnyVal {
  def queryPaginated[T: DynamoFormat](...) = // same function as earlier
}

then you could do table.queryPaginated('something -> 123, limit, Some(startKey))

from scanamo.

mostly for client-side pagination, where the front-end client only needs to display few items at a time, and a way to resume from where they left off (in my case, since the client can't pass in a java.util.Map[String, AttributeValue], I'd encode/decode it as a String)

in my own code, I ended up just doing something like:

case class DynamoPage[T](
  items: Seq[T],
  lastEvaluatedKey: java.util.Map[String, AttributeValue]
)

val table: Table[Something] = ???

def queryPaginated[T: DynamoFormat](
  query: Query[_],
  limit: Option[Int],
  startKey: Option[java.util.Map[String, AttributeValue]]
): ScanamoOps[DynamoPage[T]] = {
  val req = new QueryRequest().withTableName(table.name)

  limit.foreach(req.setLimit(_))
  startKey.foreach(key ⇒ req.setExclusiveStartKey(key))

  ScanamoOps.query(query(req)).map { qr: QueryResult ⇒
    val items = qr.getItems.asScala.map(ScanamoFree.read[T])
    val key = Option(qr.getLastEvaluatedKey)

    DynamoPage(items, key)
  }
}

that's a simplified version to get the gist of it, though the actual code I have is more complicated:

• has a typeclass PageKeyFormat to convert between j.u.Map[String, AttributeValue] and a type K (Key)
• the lastEvaluatedKey in DynamoPage is actually an Option[K]
• I define a PageKeyFormat to convert between the java Map to a compressed JSON string, which is the string key that the clients pass in to specify their lastEvaluatedKey

from scanamo.

I believe this is a very common use case where a query returns many records and clients need a way to paginate through it. It would be nice if we can pass the paging params (or last evaluated key) to dynamo and have dynamo only return results specified by limit param.

from scanamo.

I did consider this when implementing limit support, but was struggling to think of a good use case for this. Could you describe what you'd want it for in more detail?

Would it be enough to be able to provide the next batch of results for a given batch of results?

from scanamo.

@williamho could you be able to provide some extra details on how to use your example outside of scanamo? Do you define your own interpreter?

from scanamo.

You can now use table.query0 or table.scan0 to access the raw dynamodb result, get the last evaluated key, and feed that into table.from in the next loop iteration.

from scanamo.