Check out Apache arrow[0], it chunks data into independent categories (e.g. date) and then indexes it contents for quick paging. It works well with the tidyverse, but not perfect, which is why I was curious as to how you guys were doing it.

1: https://arrow.apache.org/docs/r/articles/data_wrangling.html