it barely had any features then, this version is full of new functionality: streaming logs, long running processes, code interpreter and lots of other things and full docs site as well
Great read. How does it handle out-of-order events or scenarios where events need to be processed in strict order across different partitions? Are there built-in mechanisms to enforce global ordering when needed?
If events need to be processed in strict order across different partitions, then you need to send these events to a single key (and thereby to a single partition). A partition consists of multiple service keys for which Restate ensures strict order processing. It is noteworthy that different keys don't block each other from being executed (no head of line blocking across different keys).
The order in which events are processed for each key is their arrival order. If you need to handle out-of-order events, then you can implement this as part of a virtual object which can store events and re-order them based on other events that carry some form of watermark or based on time.
Nice question! Restate is not a log that retains the raw events for a long time - conceptually just until they where processed by the handlers, DB, locking, etc.
When you build stateful handlers, the state per key is in the internal DB, and that get's you a similar effect to log compaction, i.e., retain one value per key.
I have a “summarise” log entry: the current log’s contents that will be relevant to the future are summarised. For example, if it’s FY2023’s financial transactions, we compute the final balances at the end of the year. We then close the log, and write an entry to it of “no more log entries after this are valid”.
We then copy the summary transactions to a new log, and compress and archive the old log.
You can identify high throughput and low throughput types of log entries and segregate them into different log streams. For example, the “new customer/change customer info” stream probably gets way less traffic than the “customer has logged in” stream. The former is also harder to summarise. Put the hard to summarise but low volume stuff in its own log.
The cloud setup was super fast! I used it for an existing app + restate TS sdk, really took a few steps to get things up and running! Looking forward to more support for nextjs/node
We are still sorting out which SDK to implement next. Your preference for Go and Ruby is noted. The best way to stay up to date with the next developments is the GitHub issues at the moment. Creating a public product roadmap is a good idea.
I think it's important to understand that this was his opinion in 2021. Things have changed since, and hopefully, all these solutions are available now.
