Lead dev here. We've run a couple of benchmarks internally just for kicks. To create a fair comparison, you have to run the Hotspot JVM with the -Xint flag, which says interpret only. Right now our performance is anywhere from 15-25% of the speed of Hotspot with -Xint on small benchmarks. We figure that the use of go alone creates some important portion of that overhead when compared with the C++ of the Hotspot JVM. We're guessing that a well-optimized Jacobin interpreter will eventually get to 50-60% of the Hotspot's -Xint speed.
But we first want to get feature parity, before pivoting to performance. When we have feature parity, we'll run the Computer Language Benchmarks and post the results. That'll be fun to see!
Is that fair? The whole point of the tradeoffs made by HotSpot is that it's optimized for JITing.
Imho there is no way to do a fair comparison, as both implementations have completely different goals.
One thing that would be interesting is comparing the performance of some really simple command line program (e.g. `ls` or `cp` for small files) between Hotspot in interpreter mode, Jacobin and GraalVM native.
Do you (can you even have in go?) have a direct threaded interpreter? That alone may give a 2x performance difference last time I checked, or is that no longer the case?
I don't think you can even implement indirect threading in Go. Go supports neither computed goto nor mutually recursive tail-call optimization (only self-recursive).
Jump tables for switch statements was only implemented last year. If you squint that's close to indirect threading, but still with at least one unnecessary conditional per op.
For the curious, here's their giant switch: https://github.com/platypusguy/jacobin/blob/c508ec50f55ef381... In practice compilers have always been finicky when it comes to coaxing them to emit jump tables from switch statements, and I bet this is especially truly for Go.
Yeah, I think any bytecode interpreter ends up with a giant switch in the critical path at some point :)
Around the time that change was made to Go, Andrew and I were looking at this and wondering how big of a performance hit it was and if there were a better way to structure that. I had a hunch that the compiler should be smart enough to not compile that as a switch/giant if block, and a quick trip to a disassembler showed it using binary search. This commit: https://github.com/golang/go/commit/1ba96d8c0909eca59e28c048... added the jump table and has some nice analysis on where it makes sense to do binary search vs jump tables.
As far as I can tell, with certain restrictions (that are fine in this case), it is pretty reliable at optimizing giant if/else blocks and switches.
But we first want to get feature parity, before pivoting to performance. When we have feature parity, we'll run the Computer Language Benchmarks and post the results. That'll be fun to see!