They used a flawed benchmarking methodology.

JITs only optimize hot code which means any benchmark without a warmup is going to measure cold + hot code time.

Further, JITs will optimize away unused results. They aren't using the leftpad results in any meaningful way.

What they are likely measuring is how long it takes for the JIT to optimize the benchmarking framework.

watch: https://vimeo.com/78900556 for how to microbenchmark a JITed language.

Yeah benchmarks can be much more difficult to get right than it might seem at first glance. Good thing they didn't try to write their own benchmarking code, otherwise they might have fallen into those traps you just mentioned.

Luckily, they didn't, and instead pulled in the the `benchmark` library as a development dependency[1]. The author of said library works on V8, and already considered all those problems and much, much, more[2].

[1]: https://github.com/left-pad/left-pad/blob/master/perf/perf.j...

[2]: https://mathiasbynens.be/notes/javascript-benchmarking

I know, I read the benchmark suite code. The benchmark suite itself isn't doing the things I said it isn't doing.

Sure, It's popular. It's also wrong.

Okay, you win. I'm not going to read the entire source of that package just to make a point. Though I do find it strange that the author of that library would write an entire blog post on the topic and then not take his own advice in the implementation of the library he wrote.