> It's definitely beyond my technical abilities to help
I think this goes for most people that would like to help. This is very mature code and ripping out any portion and replacing it without a massive amount of groundwork isn't going to fly, Erlang is used for semi-realtime mission critical stuff all over the globe. That's 'tread very carefully' ground.
One can help on a technical level without providing fully baked solutions.
Finding (or developing) real or at least reasonable uses of Erlang where performance is lacking in an unreasonable way.
Narrowing those use cases into a minimal test case.
(Much harder!) Finding a way to make such a test case work significantly faster, even if it breaks other things, can be a helpful step towards a polished improvement.
Some areas of OTP are very well polished and difficult to improve. Some areas are kind of the first thing that worked and it was good enough and never got much polish. There's less of the second kind than there was before, but when you hit those, it's often pretty easy to improve, and a bug report or proposed patch will likely be processed by the upstream team quickly.
I find Erlang amazing, but it's important to note that a lot of OTP is written in Erlang and usually easy to read. Most of the C code is fairly easy to read too, once you get used to it. It's a very approachable ecosystem, and I think people should be encouraged to peek under the hood, and tinker.
Here's a free idea for performance improvement, although it may not be good. Erlang's JIT is very simple, because it needs to run for every beam file at load time. So this idea comes in two parts. a) Could you get a minor reduction in beam loading overhead by caching the native code output of the JIT processing. b) Are there ways to improve the JIT output that are too slow to run just-in-time, but could be run to improve the cached output and loaded in the future. Of course, there's the elephant in the room of making sure cached native code is compatible with the current BEAM, in case one JITed with a newer or older version. And the current JIT is successful by being simple, an earlier version of JIT tried to output optimized native code and was too complex and was abandoned. I think a motivated person with a good amount of experience with systems programming, mixed language experience, and willing to beat their head against the wall of dlopen or however the JITed code gets loaded could learn a lot working on this. And if it works or at least shows promise despite not completely working, share it and see where it goes.
An area impacted the most, is R&D into improving raw performance.
It's effectively just Lukas Larsson & Björn Gustavsson trying to solve this problem for the past 10-years or more.
Their work is hugely appreciated, yet they get little to no support on this front.
It's definitely beyond my technical abilities to help, but I would financially sponsor focus in this area.
Does anyone know how I can donate/sponsor more focus on improving raw performance?