Let me make my point clearly: Google depends on the Linux kernel stack as much as the top-of-the-thread comment suggests that they did, and the things they're doing in user-mode, they would also be doing in user-mode on FreeBSD.
That's all I'm here to say.
As these things go, in the course of making your argument, you made a falsifiable and, I believe, flatly incorrect claim:
Most of the larger tech companies these days are using userland networking and bypass the kernel almost completely for networking
At least in Google's case, this isn't true. People doing custom network stack stuff totally do bypass the kernel stack (sometimes with usermode stacks, and sometimes with eBPF, and sometimes with offload). But the way you phrased this, you implied pretty directly that networking writ large was usermode at Google, and while I entertained the possibility that this might be true, when I investigated, it wasn't (unsurprisingly, given how annoying user mode networking is to interact directly with, as opposed to in middlebox applications).
Ok, this is a very hostile and could not possibly be considered a charitable interpretation of what I said, in fact I'd say it borders on trying to pick an argument where there isn't one. I did not "flat out lie" and I detest the insinuation. I expect better of you honestly.
To answer: "Most of the larger tech companies these days are using userland networking and bypass the kernel almost completely for networking"
You could read "almost completely" as in "almost across the whole company" which would be a weird way to read it, but you seem to have read it this way.
I intended it to mean: when they bypass the kernel; it is a near complete bypass.
Since there actually is still a network connection going through the kernel (the host itself will still be connected of course), which is of course the inverse of what you seem to have taken away; in that user-mode networking is used even less than entirely even on a single node.
edit: In fact, I stated multiple times in my post: "Google is mostly happy to just throw hardware at this" which you seemed to just.. ignore? Google are absolutely happy to throw hardware at issues until they can't anymore or the gains are too enormous to avoid. I thought I was extremely clear about that.
Your point about user-land networking in FreeBSD is just a nonsense one to make -- and not the point we were discussing anyway, like suggesting "if it did rain beer, would we all get drunk?" it's completely hypothetical and not based in any subjective reality or objective truth. You have absolutely no way of knowing if FreeBSD could do those things, the statement that the architecture permits is is shown somewhat in FreeBSD's use in Netflix, which has been commented elsewhere in this thread to achieve close to 0.8TiB of data transfer, but knowing what google would have done with freebsd would require seeing into alternative realities.
I don't know where you work, but as far as I know: nobody has managed to perfect that technology yet.
This is a weird cursed thread that started out with a pretty silly† claim about FreeBSD vs. Linux network stack performance (neither of us started it; it's a standard platform war argument). Someone made a comment that the hyperscalers all depend on the Linux kernel stack, to a far greater degree than they do on FreeBSD. That's a true statement; at this point, you and I have both agreed on it.
When that point was pressed earlier, you and another comment brought up kernel bypass (usermode networking, specifically) as a way of dismissing hyperscaler Linux dependencies. But that's not really a valid argument. Hyperscalers do kernel bypass stuff! Of course they do! But they're doing it for the things that you'd formerly have bought dedicated network equipment for, and in no case are they doing it in a situation where deploying FreeBSD and using the FreeBSD stack would be a valid alternative.
The disconnect between us is that I'm still talking about the subject that the thread was originally about --- whether Google using the Linux stack is a valid point backing up its fitness for purpose. I think it pretty clearly is a valid point. I think the usermode networking stuff is interesting, but is a sideshow.
† "Silly" because these kinds of claims never, ever get resolved, and just bring out each side's cheering section, not because I have low opinions of FreeBSD's kernel stack --- I came up on that stack and find it easier to follow than Linux's, though I rather doubt the claim that it has a decisive performance advantage.
Thank you for clarifying, and I apologise as I also became quite hostile.
I agree that we agree on many points; but I think where we diverge (and perhaps fundamentally) is in the base assumption that "because google uses it, it must be the best, because even if it wasn't google would make it so" (at least, this is my interpretation of GPs comment).
I have little doubt that the low hanging... mid hanging and perhaps even most of the high-hanging fruit has been well and truly plucked when it comes to linux throughput at the behest of the large tech companies; because a few percent improvement translates a lot at their scale. -- However I am reminded of an allegory given in a talk (that I can't find) regarding bowling.
In the talk the speaker mentions how they got "really good" at bowling with completely the wrong technique; but it worked for them, up to a point in which they could not improve no matter what they did. They had to go back to the basics and learn proper technique and become much worse before they were able to overtake their previous scores with the bad technique.... but after that point there were further improvements to be had.
My argument that this is the case is merely: doing an architectural re-write of the linux kernel to be more scalable in the way FreeBSD's is would be very punishing for too many people, and additionally that the economics are not favourable when, if you do get to a point where you cannot scale due to the kernel, you could just break out into userland. -- then simply suffer the adequate but not insane performance everywhere else where it's not needed anyway.
So, to summarise my points:
* Because a big company uses something does not mean it is perfect in all areas
* That Linux has a lot of attention on it does not mean necessarily that it has the most potential: though I don't doubt that the majority of it's potential has been reached.
* Diminishing returns means once it's "good enough" people will try to get performance elsewhere if they need it.
* Rewriting the network stack in Linux completely would likely be harmful to many and subtly so, I haven't seen people moving towards this idea either, this feels like it could be political as well as technical.
* Hyperscalers will often trade convenience over performance: regardless, CPU time is much cheaper for them than it is for us.
I think it's totally legit to say that hyperscaler Linux adoption isn't dispositive of the Linux's stack's performance advantage over FreeBSD. I basically think of this FreeBSD vs. Linux stack debate as unknowable (it probably isn't, but it's liberating to decide I'm not going to resolve it to anyone's satisfaction). So I'm not here to say "Google uses Linux ergo it's faster than FreeBSD"; that adoption is a useful observation, but that's all it is.
That's all I'm here to say.
As these things go, in the course of making your argument, you made a falsifiable and, I believe, flatly incorrect claim:
Most of the larger tech companies these days are using userland networking and bypass the kernel almost completely for networking
At least in Google's case, this isn't true. People doing custom network stack stuff totally do bypass the kernel stack (sometimes with usermode stacks, and sometimes with eBPF, and sometimes with offload). But the way you phrased this, you implied pretty directly that networking writ large was usermode at Google, and while I entertained the possibility that this might be true, when I investigated, it wasn't (unsurprisingly, given how annoying user mode networking is to interact directly with, as opposed to in middlebox applications).