Most of this advice has already clicked with me, but this part:
> If you can't easily explain why something is difficult, then it's incidental complexity, which is probably worth addressing
That's a real eye-opener. Hope I remember this next time I implement something complex.
The thing is, I don't think this stuff would've helped me much when I was a junior dev. A lot of them are just too nuanced.
'Addressing' can be ambiguous. Writing comments to explain why it's right and needs to be this way? Or changing the code without considering if they have enough knowledge of the rest of the codebase.
There's also a chance it's not understood yet to easily explain it.
A key part of joining a team is remaining open to the fact that just because I don't understand something, doesn't mean there isn't understanding in it.
Sometimes there's understanding, sometimes the person who did understand has left the team. This is why it's worth writing good commit messages and going back to read them when there's a funny piece of code. A lot of my commit messages are to the effect of "I did it this way to be backwards compatible with how it always worked. It still looks funny to me though"
Then at least future person knows I don't have a good reason for it and they can dig deeper or get PMs to sign off on something better if they want.
Reading code is harder than writing it, and too many people don't have the patience for it.
Instead, in cases like these they'd rather invest the time re-learning the lessons of the past because they thought it should be a certain way, and if they can't instantly see a reason why it's done a certain way, then they behave (unintentionally or otherwise) as if understanding is not possible.
Especially where it's tricky it's worth including a larger pre-amble in the code itself, as well as source code control.
Often one or the other are suffering, and it's sometimes worth giving a description of how the components work together in detail, or are relied up on by more than one use case (therefore not a simple change or solution).
Nobody could easily explain why early (1950s - 1970s) fusion reactors were so difficult. It must have been that everyone was stupid and creating their own problems.
No one can explain why the Collatz Conjecture is so hard to prove. It must be because everyone is so stupid and is adding incidental complexity to the problem. Why is everyone so stupid?
Or, maybe, possibly, that advice is utter bullshit nonsense and some problems are just hard for reasons that we don't fully understand. Go write a reliable reactive library that works how people expect. Oh, you can't? Better explain exactly why it's so hard then, or it's obviously all your fault.
That advice is so utterly at odds with actual reality that it makes it clear this author should be completely ignored.
A sign of a good educator is to - while it may take a lot of time and a lot of building up of base concepts - enlighten people through plain language the complexities of their topic.
Oftentimes, research papers and similar are written for an audience that is already deeply ingrained in their representative topic, such that those first principles aren't discussed anymore, so perhaps that's where you're coming from when you speak about how "no one can explain ..."; but I would argue that many people can, and have. It may just take a lot of speaking, paper, drawings and similar.
And even if / when there are obscenely difficult things to prove, the chances that the thing the proverbial "you" is working on is one of the incredibly difficult things is very unlikely.
There's a saying that a sign of actually understanding something is being able to describe it to a child. It's also why Thing Explainer and ELI5 exists.
"Good educators" are not solving difficult problems, they're teaching about long-standing solutions to long-understood problems. Long-standing solutions to long-understood problems in software engineering are called "packages" or "libraries" and it usually takes a single command to include them. We're talking about building software, not teaching.
The "explain it to a child" comes after decades of humanity's understanding something. Feynman might have been able to explain quantum mechanics to a child, but quantum mechanics was roughly 90 years old at the time. Nobody would have said Michelson and Morley were introducing "incidental complexity" into their experiment because they couldn't explain why the diffraction pattern looked so weird. What are the odds that they're working on an actual difficult problem!? They must be stupid.
You ignored my comment about fusion power and didn't give any examples of someone actually explaining why the Collatz Conjecture is hard. Nobody knows. The first person who deeply understands why the Collatz Conjecture is so hard is going to be the one who solves it. That's what life is like when you're near the edge of humanity's understanding.
> the chances that the thing the proverbial "you" is working on is one of the incredibly difficult things is very unlikely.
On the contrary. Being on the cutting edge of software engineering is incredibly easy. The discipline is (arguably) about 100 years old. The space of problems is unfathomably vast. Software engineering is applicable to literally any other discipline: pick any one and you should immediately find throngs of unsolved (hard) problems around how to apply software approaches to that problem.
And when anyone in the industry solves a specific problem, that specific problem is largely solved forever, for everyone, for free. So if you're not working on a difficult problem, what are you doing? Copy/pasting what someone else did?
Fusion: Turbulence, plus Naiver-Stokes was known to be hard, plus adding in EM fields, plus the exothermic state change of the nuclei fusing introduces discontinuous jumps in the free energy.
Collatz: if you replace 1 and 3 with arbitrary parameters, Conway showed that this class of problems is undecidable, so any particular instance could be arbitrarily hard.
Genuinely great responses. I'm not sure "it can be generalized to an undecidable problem" really explains why it's hard though, as the 5n + 1 problem is very easy (to prove false) and generalizes just the same. And all good points about Fusion, but my point was that it took some time (months, at the very least) for them to realize that's what was spoiling their machines. Remember that this is all a counterexample to the statement that "If you can't easily explain why something is difficult, then it's incidental complexity" -- meaning all the difficulty you are encountering is your fault (and therefore the actual problem must be easy, and you must be stupid.) That's literally what TFA says.
> If you can't easily explain why something is difficult, then it's incidental complexity, which is probably worth addressing
That's a real eye-opener. Hope I remember this next time I implement something complex. The thing is, I don't think this stuff would've helped me much when I was a junior dev. A lot of them are just too nuanced.