When I look at my management chain at Google, it's engineers all the way up. My manager was a consultant for 20 years working on DARPA AI projects. My director wrote much of Google's critical early search infrastructure. My VP invented Google's ranking algorithm. My SVP used to work on chip design at Digital's Western Research Lab. My CEO invented PageRank.

Things don't change unless people make them change. Where you see dysfunctional management, I see clueless lumbering behemoths that are tasty prey for a startup.

To me that's the difference between "developer, technologist, engineer" & "entrepreneur". The former will simply be happy if the conditions are suitable. The later wants to build their own company to create the conditions they want. In effect be their own boss.

There was a period of time (roughly 2004-2007) where Google hired a bunch of managers without engineering backgrounds, but they discontinued that practice a long time ago because they found it doesn't work so hot. I've heard that many of the worst offenders have since attritioned away.

For a team with mature, responsible and professional developers, it shouldn't be such a big deal to have a non-technical manager anyway. Most management responsibilities have nothing to do with technical matters. A manager should serve the developers, not order them around. A good manager, technical background or not, will be glad to get out of the way of a good team.

I myself am a manager with a technical background. I consider myself no more than mediocre as a manager. There are a lot of non-technical managers out there who could probably do a way better job managing , but they would have to be told by the programmers what they need from them.

But most programmers don't speak up. Most programmers don't say no. Most programmers don't explain why interruptions are so costly, let alone outright refuse to let that ruin their work. Most programmers have never read pg's essay about maker time and manager time.

A professional chef will walk out if his boss told him to burn every steak to a crisp, but most programmers will grumble a bit, get on with it and then (but only if they really care) bitch about it online. This despite the fact that chefs have way more to lose given the job market.

With most programmers being so unprofessional, assigning a grade school teacher doesn't even sound like such a far fetched idea...

It's not exclusively a senior management problem. Senior management doesn't know there is a problem unless they happen to have a technical background themselves, which is an unrealistic expectation, given that software is everywhere these days.

It is first and foremost our problem. We're the ones that know what is wrong and how to fix it. We are the professionals this business relies on.

Only assigning managers with a technical background isn't going to work anyway, because there are not enough of them to go round.

As a programmer, you have to be a bit of masochist to put the peter principle to the test on yourself while your technical knowledge is still relevant, and people are willing to pay you good money just to code.

Exactly. That's what good management does: insulate the programmers from as much nonsense that is possible. A good manager has some grasp of the field so that he can block at least the most outrageous feature requests from the devs or at least delay asking about them until a weekly meeting or something. If the devs have problem, the manager will drive the issues through the cogs and clogs of the organization and bureaucracy instead of using valuable developer time to track those issues.

An anecdote: I have had travelling arranged by a competent office manager and a company-wide travel reservation and approval system. Guess which saved so much time from everyone, sending an email to the office manager "I need to fly to X next Tuesday, my boss on Cc: with this" and afterwards dropping a stack of receipts back to her for filing reimbursements, or a dozen people trying to figure out what's the best way through the travel system without sacrificing too much work time on that and still ending up scanning individual receipts for uploading to the system.

A good manager has to have a minimum of understanding and/or a maximum of trust in some of his subordinates. Far too many in our field have either.

Wow!

FWIW, I am one who is genetically incapable of keeping his mouth shut. If something's stupid or sucks, I'll say so. If people disagree with me and have arguments to back up their disagreements I'll listen. I have zero tolerance for responses along the lines of "because I said so, just do it."

In my experience, a lot of programmers are like this as well.

There came a time when the CEO of the company where gradeschool teachers were the management was replaced... by someone who also knew nothing about development. She recognized all the problems with the product (which was being mismanaged) and I told her that the problem was management, how we'd tried to have reasonable practices, including repeated requests for a plan that we could stick to, and that the grade school teachers were in over their head. I recommend that they be put into a position to have input on the educational aspects of the product, but that engineering should be run by someone who understood engineering. She told me that there would be no management changes, and she wanted me-- given that I was the most senior engineer-- to give her a plan for fixing things. (Seemingly oblivious to the fact that I'd just given her the solution.)

I told her I'd been working on a plan and that I'd give it to her in 30 minutes. Went back to my office, wrote a resignation letter, named the fact that management made product progress impossible as the reason, and handed it to her. She said "That was fast" and I said, it's not what you're expecting, and walked out of the office, never to return. (I had been on the bubble already, and would have resigned earlier if the events that led to her becoming the new CEO hadn't occurred.)

I cannot think of anything more compelling than explaining why things were broken, making suggestions for how to fix it, those being rejected out of hand, and then emphasizing the point with a resignation. I was committed enough that I gave up my job, without a replacement lined up. I don't know what more could have possibly underlined the seriousness of my position to her.

Do you think she learned anything by this? No management changes were made, but apparently I did have some impact: The gradeschool teachers got even more hysterical and neurotic after I left, starting to accuse employees of paranoid conspiracies, etc. More and more people started resigning. The company failed not long after this.

It is my considered impression that most people who are clueless are not interested in thinking about things-- when something unusual happens, like my resignation-- they just assume that its a reflection on me, rather than the situation. Everyone working on the product knew the problem, there wasn't debate among us, but management was too ignorant to even understand that the problem was management.

The grade school teachers saw everything thru that lens-- they thought "their kids" were screwing around or wasting time, or wanting to do thing their own way rather than the way they needed to be done, and so they believed it was just irresponsibility that was causing the product problems. Meanwhile we were getting whiplash as they changed their opinions and positions and demands for what the product would do on a weekly basis.

One thing I've noticed in bad managers that is pretty consistent-- they tend to change their mind about what the product should do, and then think that their subordinates are slacking when the product doesn't do the new thing. Even thought they may have wasted 6 months under a prior plan, they don't account for that-- all those six months are credited towards the new plan, and why wasn't it done already if you had 6 months?

In the case of the grade school teachers, this product was a magical wonderland where anything they could dream up could be created-- and any time their thoughts changed the product could change without any cost at all, cause to them it was totally plastic.

So anytime someone said that these changes would take time and have impact on other things, they thought the person was just being lazy and didn't want to make the changes.

I remember once, there was a rumor that a VC firm associated with Bill Gates might be investing, and so my boss came in and told me we had to replace all of our linux based infrastructure (file server, email server, web server, etc) with Windows. "If bill gate finds out we're using linux we won't get funded!" I told him it was silly as things were working fine... so he brought in a "windows consultant" who told us we needed to buy 4 new top of the line servers to replace our one spare developer PC that was doing all the work currently, cause "You can't run a web server and and FTP server on the same machine!" Naturally the grade school teacher believed his over priced consultant- who was telling him what he wanted to hear- over his engineer.

How was he to know any different? He's a grade school teacher. (He now has a significant position at Microsoft, which I think is hilarious....)

You've only told her management is the problem, that "engineering should be run by someone who understood engineering", not that the engineers can run the engineering part themselves.

"We need better managers" is just as non-constructive as "we need more programmers to make the deadline". You cannot open a can of competent managers just like you cannot open a can of competent programmers and everything will be alright.

You want to convince management you can do a better job? Show them. Sure, it means sticking your neck out. It means you may fail. Hell, it means you may get fired for even trying, instead of being allowed the dignity of quitting yourself.

But that is commitment. That is putting your ass on the line because you believe in what you stand for. You want good engineering practices? Start applying them. You want a realistic plan? Make one. (One caveat: incumbent managers may turn out not be your main impediment. Some coworkers who will gladly join you in bitching sessions will run for cover if you actually try to change things from the inside. You know, the guys that stayed when you quit and continued to take it up the ass without protest.)

It's only when clueless management insists on telling you how to do your job and prevents you from doing it right is when you run out of options.

Don't get me wrong, I'm not saying that is what you personally should have done. Most of the mismanagement you're describing is just pure mismanagement, nothing to do with being technical or not.

But just doing it is the best chance of actually changing things, rather than to wait for a that rare manager-with-technical-background to come and rescue you.

Or of course start your own company. In which case, unless you prefer to go solo, you may at some point find yourself dealing with the same issue from the other end.

Sure, he could have tried to lead an internal revolution or whatever, but the reality is that after he hurt the wrong person's feelings on the third day he would have been fired anyway. He could have spent a few days laying out a very formal reiteration of everything he had just told the new CEO, but after she discarded the executive summary so flippantly, what would have been the point?

In fact, this is part of the reason I think engineering managers should understand engineering. Its not that one who doesn't can't be a good manager, its just that by understanding engineering they understand the consequences of the decisions they make. If a manager simply trusted his subordinates, then knowing engineering might not be necessary.

But the bad managers generally don't. They think that engineering explanations that go over their head are BSing because the engineer is lazy or doesn't want to do the work, or is padding the estimate or whatever. The tend to project onto engineers all the fault, and blame them for "not speaking up" (when in fact they did speak up but were ignored) or whatever rationalization is convenient at the time. They project this worldview onto actions to explain them, when in reality, what they're trying to "explain" has been already explained to them-- only in engineering terms.

I think bad managers also often have a superiority complex where they see a class divide between "management" and "workers". And they think that they job is to keep workers working, as if we're orcs in Warcraft who have to be beaten with a stick or we'll lie down and take a snooze.

Now I'm on the other side of the table. I am the manager. I've had challenges with employees, and they are always interesting and different.... but none of them are lazy, and none of them were making shit up to get out of work.

You may have had a Very Bad Manager. There may have been nothing you could have done about it, other than walk away. I totally accept that.

But you write: they (managers) see a class divide between "management" and "workers"

When what I see in your words is you (the programmer at the time) seeing (and reinforcing through your actions) a class divide between management and workers. You blame the bad managers for a behavioral pattern that you were just a as much a part of. At least that's what I read in your words.

And that's what I see in many other programmers. They act like children with zero responsibility, and then complain about being treated as such by evil pointy haired bosses.

It's a two way street. And given that we, the engineers, are the ones who know how it should be done, we have a considerable responsibility in changing it.

I think you should cut her a little slack and take into account her job. You're concentrating on her rational evaluation of your arguments, but that skips right past the hard part of her job. She was still trying to figure out if she could trust your characterization of the facts. Really, if you were her, would you completely reorganize management on the say-so of a single individual before getting a more complete picture?

And if you decided to completely reorganize management, would you tell people about it the instant you made your decision, or would you keep it under your hat until you figured out 1) how to present the decision to the board (or whoever she answered to), 2) how to manage the transition so a bunch of people with institutional knowledge didn't disappear before she could replace them, and 3) how to avoid getting undermined by the people she was plotting against, who may have had significant credibility with the board and with management in other parts of the company?

And if you decided to essentially demote some people by sidelining them in the org chart, would you give one of their subordinates a heads-up before you had a chance to tell them about it?

Finally, it's reasonable to ask you to come up with a solution that differs from the optimal one. You do it in engineering all the time, and management isn't any different. You would say, "Licenses for this software account for 30% of the project budget. Suppose the license fees doubled -- can we design a solution that doesn't use this software?" She might have been thinking, "Gosh, I hope I can get some real development managers in here in three months, but what if it takes a year? I better have a plan for getting products out in the meantime."

http://en.wikipedia.org/wiki/Fundamental_attribution_error

A good manager can be good without knowing about programming. Right now I'm on a mixed team that is managed by a guy who has no background in programming. He trusts his people and delegates a lot of borderline managerial tasks, such as prioritizing technical work and evaluating the success of projects, to senior programmers. Apparently it's a viable way to manage programmers, because it works well for us. The manager lets technical folks make the technical decisions and backs his tech leads in cases of disagreement. It would all fall apart if he stopped trusting the judgment of his technical leads, but when management doesn't trust the judgment of their senior people, the situation is probably FUBAR anyway. Ditto if the senior technical folks behave unprofessionally -- it's a FUBAR situation until they act like adults or you get rid of them.

Domain knowledge can help a constructive manager become a more valuable asset for his team, but any competent manager can at least learn to get out of his team's way and provide support in small ways. If a manager is an obstruction to getting work done, then he's a bad manager, and domain understanding won't turn him around. It might even make him more effective at being an obstruction.

The problem is only compounded by the fact that everyone can have a computer while not everyone can design and assemble a nuclear reactor or highway bridge. Therefore, people easily overestimate their understanding and abilities in being a dev manager. Add to this the fact that software is paying much better than many other fields including sciences, academia, and engineering and everyone wants in.

The fallout is hellish work environments as programmers have to contend with these people (who bring over their outdated interpretation of the management role from their old careers), and a disengaged workforce looking for the exit doors. Long term this will only drive software costs up.

Because most programmers don't want to manage. The few who do and are good at it are out of most companies' price range. Remember, as Random Educational Software Shop, you're competing in the same market as Google and Facebook. 99% of the good developers are going to go there instead, because they get more money and better work. You can't just wish people will work for you; you have to actively recruit them and make it worth their while.

The specific problem with programming is it's pretty much invisible - the end result, when well done, hides the enormous levels of complexity it takes to create it. It's just 'another website' or whatever.

That's compounded by the problem that, in the U.S. at least, managers seem to think their job is having influence/power over people, and telling them/making do things. In fact, management is supposed to be a support role. Managers should be creating an environment that enable their employees to do their job as best as possible.

In my experience -many- managers exist to please the person above them, and think very little (or, at best, simply are unable to see) how their actions effect their employees happiness and productivity.

Their team lead is generally the most experienced/capable person on the team.

The programmer would explain the goal of the new project, like so: "We need to design a module that talks to the what-what the customer is using. It has to do three things: it must take an order from the what-what, then perform the appropriate action, and finally receive information back from the central system."

"So, we begin with the first part of the module: the part that will take an order from the what-what. It needs to listen for a message from the unit the customer is interacting with. Then, it needs a rule concerning what to do with each order, that is, what instructions to follow."

Once the basic form and actions of each section are established, then he or she can explain in more detail the actual working parts: "Now we need to step through the list of orders. With each order, we will check our rule set and perform the appropriate action. Once there are no more orders, we will tell the computer (or program) we are finished and it can proceed to the next step, which is receiving information back from the central system."

Throughout, the programmer only needs to explain conceptually the work he is doing: Even the difficult or intricate concepts (details of network connections, perhaps) he may illustrate. The details of the actual programming language do not need to be explained for this purpose. (That is, he does not need to say, "Now we write 'Blob userOrders = new (Blob)' to create a new blob", or "We will receive the orders from the customer into an array ("What's that?") and step through them with a 'for' loop.") He or she might use a pen and paper to draw a diagram of the module as they design it.

Once the manager has the overview of the system and knows how it is supposed to work, perhaps they can follow the writing of tests for it: "We will send it some simple test orders to see if it follows them correctly....And now we will give it an illogical order--did we check for a bad order?"

Finally, the manager or supervisor can join the programmer as he or she goes back to the code to identify the source of the trouble. (More details could be added here.)

The programmer does not need to oversimplify, but as they go deeper into the project together, the programmer can explain more technical details. Previously, these details would have been distracting, overwhelming, or incomprehensible to the the person they are teaching, but now they enable them and are necessary to see what the program is doing at this level, and why the problem occurred as it did.

Could this be a realistic scenario, in certain cases at least?

No, the supervisor's primary motivation is to avoid blame while gaining power. If they know how everything works, there is no deniability when their plan fails. The game they play is to promise the impossible to their bosses in hopes of promotion and blame the 90% that didn't go to their plan on the programmers while trying to take full credit for the 10% that was pulled off by heroic coding. To this end, no supervisor will be caught in a 'code' conversation. If they understood software engineering they would be on the hook for the 90% as well.

Where an honest supervisor needs to learn the fundamentals of software engineering is at the school and then later as a software engineer. You can get a BS in software engineering in many places and go on to obtain a Masters in technology management. Also work 5-10 years in deliberate practice as a software engineer. There is no 'sit down for an hour and learn software engineering' just like there is no 'sit down for an hour and learn how a nuclear reactor works'. What you are talking about is remedial training, but none of these guys would go for it. Their philosophy is that people willing to do technical work are simple fools to be exploited. Usually they are very glib about the idea that they skipped all the hard technical stuff.

P=Programmger M=Manager

Bad one: P.Here is the estimate. M.I want it done faster. P.Okay cut things. M.No. P.Well, it will still be done according to the estimate, as that's math. We have to change what we're doing to shorten the estimate, we can't just arbitrarily shorten it. M. I expect it done in 70% of the time on the estimate. All Done. Or Else.

Okay one: P. Here is the estimate. M. I want it done faster. P.Okay cut things. M.Take out so and so and do so and so less well.

Good one: P.Here is the estimate. M.What are things we can do to shorten this? P. Okay, we can de-risk items 1, 2 and 17 by limiting the requirements to X, and we can save a ton of time if we drop the requirement we can display right to left languages as well. M. Let me run the first thing by the client. M. The second won't fly, but even 10% is good.

I've worked with some EXCELLENT ones as well as some so so and some poor ones. The real trick is: Remember, their boss probably ALSO thinks they're not that good if they pull crap on their reports. Push back, and do well because of it.

Yes, programmers can fix this to some degree (the last chapter of the art of software estimation is great for teaching this), lots of these changes are something management has to do.