Reflections on Uber, brogrammers, and the effectiveness of working class programmers

The toxic “brogrammer” culture has been in the news again, initially with the blog posting of engineer Susan J. Fowler’s year-long experience with sexual harassment at Uber, the reaction of Uber’s CEO [1] Travis Kalanick who was shocked, shocked to discover he is surrounded by utter assholes, then new video proof that, shock, shock, Travis Kalanick himself is an utter asshole (as well as exhibiting a charmingly naive lack of awareness of the capabilities of dash-cams), followed by still more revelations of nefarious deeds [2], reaching a point where I wouldn’t be surprised to find the Trump administration simply deciding to hire Uber’s entire management team to fill their thousands of empty appointments. Oh, and did I mention that despite all this, Uber is losing fantastic sums of money?  Hey, you go guys!

But while Uber is perhaps unusually bad given the visibility and nominal value of the company, it is hardly unique, and in fact the bad-boy programmer is a long-running cultural meme: see for example the 1993 movie Jurassic Park where you get a guaranteed applause line when the “programmer”—subtly named “Dennis Nedry” and played as Jabba-the-Hut minus table manners—becomes Dilophosaurus chow. Two books I read last summer on the current programming start-up culture— Disrupted: My Misadventure in the Start-Up Bubble by Dan Lyons and Chaos Monkeys: Obscene Fortune and Random Failure in Silicon Valley by Antonio Garcia Martinez—are essentially book-length renditions of the same story, and the racist, misogynistic culture of Silicon Valley generally is a story dating back two or three decades. [3][22]

So, what gives here? These stories are so pervasive that those outside the real world of programming probably are asking why there is even grist for a blog entry here. But at least some of those of us on the inside find it puzzling: how could people with the personality disorders it apparently takes to get ahead in these companies ever write decent code? Fundamentally, programming requires intense concentration for extended periods, a high level of attention to unforgiving detail, and a willingness to set aside your ego when trying to improve a product. Not, shall we say, exactly consistent with the nonstop Animal House bacchanalia that characterizes these companies. How are these people possibly writing decent code?

The answer, I would submit: they aren’t. And the intensity of their craziness and exclusivity is simply a smokescreen for that fact.

Or to be a bit more nuanced, what the brogrammers are riding on—and their success will almost certainly be temporary—is that fact that the contemporary programming environment allows an individual, or a relatively small group of individuals, to do absolutely extraordinary things with relatively little effort. Hence a team of brogrammers with only average skill can ship a reasonably workable product more or less on time while spending, max, perhaps 20 hours a week doing something with code while not hung-over or otherwise incapacitated [4], and spending the rest of the time on office politics, talking sports, violating every known workplace discrimination law, and ingesting every imaginable intoxicant. All the while claiming, of course, to be working 80 hour weeks. The sheer aggressiveness with which they work to exclude people from outside the brogrammer culture is born of the necessity to keep this fact from becoming common knowledge.

But, no, no, it can’t be!: our entire computer infrastructure, the very fact VCs are throwing billions  of dollars at us, is dependent on hiring arrogant misogynistic assholes and party animals! They must be tolerated, lest we be reduced to a state where our iPhones are useful only for killing rats for food! Nooooo…

Well, bunko, let me explain: it ain’t like that any more. Mind you, it probably never was like that—and I do hope that Hidden Figures, both the book [5] and the movie, begin to correct the historical misconceptions—but it certainly isn’t that way now, because getting computers to do incredibly impressive things is really easy now. [6]

This, in turn, is primarily due to two innovations which only came into play in the past decade (conveniently just beyond the learning horizon of a substantial number of people who are investing money in brogrammer teams): open-source toolkits, and web-based crowd-sourced documentation [20], in particular a site called Stack Overflow. In a nutshell, the toolkits mean that almost any general problem you need to solve, unless it is really recent—typically meaning weeks—will already have been coded, and that code is available for free. If you have any problems getting that code to work, and it’s been around more than six months or so, the solutions for those issues are on the web as well. [7] You basically just fill in the gaps relevant to your specific application, and you’re done. 

Really, it’s that simple. As William Gibson puts it “The future is already here; it is just unevenly distributed.”

To explore this a bit further, consider the recent story about a Nigerian programmer being stopped at the US border by the Stasi…errr, ICE…and “asked to write a function to balance a binary tree.” [8][19] As some wag on Twitter noted, the correct response to that question is “I’d look it up on Stack Overflow.” [10.] A slightly better question would have been “Tell me the circumstances under which you would need to balance a binary tree” but even then the correct answer would be “You first tell me a situation where I could possibly justify the time involved in setting up a binary tree rather than doing equivalent tasks with existing data structures in Python.” By that point, of course, anyone giving those responses would be blindfolded, in cuffs and in the cargo hold heading back to Lagos but still, those are the correct answers. Assuming that a working programmer needs to be able to balance a binary tree is like refusing to hire a house painter unless they know how to make their own brushes and mix their own pigments. 

With decided speed [16] the world of programming has evolved into one where people with decent training but generally average skills can do absolutely extraordinary things in very short amounts of time. This is the world of self-documenting open-source software, increasingly running in effectively universal hardware environments, which is the consequence of a series of relatively recent developments about which I will have an absolutely fantastic blog entry if I can ever get around to finishing it. [11] The necessity of retaining the arrogant irreplaceable toxic genius asshole and the swarms of party animal brogrammers indulged with a “work” environment characterized by substantial investments in toys and endless keggers was a gamble with the odds stacked against it in the best of times, and is completely unnecessary now.

Which brings us to my final point, the possibility of the emergence of the working class programmer. Working class in two respects. First, recognition that programming projects can be quite competently done by a suitably trained, responsible and, well, ordinary individuals. The phrase “programmer” as a generic job description was probably last relevant in the early 1970s, before the emergence of time-sharing, or at best in the early 1980s before the emergence of graphic user interfaces.[21] After those points in time, the field fragmented—in a perfectly normal and organic fashion—into ever-increasing sub-specialties which require different sets of skills [12]  but, like all professional specializations, can be mastered to a level where one can produce very competent code after about 500 to 1000 hours of training and practical experience. The notion that only a tiny number of people can achieve this is the much maligned “talent myth,” [13] which leads clueless managers and VCs to seek out “the best people”—who just happen to always be white, male, and great companions, particularly while downing vodka shots at strip clubs [14]—rather than putting together a calm, predictable, and competent team with the appropriate skill sets.

These jobs are also, perhaps a bit anachronistically, “working class” in the sense that the salaries required to get these people are the sort associated, with the usual adjustments of inflation, with those of workers with specialized skills in the industrial age (with health insurance, nowadays without a pension or union), which means in the $60K to $100K range, and working 40 hour weeks or something reasonably approximating those. But also with the expectation that one will follow contemporary professional standards. There are plenty of such people around.

[Shameless self-promotion alert!!] The other thing you might at least consider is getting a few people with a bit—maybe even more than a bit?—of experience, because about 90% to 100% of the stuff you need to do on any project will be routine, not the stuff that can (in theory though rarely practice) only be accomplished by arrogant irreplaceable toxic genius assholes. And when it comes to “routine”, or even some things that aren’t routine, experience makes things easy. In particular, take this quote from 2017 Superbowl champion quarterback Tom Brady 

“I have the answers to the test now,” Brady said. “You can’t surprise me on defense. I’ve seen it all. I’ve processed 261 games, I’ve played them all. It’s an incredibly hard sport, but because the processes are right and are in place, for anyone with experience in their job, it’s not as hard as it used to be.”

Now move that into the programming realm: you’ve got a brilliant new idea that you want to pitch to—or may already have been funded by—DARPA or IARPA. But I know the same thing has been tried five times over the past forty years, and based on the current state of the technology, I know which parts look easy but aren’t, which parts can in fact be efficiently solved now but couldn’t be earlier, and I’ve got a pretty good idea where things are going to take much longer than you think. I know which of the data sets you think exist in fact don’t, and the details of twenty more data sets which would actually be more useful but which you’ve never heard of. I can’t do this with every project, just as I’m guessing Tom Brady’s tennis game isn’t particularly exceptional, but in my field, I can do it with a lot, and in other fields, there are plenty of other people just like me, for whom “it’s not as hard as it used to be.”

Just sayin…

So where do you go from here? If your shop is festering with brogrammers, follow the advice of Nellie Forbush in South Pacific and “show’em what the door is for.” Though nine times out of ten—well, unless you are Uber [17]—just enforcing the rules in the employee handbook will be sufficient. Based on what I’ve seen, these brogrammers with no discernible talents beyond office politics,  partying, and harassing women and minorities would fit in pretty well as roofers—hanging dry-wall takes too much attention to detail—and will enjoy the camaraderie of following hailstorms across the Great Plains. [15]

But, but, I can’t do that! I have to lose money! Lots of money! In order to be successful and attract vast sums of investment capital, I have to make absolutely sure my start-up is horribly inefficient, misses deadlines and ships crappy code! If I don’t do that, the VCs won’t take my company seriously, and pretty soon I’ll be hanging drywall! You…just…don’t…understand!!!

Calm down, calm down…surely we can work this out. Yes, there are vast pots of dumb money floating around, and only so many good ideas, though if you knew how to assemble a competent team for programming, not just tail-gating, more good ideas than you might think. So, how about this:  just put a bunch of Syrian refugees on your payroll, tell them just to take the money and spend their time getting their lives back together and watch while their kids rapidly learn English to a level of fluency superior to that of the average Middlebury undergraduate, and tell the VCs the Syrians have brought you secret algorithms stolen from the Russians and smuggled out of Aleppo in the final days of the siege. You know, Rogue One. Key thing is they’ll be burning money, which the VCs want to see, and unlike the brogrammers, the refugees won’t be getting in the way of people actually adding value to your product. 

This, bunko, is a pathway to being hailed as a management genius and pretty soon, a TedX talk!

Footnotes

1. Chief enabling offender

2. That’s not to say Uber doesn’t have an eventually viable business model, just as sending around a bunch large guys carrying lead pipes to threaten to break the kneecaps of owners of pizza parlors is a viable business model. [18]  But recognize it for what it is, and stop pretending that it encompasses some sort of fantastic breakthrough in software and systems design: all Uber has done is implement a pretty obvious and easily duplicated idea with a reasonable level of competence. Albeit by most accounts, the sorts of operations that dispatch guys with lead pipes to engage the owners of pizza parlor owners turn a profit. Which Uber, as noted above, does not.

3. Though within the community, the Bastard [Systems] Operator from Hell—BOFH—was a long-standing meme dating to the early 1990s . Though unlike Uber managers, the BOFH was not presumed to be disproportionately harassing women, or at least that wasn’t part of the memes I was hearing—it probably does exist somewhere in the genre.

4. From an outsider’s perspective, one of the absolutely weirdest management fads (more like a cult) over the past couple of decades is “pair programming,” where you hire two programmers to do the work of one. It has its very strong advocates (and detractors)—just Google it—and doubtlessly worked in some instances. But note that this is the unscrupulous manager’s dream: “Wow, I get to hire all of my otherwise unemployable party-boy friends, and pair each one with a nerd who will not only do all of the work, but who I’ll allow them to terrorize with impunity, and things will be golden forever!” Can’t imagine that this model hasn’t been deployed in more than one situation.

5: Shout-out to Charlottesville author Margo Lee Shetterly!

6. What is programming? In my mind, if what you are doing uses logical loops and branches, you’re programming. Ada Lovelace wrote the first loop: that’s when programming started. If this statement makes no sense to you, you aren’t programming. Data structures also matter; the rest is pretty much just appropriately using an ever-changing collection of libraries and knowing how to efficiently debug code. Mostly the last.

7. We’re still on the leading edge on this, but almost all serious production applications will now be configured to run in the cloud, thus standardizing hardware and to some degree, operating systems. REST interfaces to Docker containers also seem to be emerging—at least for now—as a standard for hiding diversity in the underlying software.

8. The Stasi agent allegedly said “You don’t look like a programmer.” Which is rather making my point isn’t it?: “programmers” may be white, South Asian or Chinese, but never African. To say nothing of African-American. Which is to say, in terms of racial integration we’ve actually gone backwards from the opening scene in the movie Hidden Figures, where an archetypical redneck cop in tidewater Virginia could still be persuaded that African-American women were helping us beat the Russians. [9]

9. To say nothing of the anachronism that a [now] presumably Republican southern cop would view the Russian government with suspicion, rather than embracing them as BFF. My, how things do change…

10. For those unacquainted with Stack Overflow, it provides the equivalent of the fact that one can enter the phrase “which dinosaur killed programmer in Jurassic Park” into Google and instantly get a thorough answer, bypassing the inconvenience of watching the movie.

11. Or more precisely, editing it, as I’ve written 16 pages, and these entries should be about half that. 10 pages already written on the identity/economics crisis in the Democratic Party, 35 pages on the future of Western democracy…someday…

12. Generally not including balancing binary trees.

13. “Talent myth” links: originally the concept was Malcolm Gladwell’s in 2002; here’s a 2015 update from Huffington post http://www.huffingtonpost.com/amol-sarva/talent-is-a-myth_b_6793870.html and here’s a fairly influential version specifically directed at the programming community https://lwn.net/Articles/641779/ But really, any task whose effectiveness comes from summing a series of sub-tasks for which skill levels are randomly distributed, according to any distribution, will end up with the composite effectiveness having a Gaussian (bell-shaped) distribution: that the law! (specifically the Central Limit Theorem). Whereas it is nearly impossible to come up with a data generating process that would produce the U-shaped curves assumed by the talent myth.

14. The vast amounts of money funding keggers and tolerance of behaviors that keep HR’s lawyers awake at night are proof positive that high-income tax rates are too low, not too high. Another reliable rule-of-thumb: If you are spending more than a quarter of your time dealing with personnel conflicts or office politics, you are over-staffed.

15. Of course, where they will actually end up is in finance, not roofing—again, unless under-secretary positions are still available in the Trump administration—though that apparently doesn’t provide quite the number of jobs for the well-connected party-boy incompetents as it once did. Though it is unsurprising that VCs and hedge funds find this model attractive, as it is essentially the same as their own model: put up with large amounts of stuff that will fail (as no less than Warren Buffet has pointed out, hedge funds are actually a terrible deal) in the hope that one or two will pay off big time. It is true that a few arrogant genius asshole programmers actually might, in the right set of circumstances, add value to a project, and it is just possible that maybe one of them will make your company a unicorn. But the chances are low that you’ve actually managed to hire such an individual, whereas the chances are quite good that the ones you actually did hire will create such a sufficiently dysfunctional environment that your company will fail, or yet least not do as well as it would otherwise.

16. Doing my background research, I was astonished to see that Stack Overflow dates only to 2008: it feels as though it has been around forever.

17. Or Kay and Jared Jewelers

18. As it happens—I’m in meetings where this sort of thing is discussed seriously in the context of political instability—extortion, rather than dealing drugs, is actually the most straightforward way for most gangs to make money. They also don’t make nearly as much money as you probably imagine. Then again, neither does Uber.

19. Presumably the Stasi/ICE guy saw this question somewhere on LinkedIn.

20. I’m pretty sure this seemingly magical process of self-documentation accounts for the almost cult-like infatuation seen in the tech community for a forthcoming “singularity” where networked machines effectively become conscious. But no, it’s just people trying to be helpful in exchange for a bit of recognition, and that is a very fundamentally human thing—in fact quite possible one of the single most important things that makes us human—not a machine thing.

21. Not long after the Macintosh went on the market, I wrote a commercially successful ancillary for what became a best-selling political science textbook. “Commercial” in the sense that it was for a major textbook publisher, and I was paid reasonably well for my efforts. “Successful” in the sense that the textbook sold well—not necessarily for reasons related to my program—and we won a couple awards for the program, and the publisher continued it into multiple editions, eventually taking the work in-house. Writing for the Mac was a real eye-opener: less than ten years earlier, Kernigan and Ritchie had defined not only the C language, but major elements of structured programming, in only 200 pages. The books showing how to program the Mac ran to four—eventually five—thick volumes, with “every one assuming you had already mastered the others.” It was a completely new world.

22. I also should note that this rant is not directed at any people or projects I’ve worked with directly: it is motivated by the very consistent set of stories coming through in the autobiographical accounts of others. I’ve certainly worked on teams that had programmers who were handsomely paid while contributing absolutely nothing to the project—and am embarrassed to say that on at least a couple of occasions (none recent!!) I’ve been that person—but they’ve been pleasant about it. I’ve also run into plenty of the arrogant irreplaceable toxic genius asshole programmer types in academic settings and come to think of it, on one occasion made the very serious mistake of letting one into a project I was directing, but in my government and commercial work have successfully avoided—or perhaps more accurately, my various project managers have created teams that have avoided—encountering them. Then again, I’ve generally worked on projects that have produced pretty decent code that does what it is supposed to do, rather than merely blowing through billions of dollars of dumb money. Funny, that.