Happy 60th Birthday, DARPA: you’re doomed

Today marks the mid-point of a massive self-congratulatory 60th anniversary celebration by DARPA [1]. So, DARPA, happy birthday! And many happy returns!! YEA!!!

That’s a joke, right? Why yes, how did you guess?

A 60th anniversary, of course, is very important landmark, but not in a good way: Chinese folklore says that neither fortune nor misfortune persist for more than three generations,[2] and the 14th century historian and political theorist Ibn Khaldun pegged three generations as the time it took a dynasty to go from triumph to decay. Calculating a human generation as 20 years and, gulp, that makes 60.

Vignette #1:

DARPA, perhaps aware of some of the issues I will be raising here, has embarked on some programs with “simplified” proposal processes (e.g. this https://www.darpa.mil/news-events/2018-07-20a). In DARPA-speak, “simplified” means a 20 to 30 page program description with at least 7 required file templates, the first being an obligatory PowerPoint™ slide. In industry-speak, this is referred to as “seven friggin’ PDF files and WTF a friggin’ required PowerPoint™ slide??—in 2018 who TF uses friggin’ PowerPoint™???” [3]

Vignette #2:

A few months back, I’d been alerted to an interesting DARPA DSO BAA under the aforementioned program, and concocted an approach involving another Charlottesville-based tech outfit (well, their CTO is in CVille: the company is 100% remote on technical work, across a number of countries) with access to vast amounts of relevant data. The CTO and I had lunch on a Friday—during which I learned the company had developed out of an earlier DARPA-funded project—and he was all ready to move ahead with this.

On Monday the project was dead, vetoed by their CFO: they have plenty of work to do already, and it is simply too expensive to work with DARPA as DARPA involves an entirely different set of contracting and collaboration norms than the rest of the industry. Sad.

Arlington, we have a problem.

But before we go any further, I already know what y’all are thinking: “Hey, Schrodt, so things have finally caught up with your obnoxious little feral strategy, eh? Left academia, no longer have access to an Office of Sponsored Research [5][6] so you can’t apply for DARPA funding any more. Nah, nah, nah! LOSER! LOSER!! LOSER!!!

Well, yeah, elements of that: per vignette #2, there are definitely DARPA [7] programs I’d like to be participating in, but no longer can, or rather, cannot assemble any conceivable rationale for attempting. Having sketched out this diatribe [8], I was on the verge of abandoning it as mere sour grapes when The Economist [1 September 2018] arrived with a cover story based on almost precisely the same complex social systems argument I’d already outlined for DARPA, albeit about Silicon Valley generally. So maybe I’m on to something. Thus we will continue.

As I was reminded at a recent workshop, DARPA was inspired by the scientific/engineering crisis of Sputnik. [9] DARPA’s challenge in the 21st century, however, is that it continues to presuppose the corporate laboratory structures of the Sputnik era, where business requirements and incentives were [almost] completely reversed from what they are today: the days of the technical supremacy of Bell Labs and Xerox PARC are gone, and they aren’t coming back. [10]

As The Economist points out in the context of the demise of Silicon Valley as an attractive geographical destination, Silicon Valley’s very technological advances—many originally funded by DARPA—have sown the seeds of its geographical destruction. DARPA faces bureaucratic rather than geographical challenges, but is essentially in the same situation at least in the world of artificial intelligence/machine learning/data science (AI/ML/DS) where DARPA appears to be desperately trying to play catch-up.

A few of the insurmountable social/economic changes DARPA is facing:

• AI/ML/DS innovations can be implemented almost instantly with essentially no capital investment.[11] As The Economist [25 August 2018] notes, in 1975 only 17% of the value of the S&P 500 companies was in intangibles; by 2015 this was 84%.
• The bifurcation/concentration of the economy, particularly in technical areas: the rate of start-ups has slowed, and those that exist quickly get snatched up by the monsters. Consider for example the evolution of the Borg-like SAIC/Leidos [12], which first gobbled up hundreds of once-independent defense consulting firms, then split, and now Leidos is getting purchased by Lockheed. You will be assimilated!
• As some recent well-publicized instances have demonstrated, working with DARPA—or the defense/intelligence community more generally—will be actively opposed by some not insignificant fraction of the all-too-mobile employees of the technology behemoths. Good luck changing that.

As I’ve documented in quite an assortment of posts in this blog—I’ve been successfully walking this particular walk for more than five years now—these changes have led an an accelerating rise, particularly in the AI/ML/DS field, of the independent remote contractor—either an individual or a small self-managing team—due to at least five factors

• Ubiquity of open source software which has zero monetary cost of entry and provides a standard platform across potential clients.
• Cloud computing resources which can be purchased and cast aside in a microsecond with no more infrastructure than a credit card.
• StackOverflow and GitHub putting the answers to almost any technical question a few keystrokes away: the relative advantage of having access to local and/or internal company expertise has diminished markedly.
• A variety of web-based collaborative environments such as free audio and video conferencing, shared document environments, and collaboration-oriented communication platforms such as Slack, DropBox and the like.
• Legitimation of the “gig economy” from both the demand and supply side: freelancers are vastly less expensive to hire and are now viewed as entrepreneurial trailblazers rather than as losers who can’t get real jobs. In fact, because of its autonomy, remote work is now considered highly desirable.

The upshot, as explosion.ai’s (spaCy, prodigy) Ines Montani explains in a recent EuroPython talk, small companies are now fully capable of doing what only massive companies could do a decade or so ago. Except, of course, dealing with seven friggin’ PDF files including a required friggin’ PowerPoint™ slide to even bid on a project with some indeterminate chance of being funded following a six to nine month delay. More shit sandwiches?: oh, so sorry, just pass the plate as I’ve already had my share.

As those who follow my blog are aware, I spend my days in a pleasant little office in a converted Victorian three blocks from the Charlottesville, Virginia pedestrian mall [13] in the foothills of the Blue Ridge, uninterrupted except by the occasional teleconference. I have nearly complete control of my tasks and my time, and as an introvert whose work requires a high level of concentration, this is heaven. My indirect costs are around 15%. In the five years I’ve supported myself in this fashion, my agreements with clients typically involve a few conversations, a one or two page SOW, and then we get to work.  

DARPA-compatible alternatives to this sort of remote work, of course, would involve transitioning to some open-office-plan hellhole beset with constant interruptions and “supervision” by clueless middle-managers who spend their days calling meetings and writing corporate mission statements because, well, that’s just what clueless middle managers are paid to do.[14] These work environments are horribly soul-sapping and inefficient—with indirect costs far exceeding mine—except for that rather sizable proportion of the employees who are in fact not adding any value to the enterprise but are enjoying an indefinitely extended adolescent experience where, with any luck at all, they can continue terrorizing the introverts who actually are writing quality code, just as they did in junior high school, which is pretty much what open-office-plan hellholes try to replicate. I digress.

So, I suppose, indeed I am irritated because there are opportunities out there I can’t even compete for without radically downgrading my situation, and even though I, and the contemporary independent contractor community more generally, could probably do these tasks at lower cost and higher quality than is being done by the corporate behemoths who will invarably end up with all that money, this despite the fact that a migration to remote teams with lower costs and higher output is precisely what we are seeing in the commercial sector. Says no less than The Economist.

Okay, okay, so the FAANG are leary about even talking to DARPA, and we’ve already established that the existing contractors aren’t giving DARPA what it is looking for [15], but you’ve still got academic computer science to fall back on, right? Right?

Uh, not really.

Once again, any reliance on academia has DARPA doing the time warp again and heading back to the glory days of the Sputnik crisis when, in fact, academic research was probably a pretty good deal. But now:

• Tuition—which will be covered directly or indirectly—at all research universities has soared as the public funding readily available in the 1950s has collapsed.
• Universities no longer have the newest and shiniest toys: those are in the private sector.
• The best and brightest students zip through their graduate programs in record time, with FOMO private sector opportunities nipping at their tails. The ones who stick around…well, you decide.
• The best and brightest professors have far more to gain from their startups and consultancies than from filling out seven friggin’ PDF files including one friggin’ required PowerPoint™ slide. Those with no such prospects, and the people building empires for the sake of empire building and/or aspirations to become deans, associate deans, assistant deans, deanlings or deanlets, yeah, you might get some of those. Congrats. Or something.

And these are impediments before we consider the highly dysfunctional publication incentives which have reduced academic computer science to only a single true challenge, the academic Turing Test—probably passed several years ago but the reality of this still hidden—for who will be the first to write a bot which can successfully generate unlimited publishable AI/ML/DS papers.[16] This and the fact that computer science graduate students tend to be like small birds, spending most of their time flitting around in pursuit of novelties in the form of software packages and frameworks with lifespans comparable to that of a dime-store goldfish. And all graduate students, on entering even the most lowly M.S.-level program, are sworn to a dark oath, enforced with the thoroughness of a Mafia pizza parlor protection racket, to never, ever, under any circumstances, comment, document or test their code. [21]

Academia will not save you. No one will save you.

HARUMPHHH!!! So if you bad-attitude remote contractors are so damn smart and so damn efficient, there’s an obvious market failure/arbitrage opportunity here which will self-correct because, as we all know, markets always work perfectly.

Well, maybe, but I’d suggest this is going to be tough, for at least three reasons.

The first issue, for the remote independent contractors as well as the FAANG, is simply “why bother?”: I’m not seeing a whole lot of press about AI/ML/DS unemployment, and if you can get work with a couple of phone calls and one-page SOW, why deal with seven friggin’ PDF forms and a friggin’ required PowerPoint™ slide?

Then there’s the unpleasant fact that anyone attempting to arbitrage the inefficiencies here is wandering into the arena with the likes of SAIC, Lockheed and BBN, massive established players more than happy to destroy you, and they consider seven friggin’ PDFs and all other barriers to entry a feature rather than a bug, as well as deploying legions of Armani-shod lobbyists to make damn sure things stay that way. But mostly, they’ll come after any threats to their lucrative niche faster than a rattlesnake chasing a pocket gopher. I suppose it could be done, but is not for the light of heart. Or bank account.

The final issue is that because DARPA [famously, and probably apocryphally] expects its projects to fail 80% of the time, there’s a frog-in-boiling-water aspect where DARPA won’t notice—until things are too late—structural problems which now cause projects to fail where they would have succeeded in the absence of those new conditions.  Well, until the Chinese get there first.[17]

There is, in the end, a [delightful?] irony here: one of the four foci within the DARPA Defense Sciences Office, those folks whose idea of “simple” is a 30 page BAA and seven friggin’ PDF files starting with a friggin’ obligatory PowerPoint™ slide, is called “complex social systems,” which in most definitions would include self-modifying systems.[18] And a second of those foci deals with “anticipating surprise.”

Well, buckeroos, you’ve got both of these phenomena going on right there in the sweet little River City of Arlington, VA: a complex self-modifying system that’s dropped a big surprise and in all likelihood there’s nothing you can do about.

Okay, maybe a tad too dramatic: at the most basic level, all that is going on here is a case of the well-understood phenomenon of disruptive innovation—please note my clever use of a link to that leftist-hippy-commy rag, the Harvard Business Review—where new technologies enable the development of an alternative to the established/entrenched order which is typically in the initial stages not in fact “better” than the prevailing technology, but attains a foothold by being faster, cheaper and/or easier to use, thus appealing to those who don’t actually need “the best.”

Project proposals provided by remote independent contractors with 15% IDC will—assuming they even try—be inferior to those of the entrenched contractors with 60% IDC, since in addition to employing legions of Armani-shod lobbyists they also employ platoons of PowerPoint™ artistes, echelons of document editors, managers overseeing countless layers of internal reviews, and probably the occasional partridge in a pear tree.[19] You want a great proposal?: wow can these folks ever produce a great proposal!

They just can’t deliver on the final product [20] for the reasons noted above. Leaving us in this situation

What they propose

What they deliver

 

 

 

 

 

In contrast, the coin of the realm for the independent shops is their open code on GitHub: even if the contracted work will be proprietary, you’ve got to have code out where people can look at it, and that’s why contemporary companies are comfortable hiring people they’ve never met in person—and may never meet—and who will be working eight time zones away: it’s the difference between hiring someone to remodel your kitchen based on the number of glossy architectural magazines they bring to a meeting versus hiring them based on other kitchens they’ve remodeled. All of which is to say that in the contemporary AI/ML/DS environment, assembling an effective team is more Ocean’s 11 or Rogue One, much less The Office.

So on a marginally optimistic note, I’ll modify my title slightly: DARPA, until you find a structure that rewards people for writing solid code, not PowerPoint™ slides, you’re doomed.

Happy 60th.

Footnotes

1. If you don’t know what DARPA is, stop right here as the cultural nuances permeating the remainder of this diatribe will make absolutely no sense. I’m also obnoxiously refraining from defining acronyms such as DSO, BAA, SOW, PM, FAR, FOMO, FAANG, CFO, CTO, ACLED, ICEWS, F1, AUC, MAE, and IARPA because refraining from defining acronyms is like so totally intrinsic to this world.

2. This is apparently an actual Chinese proverb, though it is typically rendered as “Wealth does not pass three generations” along with many variants on the same theme. There’s a nice exposition, including an appropriate reference to Ibn Khaldun, to be found, of all places, on this martial arts site.

3. A couple weeks ago the social media in CVille—ya gotta love this place—got into an extended tiff over whether the use of the word “fuck”—or more generally “FUCK!” or “FUCK!!” or “THAT’S TOTALLY FUCKING FUCKED, YOU FUCKING FUCKWIT!!!”—was or was not a form of cultural appropriation. Of course, it’s not entirely clear what “culture” is being appropriated, and thus offended, as the word has been in common use for centuries, but presumably something local as the latter phrase is pretty much representative of contemporary public discourse, such as it is, in our fair city.[4] Okay, not quite. Fuckwit.  So to avoid offense—not from the repetitive use of an obscenity, but the possibility of that indeterminate variant on cultural appropriation—I will continue to refer to the “seven friggin’ PDFs and one friggin’ required PowerPoint™ slide.”

4. Browsing the Politics and Prose bookstore in DC last weekend—this at the new DC Wharf, where the hoi polloi can gaze upon the yachts of the lobbyists for DARPA contractors—I noticed that if you would like to write a book, but really don’t have anything to say, adding FUCK to the title is a popular contemporary marketing approach. Unfortunately, these tomes—mind you, they are typically exceedingly short, so perhaps technically they are not really “tomes”—will probably all be pulped or repurposed as mulch, but should a few escape we can foresee archeologists of the future—probably sentient cockroaches—using telepathic powers to record in [cockroach-DARPA-funded] holographic crystals “Today our excavation reached the well-documented FUCK-titled-tome layer, reliably dated to 2015-2020 CE.” Though they will more likely have to be content with the “Keurig capsule layer”, which far less precisely locates accompanying artifacts only to 1995-2025 CE.

5. Or as Mike Ward eloquently puts it: OSP == Office for the Suppression of Research.

6. As Schrodt puts it, the OSP mascot is the tapeworm.

7. And IARPA: same set of issues, less money.

8. Though inspired in part after listening to some folks at the 2018 summer Society for Political Methodology meetings—unlike the three-quarters of political science Ph.D.s who will not find tenure track positions, political methodologists are eminently employable, albeit not necessarily in academia—literally laughing out loud—and this conference being in Provo, Utah, laughing out loud while stone-cold sober—about Dept of Defense attempts to recruit high-quality collaborators in the AI/ML/DS fields.

9. In this presentation, we were told “I’m sure no one here remembers Sputnik.” Dude, I not only remember Sputnik—vividly—I can even remember when the typical Republican thought the Russians were an insidious and unscrupulous enemy!

10. From the recent obituary of game theorist Martin Shubik

After earning his doctorate at Princeton, he worked as a consultant for General Electric and for IBM, whose thinking about research scientists he later described to The New York Times: “Well, these are like giant pandas in a zoo. You don’t really quite know what a giant panda is, but you sure as hell know (1) you paid a lot of money for it, and (2) other people want it; therefore it is valuable and therefore it’s got to be well fed.

11. Capital intensity is a key caveat here: as the price of the shiniest new toys increases, so does the competitiveness of DARPA compared to the commercial sector. So, for example, in areas such as quantum computing, nanotechnologies and most work on sensors, DARPA will do just fine. AI/ML/DS: not so much. So despite my dramatic title—hey, it’s a blog!—DARPA is probably not doomed in endeavors involving bashing metals or molecules. 

12. I wasn’t really sure how to find that Vanity Fair article on SAIC—which got quite the attention when it first came out more than a decade ago—but it popped right up when I entered the search term “SAIC is evil”. Also see this.

The sordid history of the likes of SAIC and Lockheed raises the topic/straw-man of whether DARPA PMs, in comparison to private sector research managers who can contract with a few phone calls and a short SOW, “must” be hemmed in by mountains of FARs and bureaucracy lest they be irresponsible with funds from the public purse. Yet these same managers routinely are expected—all but required thanks to the legions of Armani-shod lobbyists—to dole out billions to outfits like SAIC and Lockheed which have long—like really, really long—rap sheets on totally wasting public moneys. Sad.

13. Six coffee shops and counting.

14. Okay, your typical tech middle manager is also paid to knock back vodka shots in strip clubs while exchanging pointers on how to evade HR’s efforts to reduce sexual harassment, a phenomenon I have explored in greater detail here.

15. See Sharon Weinberger’s superbly researched history of DARPA, Imagineers of War, for further discussion, particularly her analysis of DARPA’s seemingly terminal intellectual and technical drift in the post-Cold-War period.

16: Academic computer science has basically run itself into a publications cul-de-sac—mind you, possibly quite deliberately, as said cul-de-sac guarantees their faculty can spend virtually all of their time working on their start-ups and consultancies—where publication has become defined solely by the ability to get some marginal increase in standardized metrics on standardized data sets.

Vignette: I’m generally avoiding reviewing journal articles now—I have only limited access to paywalled journals, and in any case don’t want to encourage that sort of thing—but a few weeks ago finally agreed to do so (for a proprietary journal I’d never heard of) after being incessantly harangued by an editor, presumably because I was one of about five people in the world who had worked in the past with all of the technologies used in the paper, and I decided to reward the effort that must have been involved to establish this connection. The domain, of course, was forecasting political conflict, and the authors had assembled a conflict time series from the usual suspects—ACLED, Cline Center, or ICEWS—and applied four different ML methods, which produced modestly decent results—as computer scientists, they felt no obligation, of course, to look at the existing literature which extends back a mere four or five decades—with a bit of variation in the usual metrics, probably F1, AUC and MAE. There was a serious discussion of these differences, discussions of the relative level of resources required for each estimator, blah blah blah. So far, a typical ML paper.

Until I got to a graphical display of the results. The conflict time series, of course, was a complex saw-toothed sequence. Every single one of the ML “models”: a constant! THE [fuckwit] IDIOTS HAD SUBMITTED A PAPER WHERE THE PREDICTED TIME SERIES HAD ZERO VARIANCE! And those various estimators didn’t even converge to the mean, hence the differences in the measures of fit!

I politely told the editor, in all sincerity, that this was the stupidest thing I had ever read in my life, and in political science it would have never gone out for review. The somewhat apologetic response allowed that it might not be the finest contribution from the field, as the journal was new (and, I’m sure, expensive: gotta make the percentage of library budgets that go to serials asymptotic to 100%!) and was being submitted for a special issue. Right.

After completing the review, I tracked down the piece (I follow the political science norm of respecting double-blind review processes): it was from one of the top computer science shops in the country, and the final “author” (who I presume had never even glanced at the piece) was the director of a research institute with vast levels of government funding. Such is the degeneracy of contemporary academic computer science. I’m hardly the only person to notice this issue: see this from Science.

17. This, of course, being the dominant issue in political-economy for the first half of the 21st century: the Chinese have created a highly economically successful competitor to liberal market polities, and we have also seen a convergence in market concentration in the new companies dominating the heights of markets in both systems. However, we’ve got 200 years of theorizing—once called “conservative” (and before that “liberal”) in the era before “conservative” became equated with following the constantly changing whims of a deranged maniac—arguing that decentralized economic political-economic systems should provide long-term advantages over authoritarian systems. But that sure the heck isn’t clear at the moment.

18: Hegel , of course, had similar ideas 200 years ago, but wasn’t very good with PowerPoint™: sad.

19. Do these elaborate proposal preparation shops figure into the high indirect costs of the established contractors? Nah…of course not, because we know proposals are done by legions of proposal fairies who subsist purely on dewdrops and sunlight, costing nothing. Or if they did, those costs would be reimbursed by the legions of proposal leprechauns and their endless supplies of gold. None of this ever figures into indirect cost rates, right?

20. As distinct from providing 200+PowerPoint™ slide decks for day-long monthly program reviews: they’ll be great on that as well!

21. Turns out astrophysics has a wonderful name for the undocumented code people write figuring no one will ever look at it only to find it’s still in use twenty years later: “dinosource.”