Commentary on Mearsheimer and Walt

I’m a bit out of the loop here, so only recently was I alerted to Mearsheimer and Walt’s anti-quantitative methods screed. Which has now been skewered more times than the onions at a kabab joint—most notably by Steve Saideman over at Duck of Minerva—but, having been somewhat selectively quoted in the piece [1], I really should chime in.

Scientific Realism. As I discovered doing background research for 7DS , this apparently useful little term is laden with so much baggage as to be almost meaningless. M&W are apparently using it both to differentiate themselves from the pomos [2], but also invoking the Ur-source of international relations realism, Hans Morgenthau, who asserted early in Politics Among Nations that realism was a “scientific” approach.

Well, sort of: a name means whatever I say it means, and the term “science” does, of course, have a rather curious history in the systematic study of political behavior. Morgenthau claimed the mantle of “science,” but then so did—actually, with some justification—Karl Marx. But also the “American Political Science Association” in 1903, a good 30 years before its members did anything reasonably scientific. So did Sigmund Freud. So did Mary Baker Eddy. So did L. Ron Hubbard.

The grand theory approach of international relations realism probably falling somewhere between the last two.

I’m shocked, shocked… The bigger criticism of the piece is, well, meh… This is news? Most of these points have been in circulation for at least ten years, if not twenty-five. Saideman and other commentators have also noted this.

Furthermore, the piece ignores Sturgeon’s Law, usually quoted as “90% of everything is crap.” Sure, most of the frequentist hypothesis testing is crap, so what? Read any brilliant new grand theory recently? Right—neither have I.

But the piece doesn’t even get the criticisms right. Does the word “Bayesian” appear once in the article? “Causality”? “Propensity function”? “Robust methods?” “Ward, Greenhill and Bakke”? No. The simplistic frequentist hypothesis tests have been under attack by political methodologists for the last couple of decades—in fact that was one of the primary motivations for the organization of the Society for Political Methodology—but unlike M&W, we are suggesting alternatives.

Pot calling the kettle black dept. A core issue of a “scientific” theory is that it must make accurate predictions. Yes, frequentist models generally fail at this, but plenty of recent work succeeds: Integrated Conflict Early Warning System (ICEWS), Political Instability Task Force (PITF). Accuracy at levels considerably higher than a dart-throwing-chimp [3], and in operational use in the U.S. government, more than can be said for most neo-this, neo-that contemporary “theory.” (The big theories, of course, are—for better or worse— quite influential, as Keynes noted so eloquently.[9])

And the track record of the grand theorists of realism…well, not so good, eh? Remember the US response to the OPEC price increases of the 1970s? The annexation of Alberta? The seizure of Mexican off-shore oil wells by the overwhelming power of the US naval fleet operating along interior lines of supply in its historical sphere of influence?  Neither do I.

And then we have Mearsheimer’s famous prediction that Europe could only be stabilized by a nuclear-armed Germany. Yes indeedy, the mere prospect of the Acropolis, Coliseum, Prado and Temple Bar reduced to glowing radioactive rubble under the assault of German IRBMs launched from Peenemünde would have kept the PIIGS in line, yessiree Bob!

Quantitative approaches—particularly the misapplication of hypothesis testing methods which make complete sense in the context of survey research but no sense whatsoever in the context of the analysis of one-off populations—may be wrong, but at least we can systematically say why they are wrong.[4] Grand theory?—welcome to the narrative fallacy and that wonderful little hit of dopamine that your brain gives you in response to any coherent story.[5] And that’s all they’ve got to work with.

Virtually every qualitative theory—possibly excepting those citing Derrida and Foucault—will credibly account for most if not all of the available evidence: That’s how argumentation works, and likely has pretty much since the dawn of language. In the slow-moving world of international relations—all the more so at the level of grand theory—these theories may likely be effectively non-falsifiable within a human lifetime. Though realism doesn’t even pass that standard.  Quantitative theories, if properly tested in split samples (and, per Achen’s arguments, quantitative models vastly more parsimonious than what we usually see), do not have this problem.

Data. M&W assert that we don’t have enough data. Au contraire, the problem at the moment is that we have way too much data and not nearly enough [competent] analysis. Granted, not always the right data, but more than we’ve analyzed: I was recently involved in a long-term project which cataloged the independent variables it had available, and the total came to around 2,700. [that isn't a typo.] And we can generate plenty more, pretty much in real time now.

So why do most of the published articles focus on a small number of data sets? Well, Sturgeon’s Law again, and the risk-aversion of the refereed publication system, and a lot of folks are just plain lazy. But is this worse than the realist obsession with the outbreak of World War I, a sui generis “black swan” if there ever was one? I think not.

Boomeritis. Finally, by far the most incongruous bit in the entire article is the note that this whole series of most dreadful and unfortunate events has come about because students are spending insufficient time in graduate school.

Really?

For starters, the implication here is that these brilliant new grand theories are going to be developed by graduate students? Oh, wait, no, of course they will: by graduate students working under the abysmal conditions of indentured servitude  enlightened tutelage of Messrs. Mearsheimer and Walt. Yeah, right.

Folks, outside of the realm of post-modernism, the absolute last thing I want to read is a grand theory written by a 25-year-old. Even from the University of Chicago. Particularly from the University of Chicago.[6]

The last thing anybody needs—for themselves—is more years in grad school. Get the basics down, learn enough professional literature to enter into the academic conversation that has been going on for 2,600 or so years, learn how to learn, then get out on your own without someone telling you what to think and countless dark hours spent lying awake and drenched in sweat at 4 a.m. in some miserable apartment wondering how you are ever going to reconcile the contradictory demands of the members of your all-powerful but cruel and uncaring dissertation committee.[7] Write your magnum opus after you’ve got some experience under your belt (and your prefrontal cortex is fully connected). Not in graduate school.

So what is going on here in M&W? Well, let me alert you to a controversial assertion at the core of an ancient Asian tradition that has been hidden from most Americans born between 1946 and 1965: old age and death are inevitable. And poor M&W are facing the experience aptly summarized in this recent Doonesbury cartoon, and no longer seeing Death in the rearview mirror [8], but damn, that guy in the black cloak has climbed into the car, is sitting in the back seat, and for godsakes is even criticizing how I drive!

Boomers. A theme we will probably return to in future posts.

Notes

1. There is no distortion of my work in there—I’ve made it rather clear that I’m not overly fond of frequentist hypothesis-grubbing either—though I note that the authors weren’t interested in quoting my observations in 7DS about bad theory.

2. Post-modernists, a drug-addled Boomer affectation which, as I’ve noted elsewhere, tore through political science like intestinal flu in a day care center, with comparable intellectual coherence and outcomes. If you aren’t familiar with the approach, just think of your acquaintances whose primary objective is to stay stoned 24/7 and enlighten you with observations like “Wow, man, have you ever noticed that when you turn a box of brownie mix upside down and look at the list of ingredients, it looks just like the stars in the Andromeda Galaxy? Like, wow, man…” Expect to see the development of a major focus on post-modernism in Colorado and Washington [state].

3. Tetlock‘s, not Ulfelder’s

4. And again, we know of plenty of alternatives, including the rapid emergence of Bayesian model averaging which is likely to wipe out the cult of incremental frequentist garbage can models. The cult otherwise known by the initials APSR and AJPS.

5. Including that posting on Facebook from your college roommate’s best friend’s ex-wife’s cousin about the cute fifth grade Christmas/Chanukah/Kwanza/Dies Natalis Solis Invicti/Yule choral recital his daughter attended.

6. The saddest sights in that City of Big Shoulders are the legions of Chicago grad students lulled into thinking that another year in school will improve their chances on the job market. Yet year follows year, and eventually the spouse/partner who has been supporting them leaves for someone else with better future financial prospects—a chimney sweep, perhaps, or someone who repairs player pianos—and/or their trust fund is wiped out by an incompetent typical financial manager. They roam the streets and coffee houses, a lost ghost army of Dunharrow. And still I hear their pale, plaintive voices…when I dial an 800-number to find out why the ice maker in my refrigerator stopped working.

7. Who will only skim the document, if that.

8. The image popularized by Carlos Castaneda’s fictional Yaqui shaman Don Juan Matus. Yeah, fictional. Okay, so it was originally written as a nominally non-fictional Ph.D. dissertation, but then became a cash-cow for the University of California Press, which certainly needs one…well, grand theory…

9. “The ideas of economists and political philosophers, both when they are right and when they are wrong, are more powerful than is commonly understood. Indeed the world is ruled by little else. Practical men, who believe themselves to be quite exempt from any intellectual influence, are usually the slaves of some defunct economist. Madmen in authority, who hear voices in the air, are distilling their frenzy from some academic scribbler of a few years back. I am sure that the power of vested interests is vastly exaggerated compared with the gradual encroachment of ideas.”
The General Theory of Employment, Interest and Money (1935) Ch. 24 “Concluding Notes” p. 383

About these ads
This entry was posted in Higher Education, Methodology. Bookmark the permalink.

14 Responses to Commentary on Mearsheimer and Walt

  1. Pingback: Addendum: Best Papers I have Read of Late | Will Opines

  2. Prison Rodeo says:

    “AGE: That period of life in which we compound for the vices that remain by reviling those we have no longer the vigor to commit.”

    - Ambrose Bierce

  3. Withrow says:

    “6. The saddest sight in that City of Big Shoulders are…” Arg! Should be “The saddest sight in that City of Big Shoulders IS…” I’m willing to accept inaccuracies in high quality snark, but, in the name of all that we hold sacred in Hyde Park, get the grammar right.

  4. LFC says:

    I didn’t go through your whole post, but re scientific realism: Putting aside how M&W are or aren’t using it, scientific realism is, as I understand it, a fairly coherent position in philosophy-of-science debates. It has nothing to do with IR Realism (political realism), also nothing to do with Morgenthau’s assertion that his approach was “scientific” or with E.H. Carr’s statement that “the science of international politics is in its infancy” etc etc. Rather it has more to do with whether one takes unobservable entities (such as the state) to be real things that can be the (direct) object of study, generalization, and so on. (See eg Wendt in STIP and P.T. Jackson in ‘The Conduct of Inquiry in IR’. Also see PTJ’s comment on this point in the comment thread attached to Saideman’s post.)

    • schrodt735 says:

      Right, but I’m getting mixed vibes on it. When I first encountered the concept, it looked really useful (pretty much along the lines you’ve outlined) and it seems to have developed as a reaction to the whole “science/culture wars” — Sokal etc, but in many ways a lot of this going back to Kuhn — with the basic idea that in fact, science isn’t an unrestricted social construction, but there are a lot of folks who have called themselves “scientists”, and followed a model of “science” more or less along the lines of Francis Bacon (the Elizabethan, not the contemporary postmodern artist), and they do much the same thing, and share a huge number of assumptions about the world, and you can teach the stuff and expect to get fairly consistent results out of it. All well and good. But then I started getting polite pushback (this at various Seven Deadly Sins presentations) from folks in the philosophy of science communities saying, hey, you don’t want to go there, it’s a can of worms. But for our purposes, as political scientists, it may be good enough.
      I’m hoping to get a bit further into this in the longer version of 7DS, but haven’t done so yet.

      • LFC says:

        I’m afraid I haven’t read your Seven Deadly Sins and I’m also not really the right person to have a philosophy-of-science discussion with. (A fair amount of my time on the Internet seems to involve making disclaimers about what I am not an expert on, unfortunately.) So I’ll content myself with noting that many of the 28 comments attached to Steve Saideman’s post at Duck of Minerva are, I think, worth reading, although the comment thread got a bit convoluted at points. (P.T. Jackson’s comment on scientific realism there, in the exchange at the top of the thread, actually has to do with the issue of falsification.)

        I’ve now read the rest of your post, btw, and it’s amusing. Which is not to say I agree with all of it. After all, grand theory and even some postmodernism can be sort of fun, and as someone born in that 1946-1965 period, I have to take that element into account. ;)

      • dnexon says:

        1. If you haven’t already, you should take a look at Fred Chernoff’s collection of writings on Social Science Realism (SSR), as well as PTJ’s book, Conduct of Inquiry. One takeaway is that in advocating SSR, as you do, you aren’t beholden to any particular flavor of scientific realism, but should think hard about what the justification is for embracing SSR, let alone the need for doing so.

        2. FWIW, almost no one of consequence argues that “science is an unrestricted social construction.” Certainly not Kuhn.

      • schrodt735 says:

        Thanks for the cites: again, this is definitely something I need to look into more thoroughly.

        Social construction: Well, depends on what you mean by “one of consequence” — the whole point of the Sokal exercise, cruel as it may have been (here’s the obligatory Wikipedia entry for those unfamiliar with the event, and this was after all almost twenty years ago: http://en.wikipedia.org/wiki/Sokal_affair) was that the post-modernists were making just that contention. As for the role of Kuhn in this, Kuhn obviously wasn’t into the “unlimited” part but Kuhn (or rather, the popularity of Kuhn) did cause the philosophy of science to take an abrupt turn away from the earlier path of the logical positivists (whose agenda, to be sure, had hit a dead end by the 1950s) and into a “sociology of science” which could eventually wind up in the cul-de-sac Sokal could exploit (or illuminate, depending on your perspective)

        I vividly remember — and no, I’m not making this up — arguing with a political science post-modernist in the late 1980s and at one point — we were probably discussing the Middle East — asking “So are you telling me death is a social construction?” Long pause and then the astonishing response: “Yes, death is a social construction.” Cue Edward Abbey [paraphrasing slightly, can't put my hands on the original quote but I think it is from _Desert Solitaire_]: “If a philosophically-inclined individual tells you that everything in life is an illusion, throw a rock at his head. If he ducks, he’s a liar”

  5. dnexon says:

    I don’t know what position this “political science post-modernist” meant to take, but it strikes me as:

    1. Trivially true that the term “death” refers both to an objective phenomena *and* a collection of social facts associated with it; and
    2. Social scientists will sometimes find the latter a more salient object of analysis.

    But this is rather distinctive than the problems in philosophy of science that we blunder into, such as how we make sense of (natural) scientific progress, what epistemological status theoretical terms (in the natural sciences) have, and how to reconcile (or not) the concrete history of (natural) scientific investigation with philosophical claims about it.

    Which is why I’m very much looking forward to seeing 7DS as it develops!

  6. ahom says:

    Just three unrelated thoughts:

    1. The ‘narrative fallacy’, at least as I understand it from the well-known formulation in The Black Swan, isn’t restricted to narrative approaches to research. It looms as a possibility for anyone positing a qualitative, narrative, grand theoretical, quantitative, or hypothetical connection between any two events, experiences, or things. To the extent that the desire or tendency to try to draw comprehensible and meaningful connections between such things is biologically ingrained and/or a product of evolution (see Brian Boyd on this), presenting this as a fallacy isn’t saying much more than that humans are imperfect in yet another way. Without the tendency to connect disparate things, how would we explain unexpected or uintelligible but very impactful events? How would we analyze ever-growing mountains of data?

    Perhaps quantitative methods include better or more clear-cut and step-by-step ways for assessing whether a particular connection is fallacious. Yet one of the implications of recent critiques of AJPS and other outlets on the grounds that their multivariate and significance fetishes ignore assumptions, data manipulation, etc., is that there’s more than one way to draw, defend, and publish a fallacious connection between events. Yes, we can tell a good but spurious story and perhaps be blinded by the dopamine rush, but surely there’s a similarly nice-but-not-entirely-helpful high that comes from *finally* getting a statistically significant result from a data set if it’s been confounding you for a while. Who doesn’t enjoy a ‘eureka!’ moment? In either case, presenting such stories or results without proper reflection, vetting, dialogue, and the like at least exposes one to ridicule and relegation, and at worst perpetuates the fallacy in question, but the initial phenomenon isn’t a specifically ‘narrative’ fallacy (unless we subsume statistics under narrative) so much as a basic human trait.

    2. You are currently reading something I’ve written that touches on these ideas in a way critical of quant. In case it doesn’t come through in that, I do think there’s room for a bridge between quant and qual, and even quant and narrative theory, although I realize that asking quant folks to accept that narrative underpins their approach may be too high an asking price. In any case, having read this post, I am ever more fearful but even more eager to hear your response.

    3. Re: postmodernism and scientific realism, I am even less qualified to comment on this than previous responses, but fwiw, there’s a weird and ironic epistemological dynamic at work in the posty vs. realist debate. ‘Unrestricted social construction’ hits close to the mark, but not because posties insist upon this. Most posites of whom I’m aware would restrict valid social constructions to those which further emancipation, diminish bondage, or otherwise respect a multi-valent humanity. Other social constructions must be contested, disrupted, refused, etc. Fairly standard liberal stuff behind the epistemological and theoretical project(s). Where are the posties supporting human trafficking, subjugation of women, the restriction of immigration and citizenship rights, and the like (well, on immigration and citizenship, you can find them on FoxNews)? It’s not so much ‘reality is ANYTHING ANYONE says it is’ as ‘reality as currently represented in social scientific or political discourse constrains and subjugates certain portions of a population and we think this is wrong, so here’s an alternative discourse that we hope will correct’. Though some posties may evince an aversion to asserting the ‘reality’ of certain concepts, they do tend to want to contribute to *realizing* a specific state of affairs.

    Scientific realism applied to social phenomena, on the other hand, flirts with unrestricted social construction even as it claims to access a ‘real reality’ beneath the fallible layers of experience. Since such depths are not directly accessible to experience, a realist has to provide an explanatory justification for the existence of some thing,and this usually involves story and language. Inasmuch as a sci realist can provide a plausible story about why or how a real entitie’s observable effects failed to launch themselves ‘up’ to the level of observation, we can’t deny the reality of that entity. Sounds a lot like reality is whatever we say it is, although again the plausibility requirement does restrict the construction.

    Not so different, in a lot of ways, at least to my eyes. Of course, I may be completely wrong. But pushed beyond the initial distinction, it often seems to come down to whether or not one is ‘science’ and ‘reality’ averse or ‘science’ and ‘reality’ obsessed.

    • schrodt735 says:

      Thanks for the comments.

      Narrative fallacy: The approach of Kahneman, Talib, Tetlock (not sure if he uses the phrase as it is earlier, but I’m guessing he would be sympathetic) is just what you’ve said: this is a quirk/feature of the human brain, and it strongly affects our ability to interpret events. And yes, it certainly isn’t restricted to narrative research (in fact, if by “narrative research” you mean going out and asking people why they did what they did, which I regard as a perfectly legitimate form of political science research, it wouldn’t affect the *researcher* at all, at least at the data collection stage), but generally affects what is and is not considered “a good explanation.” “Fallacy”, in the sense of “logical fallacy”, is probably a poor choice of terms for this, but it’s what we’ve got at the moment (term-of-art), as it were.

      The “explanatory” interpretation of causal linkages in a quantitative model is subject to this (and gets really whacky when people equal “statistically significant” with “causality”, and that [one of several] ways where the AJPS style of research goes awry. Quantitative prediction (assuming some reasonably objective definition of variables) doesn’t have this problem: the model works, or it doesn’t (or, more generally, it works X% of the time). So dopamine doesn’t come into play. Hence Moneyball, ICEWS and the like

      Scientific realism: Again, this is [obviously] going to need to be a later post in the blog, but to me the issue is whether it is reasonable to assume that there are important aspects of political life where there is a sufficiently high level of inter-subjective agreement that it is reasonable to call them “facts” and to systematically study them. In my opionion, such propositions are going to look more like statements like “The level of material wealth in Norway is greater than it is in Senegal” or “French forces are currently fighting in Mali” than “Fidel Castro’s regime has been very good/bad for Cuba.” But this argument goes way, way, way back: you can find much of this in the politica;l context in the differing approaches of the Sophists, Plato and Aristotle, or in China, Zhuangzi, so we’ve got about 2,400 or so years to work with here. I think on the specific issue of “scientific realism” one can avoid going all the way down the rabbit holes, but I need to do more reading.

  7. Pingback: When the ISA outlaws blogs, only outlaws will blog | asecondmouse

  8. Pingback: What The Fox Doesn’t Know?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s