Measuring Bipartisanship, Or the Overquantification of Political Science

Screen Shot 2015-09-18 at 11.38.56 PM

UPDATE: I’m not sure anything I wrote about DW NOMINATE is right here, I’ll be reading up on it. If you happen to be an expert on quantitative measures of polarization you can email me at jackhlandry@gmail.com    

 Over at Vox, Matt Yglesias posts this graph showing a slight increase of bipartisanship the last two Congresses. More specifically, it shows that the share of bills introduced with at least one co-sponsor from the opposite party. Yglesias notes that this uptick in bipartisanship is increasing despite the fact that the ideological composition of Congress is largely the same.

     What evidence is there that members of Congress haven’t become more moderate in recent years? While Yglesias doesn’t cite it the strongest and most common measure used in political science literature about polarization is DW-NOMINATE. It uses a complex statistical method to compare the ideology of members of Congress relative to each other. As you can see, ideologies have increasingly polarized in both chambers since 1960 and shows no sign of abating.

Screen Shot 2015-09-18 at 11.39.06 PM

     The problem with these measures is that they’re both purely quantitative. Neither consider that actual content within bills, they only measure the votes on them. So theoretically, a significant portion bills that have bipartisan cosponsors could be naming post offices. You may think I’m being facetious, but according to the Congressional Research Service, “More than one in five of the public laws passed by the 110th and 109th Congresses were post office naming bills.” The first two post offices bills I found had bipartisan cosponsors from multiple states. They were also voted on unanimously, which would impact DW NOMINATE scores. Yet these bills are obviously irrelevant to the overall level of partisanship in Congress. While more rigorous analysis is needed to see exactly how significant naming post offices is on these statistics, the lesson is clear. Without examining a bill’s actual content, you can’t make judgements about partisanship based on purely quantitative statistics.

     What if Congress Members are trying to artificially inflate how “bipartisan” they look by co-sponsoring trivial legislation with their colleagues across the ille. While it’s probably far fetched, my two-bill analysis didn’t rule it out. And as political science stands now, it seem unlikely that anyone would catch it.

     What if, a few Republicans co-sponsored the Affordable Care Act and more voted for it instead of zero. While this would only be one vote in both the DW NOMINATE and Quorum’s bipartisanship measure, it would of drastically changed the narrative for the 2010 elections and been a defacto reduction in the polarization in Congress.

     Those that support the quantitative approach would probably respond that a bipartisan vote on Obamacare would lead to more bipartisan votes on other matters and ultimately be reflected in the scores. My response is that if the quantitative approaches are really robust, they should be checked against more subjective accounts of votes and bills’ actual content.

     The emphasis on quantitative rigor and avoiding subjectivity in political science has led to the end of systematically examining the content of bills. While there are obviously drawbacks to using more subjective measures for assessing polarization, quantitative measures have drawbacks too. It’s best to use both of the tools of political science, quantitative and qualitative, to make judgements about trends. An obsession with quantitative rigor could blind political science to the reality of what’s really happening in Congress

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s