Author Archive

Campaign Finance Based Forecasts for the 2010 Midterm Elections

November 1, 2010 2 comments

As promised, here are my predictions for the 2010 Midterm Elections. The model parts with poll-based forecasting models in predicting that the Democrats will maintain control of the House. It predicts that Democrats will win between 217 and 238 seats, which translates into a loss of 19 to 40 seats. (The complete set of House predictions is available for downloaded as a .csv or a .xls.)

The histogram displays the predicted number of Democratic Seats from 10,000 bootstrap simulations.

If Congress operated more like a Westminster parliamentary system, fixating on which party will win a majority of seats would be more sensible. In such a setting, after assuming power the majority party (or coalition) is free to enact legislation with as little input from the minority as it pleases. This is not the case in Congress. The past two years have been a constant reminder to Democrats that even large electoral majorities do not grant similar levels of legislative control. Current theories of congressional behavior tell us that the position of the median member of Congress can be as important to policy outcomes as which party is in the majority. An advantage of my forecasting model is that it can predict ideological quantities of interest other than seat shares. For example, I can predict the position of the median member in the next Congress and the extent to which partisan polarization will increase or decrease.

The model projects that the position of the median House member in the 112th Congress will be -0.05 with a 95 percent CI between -0.13 and .13. This represents a sizable shift to the right from the median legislator in the 111th Congress, who was located at -0.24. Yet, this will only bring the median back to where it was during the 110th Congress. To place this in perspective, the median House member in the 111th Congress was in the region of Joe Baca (D-CA) and James Oberstar (D-MN). The model predictions place the median member for the 112th Congress in the region of Arthur Davis (D-AL) and James Marshall (D-GA) but could be as far to the right as John McHugh (R-NY) or former Senator Lincoln Chaffee (R-RI), which is still very moderate. According to the model, even in the best-case scenarios, the House median will be much more moderate than what Republicans experienced during the 104th-109th Congress.

The figure below displays the trend lines for the median House member and the means for each party since 1990. Regardless of which party claims a majority after the election, the model projects an increase in partisan polarization. The mean Republican will experience its largest shift to the right ever recorded, while the mean Democrat also will move further to the left as Republican challengers pick off moderate Democratic incumbents. The general rule of thumb for this election is: the larger Republican gains, the greater the increase in polarization.

I report the model predictions faithfully here, but I remain somewhat skeptical of the model predictions for two reasons. The first is that the realm of campaign finance has undergone changes since the previous election cycle. Not accounting for independent expenditures by outside groups might have biased the model in favor of Democrats. On the other hand, the BCRA arguably represented a much larger shock to the campaign finance system than Citizens United, yet the model predictions for the 2004 Elections were right on target. Moreover, the model does not include any variables that relate to campaign expenditures; it only conditions on fundraising patterns, which remain largely unaffected by Citizens United. The second reason is that the model predicted Democrats would win about 10 seats fewer than they actually did in the 2006 Midterm Elections. It is difficult to determine whether this reflects the Mark Foley October surprise or a failure by the model to account for partisan momentum.

I suspect the model predictions are too generous to Democrats by about 8 to 12 seats. Even with the downward adjustment, I still predict the Democrats will retain their majority, but just barely. This is in line with Sandy Gordon’s forecasting model based on calibrated expert raters. Along with Sandy’s forecasts and the recent polls showing that oversampling of landlines can bias polls in favor of Republicans, my model provides additional evidence that poll-based forecasting models are overstating Republican gains. Fortunately, we won’t have to wait long to find out.

Forecasting House Elections with FEC Records

October 27, 2010 8 comments

As election day nears, I thought it might be an interesting exercise to see how accurately I could forecast election outcomes using only information derived from campaign finance records. Campaign finance records represent a rich data source that speaks to many areas of U.S. politics. Elections are no exception. I’m not the first to incorporate information on fundraising into a forecasting model. However, to the best of my knowledge, I am the first to attempt to forecast election outcomes based solely on FEC records. Despite the handicap of excluding all information from polls, InTrade, expert raters, and other data sources used to forecast elections, the model’s predictions are remarkably accurate. In fact, the out-of-sample predictions for House races outperform the polls.

Contribution records contain far more useful information than what can be expressed by fundraising tallies. For instance, they provide a way to estimate a reliable set of candidate positions via CFscores. The CFscores update almost in real time as FEC records are released during the course of an election cycle. In other words, we don’t have to wait until the election is over to get ideological measures for candidates. The CFscores enable my forecasting model to account for factors that other models ignore, such as adjusting for whether the ideological extremity of Tea Party candidates will hinder Republican electoral prospects this November. In addition, I can forecast ideological quantities of interest such as how the location of the median member of  Congress will change  after the election. This is perhaps less useful in terms of the horse-race but is probably a better overall measure for the type of policy we should expect from the next Congress.

Also informative are the patterns of individual donors across elections. I’ve been working on assigning unique contributor IDs that link contribution records from the same donor across election cycles and across state and federal elections. This may not seem like a big deal, but the ability to track the behavior of individual donors across elections cycles unlocks a wealth of information that had previously been trapped inside the dataset. For instance, linking records across years makes it possible to calculate the proportion of a candidate’s funds that came from first-time donors as opposed to veteran contributors. At the level of campaigns, this can convey information about a candidate’s success in activating supporters. At the national level, the median CFscore across all first time donors serves as a good proxy for the enthusiasm gap.

Much of the model’s predictive power is owed to an idea I borrowed from Sandy Gordon. Sandy has an interesting paper in which he uses past performance of expert election raters to calibrate their predictions in future elections. The paper gave me the idea of treating the hundreds of thousands of donors who had given in previous election cycles as de facto expert raters by looking at the percentage of funds given to winning candidates in previous election cycles. The idea behind this is simple. Some contributors give a greater proportion of their money to candidates that go on to win, while others spend the majority of their money on losing candidates. All else equal, the more money a candidate raises from the type of donors who give to winners, the more likely he is to win. (For additional details on what goes into the model, I include at the bottom of this post a description of the other model predictors, as well as links to download the R script and dataset.)

Tom Holbrook over at Politics by the Numbers nicely overviews the accuracy of election polls. He shows that during the 2006 and 2008 election cycles approximately 85 percent of House candidates who led in the polls 45 days before the election went on to win. As a comparison, the out-of-sample predictions from my forecasting model correctly identify the winner in over 94 percent of combined 2002-2008 House elections.

It is worth noting that my sample includes a number of less competitive House races that lack polling data and hence are excluded from the poll based predictions. The larger sample accounts for some of the increase the prediction rate but not all of it.

The model’s seat share predictions are also close to the mark. The table below shows the number of seats won by Democrats that the model predicts above/below the observed outcome (e.g. a value of 4 indicates the model predicts that Democrats would win four more seats than they actually did; a value of -4 indicates the model predicts Democrats would win four fewer seats than they actually did). I ran 1000 bootstrapped simulations for each election cycle to get uncertainty estimates. The first column reports the median value from bootstrapped runs and the other two columns display the upper and lower 95 percent confidence bounds.


CI Lower Bound

CI Upper Bound

















The predictions are very close to the actual outcome in all but one election cycle. In 2006 the model under predicts Democratic gains by a considerable margin. This might be a result of the Mark Foley factor but I would need more evidence to support that claim. It is just as likely that it reflects a failure of the model to adjust for the effects of partisan momentum in landslide years. (As a note, I noticed that the confidence bounds only contain the observed value in two of the four elections. This suggests that I should probably up the amount of uncertainty in the bootstrapping scheme above the software’s default.)

A major advantage of using campaign finance data to forecast elections is that it costs next to nothing. There is no need to commission polls or pay expert raters. One needs only to collect freely available data and fit a model. That being said, campaign finance based forecasting could have the greatest impact in state level elections where polling data is sparse but fundraising abounds.  Although I’m not convinced that they would be, it might be interesting to seeing if the model predictions are as accurate for state elections as they are for federal elections. The catch is that not every state is quite up to speed with the FEC in releasing contribution records to the public in a timely manner, but this problem is fast solving itself as the disclosure process becomes increasingly digitized.

As much as I would like to have predictions ready for the upcoming election, I haven’t had the time to fully organize the dataset. Those forecasts will have to wait until this weekend. In the meantime, I’ve made the dataset and R script available for download for anyone who might be interested.


Candidate Positioning: I use candidate CFscores to measure  ideology.

Picking Winners: For each contributor, I calculate the percentage of funds given to winning candidates in previous election cycles. Then for each candidate, I calculate the mean value of his contributors weighted by the dollar amounts received.

First-Time Donors: I calculate the percentage of a campaign’s donors that gave the first time during that election cycle. This is intended as a proxy for the enthusiasm gap.

Donor Mood: I construct an aggregate variable by calculating the CFscore of the median first-time donor to proxy public mood.

Fundraising Success: This is measured by the log total amount raised by each candidate and the log sum of unique contributors.

Source of Funds: For each candidate, I calculate the percentages of funds raised from PACs, individual donors, party committees, and self-funding.

A Visual History of Senate Polarization from 1967 to 2010

August 3, 2010 3 comments

(Maximize the video and set the quality to 720p HD for best viewing. The video is also available for download from here.)

Each senator is marked by a two-letter state code and is color coded by party membership (Democrat, Republican, Independent). The top grey bar is the distance between the mean Republican and the mean Democrat (the standard measure of partisan polarization). The size of each party’s coalition is displayed at the end of the bars. The lower grey bar is the gridlock interval, which is measured as the distance between the filibuster pivots (the 34th and 66th most conservative senators before the Senate rule change in 1975 and the 41st and 60th thereafter). The ‘M’ imposed on top of the bar marks the position of the median senator.

The ideological estimates are constructed by scaling senate voting records with dynamic optimal classification. Keith Poole’s paper on optimal classification can be found here, and the paper which explains how to extend the method to smooth legislator ideal points over time can be found here.

Citizens United and the Myth of a Conservative Corporate America

July 12, 2010 3 comments

In my previous post, I looked at the ideology of Google’s employees and board of directors. I have since extended the analysis to board members from twenty major U.S. corporations, including the top ten corporations on the Fortune 500 list. The ideological measures include information from all contributions to state and federal candidates as well as contributions to party and ballot committees. Nearly all members of major corporate boards have made political donations in some form of another. Overall, I was able to estimate positions for about 90 percent of board members from their contributions to candidates or party committees between 1992 and 2008.

The graph below displays the ideological positions of members grouped by corporation. Each row represents a corporation and the points along the line are the locations of its board members. The point size represents the log-scaled total amount given. In addition to the board member positions, I included the median position for each corporate board (black triangles).

These results challenge conventional beliefs about the political leanings of corporate leaders. Republicans have long been seen as the party of big business. To whatever extent this label should apply, it probably owes more to the party’s policies than the composition of its support base. Although board members from some sectors exhibit conservative allegiances—notably the oil, gas, and coal industries—most corporate boards are either dispersed across the ideological spectrum, or seem to have aligned with the left, as is the case of many of the growth stories of the new economy. To provide some context, an ideal point at zero places a contributor directly at the political center, almost exactly at the midpoint between the mean Republican candidate and the mean Democratic candidate. The median ideal point is 0.12 for board members from the top ten corporations on the Forbes 500 list, and is -0.03 for a not quite random sample of 409 board members from other major corporations.

There has been much discussion about how the Supreme Courts’ recent ruling in Citizens United v. FEC will change how corporations involve themselves in elections. The estimates of corporate board members might provide insight on the matter. That being said, I have two preliminary predictions:

1) Corporate boards dominated by liberals/conservatives will be far more likely to become involved in partisan elections.

Boards composed exclusively of liberals/conservatives should have little difficulty agreeing on which side to support. Whatever disagreement does arise will center on whether allocating the funds is worth it. On the other hand, deciding which candidate to support could be a much more contentious issue for boards with both liberal and conservative members. Even if a clear majority agrees to support the Democratic/Republican candidate, forcing a vote would risk upsetting or alienating the conservative/liberal board members. It just does not make sense for a board to engage in partisan conflict when they could easily compromise on not spending the money on either candidate, or better yet, spending it on the type of non-partisan issue ads that are already common. Simply put, bi-partisan boards will rarely take part in partisan politics.

2) Republicans will not be the clear beneficiaries of Citizens United.

Although Citizens United has routinely been interpreted as a favorable development for Republicans, the ideological positions of board members suggest this assessment is premature and might prove incorrect. Although the survey of corporate boards is incomplete, liberals appear to control as many boards as conservatives do.

Insofar as my first prediction is correct, the preponderance of bi-partisan boards means that the corporate money flowing into elections will be closer to a trickle than a flood. If not, then perhaps it will be a relief that the ideological distribution of the people who control large corporations looks quite similar to the population at large.

The University of Google: Was the decision to exit China ideological or business as usual?

March 30, 2010 Leave a comment

After years of unease about the Chinese government’s censorship policies, Google announced this month that it would be shutting down its Internet search service in Mainland China, citing the recent cyber attacks on its systems, widely believed to have been orchestrated by the Chinese government, as “the straw that broke the camel’s back.” Instead of selling off their existing Chinese operations to the highest bidder and leaving town, Google pursued the more risky strategy of rerouting searches through its Hong Kong servers, which remain free from government censors.

Speculation surrounding Google’s motives seems to have generated as much media attention as the business implications of the fallout. While watching the story develop during the past week, I came up with three general perspectives on why Google really left China:

1) Google is making a principled stand against the Chinese government’s censorship and repression of free speech at the cost of its bottom-line;

2) Google is pursuing a strategy that appears costly in the near term but will benefit its business model in the long run;

3) Google was tired of being pushed around by what it saw as an increasingly hostile foreign government.

While it may be tempting to dismiss the first account as overly naïve, I think we would be remiss to discount the possibility that Google’s ideology was central to its decision. There are two reasons that I can think of to support this claim. The first is that Google has been upfront from the beginning that its decision had a major ideological component. In an interview with the Wall Street Journal, Google co-founder Sergey Brin invoked his personal experience with totalitarianism to characterize the decision as a principled stand against Internet censorship and government surveillance.

The second reason is that, despite its status as a Fortune 500 company, Google is among a growing group of powerful corporations closely aligned with the left. Based on my estimates, Google’s employees are the fourth most liberal of any U.S. corporation, behind Genentech, Apple Inc. and Starbucks (See figure above or click here to view in table format). A table with the  To put this in perspective, consider that during the 2008 election cycle, Google employees raised $20,800 for John McCain and $55,451 for Ron Paul; compared to $89,300 for Hillary Clinton and an astounding $803,436 for Barack Obama. In fact, in terms of political contributions, the employees at these firms more closely resemble faculty at liberal universities than traditional Fortune 500 corporations. In some ways, this makes sense. With the exception of Starbucks, each of the most liberal firms, in large part, deals in research and innovation, and each has a reputation of actively recruiting Ph.Ds.

This is the where I would usually caution against putting too much stock into what the ideology of employees reveal about a corporation’s decision-making, because it is the board of directors that ultimately decides issues of this magnitude. However, Google’s board of directors also appears to be decidedly liberal.

In total, seven members of Google’s board have contribution records that can be used to gauge their ideology. Two of the board’s members, Eric Schmidt (-0.71; view records) and John Doerr (-0.72;  view records) are major fundraisers for the Democratic Party and act as advisors to the Obama Administration. Ram Shriram (-1.78; view records) and Princeton University President Shirley Tilghman (-1.43; view records) are less active contributors but have contributed exclusively to Democrats. The only political contributions made at the federal level by Google co-founders Sergey Brin (view records) and Larry Page (view records) have been to Google’s corporate PAC. However, each has contributed heavily to California ballot initiatives. Although their ideological estimates cannot be directly compared to estimates recovered from FEC data, an ideological mapping of contributions in California places Brin and Page to the left of the average Democratic candidate for the California State Assembly. Lastly, Intel CEO Paul Otellini (0.02; view records) has given in roughly equal proportions to both parties.

Ideology aside, there is a case to be made that Google’s business model thrives on free, open and democratic societies, which is one of the arguments Google has used to justify its exit from China. It has even gone so far as to suggest to Congress that the U.S. should consider withholding development aid to counties that restrict access to certain websites. This has some merit, but it sounds suspiciously like a post-hoc rationale. While censorship is harmful to Google’s business objectives—especially the wholesale blocking of YouTube and Blogger—it is difficult to see how such laws are of a different nature than the types of regulation imposed on other industries. When faced with unfavorable regulation, as long as their operations remain profitable, corporations usually respond either by meeting the minimal requirements for compliance and lobbying for reform, or when feasible, moving their operations out of state or off-shores, which is essentially what Google ended up doing.

In the end, Google’s decision to leave China was likely a mixture of ideological and business considerations; which I think makes it much more likely that Google could actually begin exerting influence over U.S. foreign policy. A corporation’s efforts to influence policy are often most potent when its ideology and profit motives align. Perhaps the best example of this during the past century was the United Fruit Co., whose distaste for leftist regimes was matched only by its profit motive. It is the United Fruit Co. and its infamous history of involvement in Latin American politics to which we owe the term banana republic. The more idealistic view is that as Google expands, it will leave a handful of liberated Google republics in its wake. This is not the first time Google has openly defied a foreign regime. Last summer, Google rushed its Farsi translation tool to market in response to the pro-democracy protests. Then again, unlike the Chinese case, Google had little to lose by angering the Iranian government. It is an admirable thought, but making enemies of governments, foreign or domestic, has not yet proven viable as a long-term business strategy—and even Google is unlikely to change that.

Improved and Extended Industry/Occupation Plots

March 23, 2010 3 comments

I updated the industry and occupational rankings graph to include all repeat contributors in election cycles between 1990 and 2008. Anyone who has seen the previous figure will notice an overall shift the right for most industries and occupations. By including two-decades worth of election cycles, the estimates tend to smooth out the shift to the left caused by the unusually favorable climate for Democrats during the 2008 election. On the other hand, it tends to understate the extent to which certain industries have moved to the extremes in recent years.

I also added a few features to the plots in response to being rightly called out for the sloppy/incomplete presentation of the previous graph by an expert on statistical graphics. These new and improved graphs  include horizontal bars that represent the 40-60 percentile of each industry/occupation that are intended as rough measures of dispersion–most industries are bimodal, which would make wider distribution bands span nearly the entire graph, hence losing most of their informational content.  In addition, the y-axis now indicates the amount given by repeat contributors within each industry/occupation. These amounts would be much larger if non-repeat contributors were included in the sample as well.

Obviously, not all industry and occupational  categories could be included in the graphs. I plan to post a spreadsheet with a complete ranking of categories sometime in the near future.

The Dynamic French Fourth Republic

March 17, 2010 Leave a comment

As far as legislatures go, the U.S. congress has relatively stable legislative ideology. Most members of Congress adopt an ideological position upon entering Congress and maintain it for the remainder of their careers. Rarely do we see dramatic ideological shifts, and when we do, there is usually a story to explain the change—e.g. Senator Wayne Morse. This is not the case for many legislatures outside the U.S.. The French Fourth Republic was perhaps the best example. A chaotic legislature in constant motion, the National Assembly of the French Fourth Republic was well known for its dynamism and instability. A major contributing factor was the prevalence of party switchers. In fact, almost 20 percent of legislators switched parties one or more times between 1946 and the Fourth Republic’s collapse in 1958.

Below is an animation constructed from a dynamic optimal classification scaling with smoothed legislator estimates. Despite the fact that the French Fourth Republic is characterized by highly ideological voting, legislator ideal points were often subject to change, as only a fraction of members held consistent ideological positions for the duration of their careers.

(Maximize the video for best viewing. The roll call dataset is courtesy of Howard Rosenthal and Erik Voeten and is available for download here. The paper on the dynamic scaling method is available for download here.)

Watching the legislators in motion is quite a sight and makes the U.S. Congress seems somewhat boring in comparison. I’m not much of an expert on mid-twentieth century French politics, so I don’t have much commentary to add to the animation.  Although, a few things to look for are 1) early party positioning and legislator sorting in the first legislature and 2) the rise and collapse of the Gaullists during the second legislature.

Categories: Uncategorized