Monthly Archives: December 2013

“Oslo is the cradle of rigorous causal inference”

Those are the words of James Heckman, from a lecture (slides, paper) at the University of Oslo last week. In particular, it is Trygve Haavelmo’s 1943 paper The statistical implications of a system of simultaneous equations (pdf) that gets the honor of being “the first rigorous treatment of causality”. A summary:

Heckman 2013 Haavelmos contributions to causality

According to Heckman, Haavelmo built on Marshall’s general idea of ceteris paribus to define fixing (“an abstract operation that assigns independent variation to the variable being fixed (p. 8)”), that is to be distinguished from classical statistical conditioning (“a statistical operation that accounts for the dependence structure in the data (p. 8)”). This fixing occur hypothetically, thus causality becomes defined in terms of thought experiments, along the earlier thoughts of Ragnar Frisch. In Heckman’s words: “Causal effects are not empirical statements or descriptions of actual worlds, but descriptions of hypothetical worlds obtained by varying – hypothetically – the inputs determining outcomes. (pp. 2-3)”. 

Much of the lecture and paper is a polemic against Pearl’s do-calculus. Those interested in that debate can read Heckman and Pinto’s paper and Pearl’s comments on it, watch a conference discussion they had last year, or read stuff that more able people than me have blogged about before. Not debatable, though, is that Heckman knows to please his hosts.

Review: The Victory Lab by Sasha Issenberg

Attending a seminar about turnout of young voters in Norway at the Institute for Social Research, I am reminded of Sasha Issenberg’s excellent book The Victory Lab: The Secret Science of Winning Campaigns from last year. Here is a mini-review:

Great story of the work of political campaigns and political participation generally. Someone interested in applying the methods in another context, such as another country, must keep in mind how much they rely on the public voting files that give information of whether someone voted. Without these, the micro-targeting made possible by matching the voter records with consumer databases and census information would not be feasible, and it would make experimentation much more difficult. Recommended, but there is one thing that bothers me: The actual politics is often lost sight of. It would be great if these advances also could contribute to improving policy.

Below is a really long summary:

Introduction
-renewed interest in small improvements after narrow Bush election, etc.
-”crucial divide” … between “new empiricists and the old guard”
-Political consultant Mark Grebner contacted Gerber and Green. Sent people copy of their and their neighbour’s (public) voting histories before election, and said they would inform also about whether voted after election. Increased turnout by 20 %. Problem: Looked dirty/like blackmail. Solution through testing and trial and error: Use a more gentle tone. Also have it look more like information than advertising.
-The problematic story of data in Al Franken’s 2008 Minnesota Senate race. Because of a close initial margin (favoring Republican Coleman), a mandatory recount had to take place. Importantly, there were many votes that were wrongly rejected the first time. These, absentee (challenged), votes were now to be included, and Franken’s team identified those who were most likely to favor him, like those “challenged for change-of-address discrepancies (p 10),” got more of those opened, and carried the race. I do not understand why they did not draw a random sample of those challenged.
-Ch 1. How the field of political science failed to take the road of randomized field experiments.
-beginning of 20th century, political science emerged, and as direct primaries and elections became more common, a study began also of the voters, not only of institutions and parties, as previously. A key man was Gosnell, who in the 1920’s in Chicago experimented with different types of reminders to voters of a coming election. “Gosnell’s conclusions were obvious – mobilization efforts can have the biggest impact in places where little else is pushing voters to the polls – but no one had ever before quantified them (p. 26).” But although his method received a positive reception, the field of randomized field experiments did not take off. Instead the field of polling was developed to great sophistication, with especially the data gathered in what later became “American National Election Studies”.
-Ch 2. Increased focus on who to target – in particular getting out one’s own voters and convincing undecided. Vince Barabba merged socioeconomic information from the 1960 Census with political information to identify precincts where there might be swing voters. Sending letters to potential donors. Malchow arranged a large poll with few questions for Wyden’s campaign to get more disaggregated data. Argued for including controls groups, but met resistance, because the campaigns wanted everyone to be included. One of his workers, Anil Mammen: ““Convincing people to ignore people they would otherwise mail or contact people they would otherwise ignore is a major hurdle. You’re making an argument that’s counterintuitive and your evidence is something they haven’t seen before.”(p. 67)”
-Ch 3. Experiments of Green and Gerber. Shallow critique of rational choice models. Brief history of field experiments. From medicine. Tests of social programs under Lyndon B. Johnson, most famously the effect of a negative income tax: Randomization of bonus payments to low-income groups to look at labor supply effects-found … But not in elections and voting, a field GG went on and made their own. In the 1998 (uncompetitive) New Haven election, they randomized postcards, scripted calls and door knocks to 30 000 New Haven voters, each form of contact additionally containing different appeals. After the election they checked the public(?) individual voter histories and found that the scripted calls had no effect, postcards had almost none, whereas the individual visits had a huge impact, increasing average turnout by 8.7 %. A clear theoretical contribution was hard to formulate, but the practical implications for campaigners were clear.
-Ch 4. The Democrats developed field operations well in the 80’s and 90’s. Paul Tully lead the work in gathering data relevant for turnout, including precinct data from the Census. Taken up by the Republicans in the 00’s.
-Ch 5. The rise of polling, and then of large corporate databases. The Republican Gage in 2001 bought a large file with data on consumers, polled 5 000 of these with about 20 questions on political issues, then looked for connections between the personal and political variables and used what he found to microtarget messages. Developed further for Bush’s 2004 campaign, often targeted the voter with a message about the only issue that the voter agreed with the Republican party with. Also approached scientifically how to elicit anger with questions on that issue. Is is not really clear how the matching between the registries occurred? By name?
-Ch 6. How the geeks took over from the gurus. After 2000, some Democrats realized they were lagging behind. In the 2004 elections, a sample of 20 000 people formed a basis for experimentation with messages when combined with repeated polling and regular mailings. Showed whether people looked at the messages, what worked, and what worked with whom. Credit scores as a model. Emergence of the Analyst Group.
-Ch 7. The power of the social element and psychology. Todd Rogers drawing on Cialdini’s research, like “what he described as injunctive norms (“ you should not litter”) were far less effective at changing behavior than descriptive norms (“ few people litter”) (p. 186),” and towels in hotel rooms. Wanted to test this in the Democratic presidential campaign (for Wesley Clark), but it was decided that the candidate could not afford to leave out a control group of untreated. Later he got the chance in the gubernatorial elections in 2005, and learned that, contrary to the gurus’ opinion, warning about low turnout not effective, whereas emphasizing that others vote is. Grebner collaborated with Gerber and Green to send letters with information of subjects’ vote histories, cloaked in various terms, all of which created effects of 30+%.
-Ch 8. Republican consultant Dave Carney tinkers in various ways. One ex is different anti-Clinton mails, which was effective but not widely implemented in the 2006 presidential election because of fears of a clashback. Teamed up with Perry. Implemented win bonuses for those working for them. Later initiated a collaboration with G&G and two other academics, Daron Shaw and James Gimpel for the 2006 Texas gubernatorial race. They tested: TV ads at different amounts after matching similar media markets-found that had immediate effect that decayed; candidate visits-TV coverage had effect, personal visits had lasting effects. Conclude that tv advertising not very effective, contrary to conventional wisdom, as seen in the money allocated to it.
-Ch 9. The increasing sophistication of microtargeting. Young consultant Dan Wagner hired by Obama 2008 campaign. Database with more or less every American voter. Ken Strasma had been an early proponent of extrapolating political information about known populations to those with otherwise similar characteristics. Special challenges related to the caucuses. Matching people using several hundred consumer variables. Sophistication of Obamas’s primary campaign. Gathering of more and more data, implementation of competition between phone vendors and checking whether they did their job. People less honest when answering a volunteer than a paid call. Tracking of opinion shifts identifying “shifters” giving a more nuanced view of the undecided.
-Ch 10. Sophistication of Obama presidential campaign. Importance of good data from the field. Creation of canvassability, callability and answerability scores to help with this. Randomization of web page features and money requests- “A/B tests”. Demonstration of superiority of online ads and street teams over television for reaching young voters. Ability of the question ““Do you think your neighbors would be willing to vote for an African-American for president? (p. 296)” to pick out those for race was important.
-Epilogue. The social element of voting. Rogers’ experiment with mailing vote histories to people inspiring a follow-up by Malchow: First a phone call asking whether intended to vote, then those that said yes received a letter and then a robocall close to the election reminding them of the pledge – very effective, created new votes at 18 dollars each. Costas Panagopoulos’ experiments: Mailing people message that would publicize those who voted vs those who did not – the latter more effective; thanking those who voted in a previous election had effect; “honor roll” of people never having missed an election or thank you note – both effective. Gerber, intrigued by finding that many believed that the vote was not necessarily secret, found that mail with explanation of how it was kept secret increased turnout dramatically for registered voters that had not voted before. Malchow experimenting on ways to register new voters.

Why do we study?

Now that the world is obsessing about the PISA scores of 2012 is a good time to think a little of the trade-offs involved. South Korea is one of the best performers in international student achievement tests, such as PISA. However it is also “the world’s top producer of unhappy schoolchildren” (h/t MR). Lant Pritchett made a related comment in his Econtalk interview:

I think the parents in Bedford got out of their school system exactly what they wanted out of it. And they wanted football teams. And my wife teachers choir, and they wanted choir. And they wanted the school to put on a musical; and they wanted the school to provide their children with a range of athletic and artistic experiences. And engagement in a variety of other activities; and that’s what the school system delivered. Because it was quite carefully and closely controlled, both formally and informally by the parents. And that produces kind of not world-beating math scores. I don’t think that’s what the parents of Bedford thought was the totality of their educational system. So, I’m a very big fan of the local control by parents of educational systems. And if that doesn’t produce scores of 600, I am actually pretty happy with that. Because I’ve seen what it takes in Korea to produce scores of 600, and no American parent is willing to put their kid through that. Nor should they be, in my opinion.

To put in something more classic as well, here is US president John Adams:

I must study politics and war, that our sons may have liberty to study mathematics and philosophy. Our sons ought to study mathematics and philosophy, geography, natural history and naval architecture, navigation, commerce and agriculture in order to give their children a right to study painting, poetry, music, architecture, statuary, tapestry and porcelain.

(Letter to Abigail Adams, 1780)

Monthly book roundup – November

Books finished in November:
(Warning: reviews are unpolished and quickly written.)

Everything Bad is Good for You: How Today’s Popular Culture is Actually Making Us Smarter, by Steven Johnson.

Recommended. Everything bad is good for you is something as unorthodox as a passionate argument about the cognitive benefits of popular culture. Johnson claims that popular culture today is so challenging and stimulating that we get heavy mental exercise by consuming it. Much of the book focuses on games, which is where I believe Johnson has the strongest case: Popular video games demand thinking and well thought strategies and plans, chores sometimes have to be performed, gratification is delayed, patience is required. Some lament how games are not like books, but like Johnson, says, the novelistic parts of games are their least interesting aspects. In games the content is not the primary benefit, but rather the mental exercise they provide. Reward and exploration are also essential parts: Players have to probe the game, explain it, figure out its rules and find its weak spots. To put it another way, they have to think about the system and what are the limits of the simulation. In this aspect – that ambiguity is essential – video games are different from board games and other traditional games. This is a highly entertaining account of games, and one that concurs very well with my own experience.

Johnson also defends other parts of popular culture, such as television shows and films, that contain many more subplots and where action is expressed with much more subtlety than in previous times. Even reality shows and tv debates get a positive rap, since they require strategy and emotional intelligence and adaption as rule change, in the case of the former, and we are good at judging people by face. To some extent all this seems right, but the question is how much of popular culture it holds for. Though it must be said that at least in the case of tv shows, Johnson argues at length that it is not only niche high brow shows that now have a bigger market to cater to, but that also middle or low brow culture have been lifted.

Johnson sees in all this an explanation of the Flynn effect, i.e. the sustained increase in measured intelligence test scores in many countries throughout most of the 20th century. That is a fascinating thought, but one that would demand more large-scale evidence than hitherto provided to be accepted. Hopefully some researchers out there are on the case.

Naked, by David Sedaris. Very funny.

Reagan In His Own Voice. Ronald Reagan’s daily radio broadcasts from the late 1975-79. In 1976 Reagan launched a failed bid to become the Republican party’s presidential candidate (against the incumbent Gerald Ford). His small radio speeches were probably a way to continue to be in the public sphere. The editors claimed that Reagan wrote most of the material himself. That is impressive if true, although he did not seem to have a formal job until he became president in 1981. Some of the addresses seem quite idiosyncratic, so that supports the claim. I disagree with many of the views expressed, but it is too bad that politicians today seem to be less free to express their personal views, with political communication having become wholly professionalized.

Ratings and additional books are in the library.