How to Study

Welcome to the Fall 2011 semester. Want to know how to do a better job of studying this semester? Check out this quick series of videos from a cognitive psychologist.

What do practitioners need to know about regression? - Statistical Modeling, Causal Inference, and Social Science

From Andrew Gelman:

What do practitioners need to know about regression? - Statistical Modeling, Causal Inference, and Social Science:

"More specifically, here are some tips:

- The difference between 'significant' and 'non-significant' is not itself statistically significant.

- Don't just analyze your variables straight out of the box. You can break continuous variables into categories (for example, instead of age and age-squared, you can use indicators for 19-29, 30-44, 45-64, 65+), and, from the other direction, you can average several related variables to create a combined score.

- You can typically treat a discrete outcome (for example, responses on a 1-5 scale) as numeric. Don't worry about ordered logit/probit/etc,, just run your regression already.

- Take the two most important input variables in your regression and throw in their interaction."

Correlation

Correlation does not equal causation. Here is a classic example of an observational study leading to the wrong conclusion. Sadly the original studies continue to be promoted, despite their shortcomings and empirical refutation.

A Simple Regression

For my econometrics class. Here is a nice example of a simple regression and an excellent graph.

Visualizing Store Openings

A dynamic map, plotting the openings of Target and another of Walmart openings. Both chains clearly have very different geographic strategies.

Voter Turnout

Voter Turnout, via Andrew Gelman.

Ray Fair's equation did pretty well. Underpredicting Obama's share by 1.1%.

Andrew has some more comments here, with plenty of data analysis.

Reading Research

The New York Times has a great Health related section this week.  In particular they have this graphic, which details how to read (skeptically) medical research.  Its good advice for reading all kinds of research.

Google Advice To Students

Google gives great advice to students and  sounds like they are describing BUS 230.

Our Googley advice to students: Major in learning
7/15/2008 05:48:00 PM
Management guru Peter Drucker noted that companies attracting the best knowledge workers will "secure the single biggest factor for competitive advantage." We and other forward-looking companies put a lot of effort into hiring such people. What are we looking for?

At the highest level, we are looking for non-routine problem-solving skills. We expect applicants to be able to solve routine problems as a matter of course. After all, that's what most education is concerned with. But the non-routine problems offer the opportunity to create competitive advantage, and solving those problems requires creative thought and tenacity.

Here's a real-life example, a challenge a team of our engineers once faced: designing a spell-checker for the Google search engine. The routine solution would be to run queries through a dictionary. The non-routine, creative solution is to use the query corrections and refinements that other users have made in the past to offer spelling suggestions for new queries. This approach enables us to correct all the words that aren't in the dictionary, helping many more users in the process.

How do we find these non-routine savants? There are many factors, of course, but we primarily look for ...

... analytical reasoning. Google is a data-driven, analytic company. When an issue arises or a decision needs to be made, we start with data. That means we can talk about what we know, instead of what we think we know.

... communication skills. Marshalling and understanding the available evidence isn't useful unless you can effectively communicate your conclusions.

... a willingness to experiment. Non-routine problems call for non-routine solutions and there is no formula for success. A well-designed experiment calls for a range of treatments, explicit control groups, and careful post-treatment analysis. Sometimes an experiment kills off a pet theory, so you need a willingness to accept the evidence even if you don't like it.

... team players. Virtually every project at Google is run by a small team. People need to work well together and perform up to the team's expectations.

... passion and leadership. This could be professional or in other life experiences: learning languages or saving forests, for example. The main thing, to paraphrase Mr. Drucker, is to be motivated by a sense of importance about what you do.

These characteristics are not just important in our business, but in every business, as well as in government, philanthropy, and academia. The challenge for the up-and-coming generation is how to acquire them. It's easy to educate for the routine, and hard to educate for the novel. Keep in mind that many required skills will change: developers today code in something called Python, but when I was in school C was all the rage. The need for reasoning, though, remains constant, so we believe in taking the most challenging courses in core disciplines: math, sciences, humanities.

And then keep on challenging yourself, because learning doesn't end with graduation. In fact, in the real world, while the answers to the odd-numbered problems are not in the back of the textbook, the tests are all open book, and your success is inexorably determined by the lessons you glean from the free market. Learning, it turns out, is a lifelong major.

Descriptors or Treatments

I too have often thought of background variables, such as race, income, and education as descriptors, rather than treatments. However it seems as if most applied econometricians today think of them as treatments. Andrew Gelman posts on this topic:

Income, education, and religion as "background variables" or "treatments"

The discussion here on the climate change attitude mystery reminded me of a funny thing about how we think when we classify people by education, or income, or religion.

The original question was to explain why college-educated Republicans are less likely (compared to non-college-educated Republicans) to believe in human-caused global warming, while, among Democrats, those with college education are more likely to believe in it. To me this was no surprise: college-educated people are more political polarized and are more likely to align their views with their political attitudes.

But many of Tyler Cowen's commenters had a different sort of explanation, along the lines of, Going to college makes Republicans more skeptical of scientific authority but convinces Democrats of these arguments.

Setting aside the specific issue of climate change, one interesting thing here is the way I, in common with most political scientists, think of education (and other variables such as income and religion) as traits, or background variables, or descriptors of people. Thus when we talk about how rich and poor people vote, or more and less educated, or Protestants and Catholics, or whatever, we think of these as different sorts of people. But you can also think of income, or education, or religious attendance, as "treatments" that affect people--for example, if you go to college and share a room with someone of a different ethnic or political group, you might become more tolerant. Or maybe if you are conservative and go to college, you'll be skeptical of what's taught in your physics class (or if you're liberal, maybe you'll be skeptical of what's covered in your econ class).

I don't really have much to add here . . . somehow it seems more reasonable to me to think of these as descriptors than as treatments, but I guess it depends on the person and on what issue is being considered.

Golf and Reliability

A great post on golf and statistical reliability.

Random Sampling

Here is an excellent post from Marginal Revolution on the perils of random sampling. Even when you think you've done it, you might have missed something.

The Wisdom of Whores

As a teenager Elizabeth Pisani discovered that she liked sneaking into the "girlie bars" of Hong Kong and talking to the hookers and the johns. Ordinarily one wouldn't expect such peculiar hobbies to pay off in a dazzling career. But beginning in the mid 1980s, people who were comfortable talking to hookers, rent boys, trannies, warias and junkies (all Pisani's terms) especially in the hot spots of Asia and Africa became scouts and spies in the war against AIDS. Information provided by these epidemiologists in the streets about where AIDS hid and how it spread has been critical to the war.

The Wisdom of Whores is Pisani's vivid account of what she has learned and why AIDS is still being fought ineffectively. I'll say more on this in the future. Here, however, is one bit relevant to the title:

We'd just finished our first survey of HIV, syphilis and risky sex among waria in Jakarta...I had the impression from the qualitative research...that waria were turning dozens of tricks a week, but the study showed they averaged only three. And since that figure came from 250 waria selected at random as the manual requires, it was certainly more accurate than the qualitative research...

"Three a week? Your insane!," snorted [sex worker and waria] Ines....Ines left school at fourteen but she is not stupid, so I explained how, through our probability sampling, we are asking a representative sample of Jakarta's waria. Ines dismissed by lecture with a wave of her manicure and a flick of her locks. "So clever, but so stupid." she sighed.

Ok, so what was the wisdom of the whore? (n.b. Pisani's term!) Bonus points if you can also describe a solution. I am going to have to get this into my econometric notes! Answer in the extension.

She explained, as if to an enthusiastic but slightly dim child, that a waria who is hanging around on a street corner to be interviewed by a research team is a waria who is not with a client. 'You are talking to all the dogs, obviously'.

Not something I learned in the lecture halls of London...but Ines is quite right. Our sample is biased towards the 'dogs,' who get picked up less than the cuter girls. So the study results underestimate the true number of clients per seller...

Ines's comments...prodded us into changing the sampling strategy...now we work with the powers-that-be (the mami, the pimps, the brothel owners) to arrange off-hours time for data collection. The principle....is that you are not cutting into people's work time, so there is less chance of talking only to the remnant sex workers who can't get a client.

Mostly Harmless Econometrics

A new Econometrics book : Mostly Harmless Econometrics.

The core methods in today's econometric toolkit are linear regression for statistical control, instrumental variables methods for the analysis of natural experiments, and differences-in-differences methods that exploit policy changes. In the modern experimentalist paradigm, these techniques answer clear causal questions such as: Do smaller classes increase learning? Do minimum wages reduce employment? Should wife batterers be arrested? How much does education raise earnings? Mostly Harmless Econometrics shows how the basic tools of applied econometrics allow the data to speak.

In addition to econometric essentials, Mostly Harmless Econometrics covers important new extensions--regression-discontinuity designs and quantile regression--as well as how to get standard errors right. Angrist and Pischke explain why fancier techniques are typically unnecessary and even dangerous. The applied econometric methods emphasized in this book are easy to use and relevant for many areas of contemporary social science.

* An irreverent review of econometric essentials
* A focus on tools that applied researchers use most
* Chapters on regression-discontinuity designs, quantile regression and standard errors
* Many empirical examples
* A clear and concise resource with wide applications

Weather as Instrumental Variable

The Freakonomics duo has an interesting column on the use of weather by empirical economists. It has been used as an instrumental variable in several cases. It is likely to be correlated with the included variable but uncorrelated with the error, or the excluded variable.

It was used in the autism study.

I'm thinking the weather might be a good instrument for some sex research. Under the assumption that bad weather leads to staying indoors, and more sex among married couples and cohabitants, but fewer matching opportunities for single people.