Is Ferguson about Race or Police Force? Twitter Has the Answer

Is Ferguson about Race or Police Force? Twitter Has the Answer

Introduction

In the wake of the Aug. 9 2014 shooting of Michael Brown in the St. Louis suburb of Ferguson, Americans have asked many questions, such as:

  • Was the police officer justified?
  • Was Michael Brown racially profiled?
  • Is the militarization of police a bad thing?
  • Has the media caused the situation to become worse?

Traditionally these questions are answered using polls and surveys. A random sample of the population is asked for their thoughts on a topic. The results are aggregated and statistically distilled to an answer such as “X% of Americans believe…” As we enter the age of big data (where opinions are measured directly from social media), is it still possible to answer these questions in a manner similar to that of polls and surveys? In this post I will look at ways to re-analyze my previous data[1] that are more in line with the conventional survey approach.

Survey Design

Surveys traditionally start with a design. This is the series of questions that will be posed to the sample. There is an art to creating great survey questions. The questions should not be leading, they should allow for a variety of opinions, and they should provide statistical tests to verify the consistency of the responses. Rather than presenting respondents with a simple yes/no question, surveys will often put forth a statement, asking respondents on how strongly they agree or disagree. For example, the question “Was the police officer justified?” may be seen on a survey as:

The police officer was justified in his actions against Michael Brown
Strongly AgreeSomewhat AgreeNeither Agree or DisagreeSomewhat DisagreeStrongly Disagree

Phrased this way, surveyors have several tools at their disposal. The response distribution can be measured for a statistical preference towards agreement or disagreement. In many cases the response distribution might be no different from random chance, in which case it is difficult to conclude anything from the question. The data can be analyzed both with and without extreme responses. In some situations, extreme responses (like strongly agree or strongly disagree) represent biases inherent in the topic. When this happens, the moderate responses may be a more accurate judge of unbiased opinion. Finally, questions can be asked in a different manner to better understand biases. In this case, a related question might be “Are the police justified in using force?”. Looking at the distribution of responses between the two questions can reveal opinion that is specific to Michael Brown’s shooting and not personal ideology.

All this to say that how we phrase the question for the data at hand is very important. Based on the data I have already collected, I can imagine the equivalent four-question survey:

My feelings towards the police are best summarized as:
Very PositiveSomewhat PositiveNeither Positive nor NegativeSomewhat NegativeVery Negative
The shooting of Michael Brown and subsequent events have changed my feelings towards the police:
Very PositivelySomewhat PositivelyNeither Negatively nor PositivelySomewhat NegativelyVery Negatively
The current state of race relations in St. Louis is best summarized as:
Very PositiveSomewhat PositiveNeither Positive nor NegativeSomewhat NegativeVery Negative
The shooting of Michael Brown and the subsequent events have change my view on race relations in St. Louis:
Very PositivelySomewhat PositivelyNeither Positively nor NegativelySomewhat NegativelyVery Negatively

Notice in this virtual survey that the question “was the police officer justified” is not asked directly. While it may be possible to mine the data for answers to these questions, the question itself is polarizing – people discussing the topic on social media have very strongly taken sides. We need to ease into the questions. I propose using a two-step process in order to answer the above questions. First, we shall see which questions in our virtual survey have statistically significant opinion. Then, we shall analyze the tweets for clues as to what is triggering these views.

Survey Results

I transformed my previous sentiment data into bins of Very Positive, Somewhat Positive, Neither Positive nor Negative, Somewhat Negative, and Very Negative. After transforming the data, I filtered the people in my survey for those that had discussed Ferguson AND one of the relevant terms: Police or Black/White. For each person, I calculated their average response before and after the shooting using the provided bins.

From Figure 1, it appears that the distribution of feeling related to the keywords Police and Black/White has changed from before and after the shooting (an ANOVA test verifies that the distributions are indeed different). For both terms, the sentiment of people in St. Louis has gotten markedly worse post-shooting.

likert

Figure 1: Results of Virtual Survey Before and After the Shooting

For police, the distribution is dominated by a “Very Negative” sentiment, even before the shooting. This can be explained if we consider that the police are often involved in very extreme situations. Removing the most extreme bin and focusing on the more moderate responses (Somewhat Negative to Very Positive) uncovers something quite interesting. Before the shooting, the moderate responses are statistically randomly distributed – people were as likely to regard the police “Somewhat Negatively” as they were “Somewhat Positively”. After the shooting, the moderate responses are most likely to be negative. This suggests that amongst people who previously had no extreme views of the police, after the shooting of Michael Brown they now regard law enforcement less positively.

Racial terms (Black/White) show a different pattern. Before the shooting the distribution weight is in the extremities, indicating that St. Louisans were polarized on race. People view race relations with either pragmatic pessimism or hopeful optimism. After the shooting, the hopeful optimists disappeared. During the events following the Aug. 9 shooting death of Michael Brown it is easy to understand how people could lose hope that race relations will ever improve.

Change Analysis

We have statistically established that the shooting and subsequent events had effected people’s feelings negatively. This is understandable and perhaps obvious in light of the images coming from Ferguson of civil unrest, militarized law enforcement, and local leaders denouncing the police. But this does not tell us whether people have changed their views because of the shooting. To measure that, we need to look at the change in people’s sentiment before and after the shooting.

I collected tweets from people mentioning Ferguson and either Police or Black/White both before and after the Aug. 9 shooting. I then measured over how many categories their opinion changed. For example, if prior to the shooting a person had a “Very Positive” view, and after the shooting had a “Very Negative” opinion, then their change is -4. Figure 2 shows the distribution of change.

likert_change

Figure 2: Change in individual opinion following the shooting.

Most of the sentiment weight for “police” is between no change and negative change. Table 1 summarizes these percentages. Combining the change in distribution from Figure 1 with the individual change in opinion from Figure 2, I’m fairly convinced that the events in Ferguson had an effect on people’s opinion of the police.

The shooting of Michael Brown and subsequent events have changed my feelings towards the police:
 Negatively No Change Positively
 20% 73% 7%

Table 1: Change in opinion toward police following Michael Brown’s shooting

Racial terms are not as clear cut. The distribution of change in Figure 2 looks very much like a normal distribution. In fact, a Shapiro-Wilk Test for Normality shows that the change is normally distributed at the 99% confidence level. This makes it much more difficult to say whether the events in Ferguson changed people’s views on race. One consideration when running surveys that measure a change in opinion is that a percentage of people will naturally change their views over time. This noise we model using a normal distribution. The change in opinion that we measure in the racial terms is pretty much what we would expect when comparing two surveys taken at different points in time. While the racial sentiment is lower following the shooting, there is no evidence that the shooting itself changed people’s mind about race relations.

Police Militarization

So what conclusions can we draw? First, the data is inconclusive about race. This is not to say that we can’t answer questions like “Was Michael Brown racially profiled?”, it is just that any answer we attempt to draw from this data set does not appear to me to be statistically relevant. This would probably be the same with survey data. From the perspective of St. Louis residents, this event does not appear to be about race.

There is, however, strong evidence that the Ferguson shooting is about the police. But what about the police, exactly? Are people changing their opinion based on whether they feel the officer was justified in using excessive force? Or is the change in opinion based on the police response to the post-shooting civil unrest? To get a better idea, I looked at a sample of positive and negative tweets. Here are some of the negative ones:

This isn’t just about an unarmed teen being murdered by police. It’s about an out-of-control #PoliceState. #Ferguson

The militarized police force has apparently decided that fighting the protest isn’t enough. They want to arrest all coverage

And here are some of the positive tweets:

Quite encouraged by what is happening with the police presence tonight … night and day difference #Ferguson

And countless more thanking the police/swat team for all they’re doing to keep the rest of us from harm.

Obviously, both the positive and negative tweets are reacting to the police presence during the civil unrest. Negative tweets view the response as overkill. Positive tweets view the police response as welcome and necessary. These are both reactions to the “militarization” of police in the ensuing civil unrest.

Having reviewed that data, I am confident in saying that we can answer quantitatively how people in St. Louis view the militarization of their police force. There is a significant negative shift in sentiment post-shooting among those with moderate views of the police. There is also significant evidence that individual views have changed negatively following the shooting. The content of the tweets indicate that the conversation was mainly about the police response to the civil unrest following August 9.

I conclude that the individual changes in opinion are a result of people’s reaction to the police response to the civil unrest — that it is directly related to how they view the militarized police response. A person whose opinion regressed viewed the militarized police response negatively. Conversely, a person changing their opinion for the better saw the militarized police response as a necessary action. With this interpretation I can quantitatively answer the question:

Is the militarization of police a bad thing?
 Yes No Don’t Know
 20% 7% 73%

I’d like to point out that this approach removes any bias that the above question might have introduced. People were not specifically asked about militarization, but the answer was interpreted through statistical analysis of their reactions to the events. In practice, it would be very difficult to design a survey that did not introduce bias into this particular question.

What About Race?

I don’t want to ignore the racial aspect. From the data, there was definitely an effect from race: racially related sentiment did decrease following the shooting. There is no evidence that the change in sentiment is a result of a change in people’s opinion regarding race. So what caused the change in sentiment?

Many of the racial tweets were sharing news headlines and blog posts:

@AP: Report: Autopsy finds unarmed black Missouri teen was shot 6 times, including twice in the head: http://t.co/I2Bh0BZSY8

@TIME: The fatal police shooting of a black St. Louis teen on Saturday has triggered a public outcry http://t.co/u15PyC3l84

#IfTheyGunnedMeDown: African-Americans protest media portrayals of black victims http://t.co/DJ2sKEjKUp #MikeBrown

While it looks like St. Louisians saw the shooting as a police tragedy, the media framed the story in racial terms. In the aftermath, people shared news stories that included the racially polarizing terms Black/White. These headlines led to much of the racial conversation, but may also have caused the change in people’s views. Headlines tend to be written in a neutral style, which may have skewed the early results.

Tweets that share headlines show a non-random change in sentiment, with the change skewing negatively. Tweets that don’t share headlines still show essentially a random change, further suggesting that the difference in racial sentiment is more media related. This is interesting as it presents the opportunity to quantify how the media coverage has actually changed people’s feelings towards race issues.

To quantify the effects of media on race, I measured the opinion change of people whose tweets either include or exclude media links. I then looked at the difference. This shows how a Twitter user’s view on race is affected by the media coverage of the Ferguson events. If the media coverage has a positive effect, then we should see a person’s opinion of race improve when they include media links in their tweets. Conversely, media coverage having a negative affect will see a person’s opinion of race degenerate. In cases where a person’s opinion has regressed, we can anticipate that they would feel media coverage has made the situation worse, and vice versa for those with a positive change. Table 3 shows the results of this virtual poll:

Is the media causing events in Ferguson to worsen?
 Yes No Don’t Know
 14% 16% 70%

Table 3: Virtual poll results for “Is the media causing events in Ferguson to worsen?”

The statistical edge is that people are finding the media has not caused events to worsen, but only by a slim margin. So while it is clear that the media is playing more on race than the populous of St. Louis originally viewed, the racial coverage has not necessarily made things worse in Ferguson.

Conclusion

I’ve laid out a systematic, statistically significant argument for converting raw Twitter counts and sentiment into results that are similar to those found in polls. This is an important step in reporting on people’s opinions using social media. Previous analysis of counts and sentiment can be difficult to convey in short sound bites. Transforming the information into a survey question representation can make it easier to convey the data to a broader audience, without losing its statistical validity.

Featured Image courtesy of Freebase

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.