In February a headline widely shared on social media decried poor reviews of the new film Birds of Prey and blamed it on male film critics hating the film for real or perceived feminist messages (and/or skewed expectations; it’s not clear). The article, by Sergio Pereira, was headlined “Birds of Prey: Most of the Negative Reviews Are from Men.”

The idea that the film was getting bad reviews because hordes of trolls or misogynists hated it was certainly plausible, and widely discussed for example in the case of the all-female Ghostbusters reboot a few years ago. As a media literacy educator and a film buff, I was curious to read more, and when I saw it on a friend’s Facebook wall I duly did what the writer wanted me (and everyone else) to do: I clicked on the link.

I half expected the article to contradict its own headline (a frustratingly common occurrence, even in mainstream news media stories), but in this case Pereira’s text accurately reflected its headline: “Director Cathy Yan’s Birds of Prey (and the Fantabulous Emancipation of One Harley Quinn), starring Margot Robbie as the Clown Princess of Crime, debuted to a Fresh rating on Rotten Tomatoes. While the score has dropped as more reviews pour in, the most noticeable thing is that the bulk of the negative reviews come from male reviewers. Naturally, just because the film is a first for superhero movies—because it’s written by a woman, directed by a woman and starring a mostly all-female cast—doesn’t absolve it from criticism. It deserves to be judged for both its strengths and weaknesses like any other piece of art. What is concerning, though, is how less than 10% of the negative reviews are from women.” In the article and later on Twitter Pereira attributed the negative reviews to an alleged disparity between what male film reviewers expected from the film and what they actually saw, describing it as “literally… where a bunch of fools got upset about the movie they THOUGHT it was, instead of what it ACTUALLY was.”

I was reminded of the important skeptical dictum that before trying to explain why something is the case, be sure that it is the case; in other words question your assumptions. This is a common error on social media, in journalism, and of course in everyday life. We shouldn’t just believe what people tell us—especially online. To be fair, the website was CBR.com (formerly known as Comic Book Resources) and not, for example, BBC News or The New York Times. It’s pop culture news, but news nonetheless.

Curious to see what Pereira was describing, I clicked the link to the Rotten Tomatoes listing and immediately knew that something wasn’t right. The film had a rating of 80% Fresh rating—meaning that most of the reviews were positive. In fact according to MSNBC, “The film charmed critics [and is] the third-highest rating for any movie in the DCEU, just behind Wonder Woman and Shazam.” Birds of Prey may not have lived up to its expectations, but the film was doing fairly well, and hardly bombing—because of male film reviewers or for any other reason.

I know something about film reviewing; I’ve been a film reviewer since 1994, and attended dozens of film festivals, both as an attendee and a journalist. I’ve also written and directed two short films and taken screenwriting courses. One thing I’ve noticed is that for whatever reason most film critics are male (a fact I double checked, learning that the field is about 80% male). So, doing some very basic math in my head, I knew there was something very wrong with the headline—and not just the headline, but the entire premise of Pereira’s article.

Here’s a quick calculation: Say there are 100 reviewers. 80 of them are men; 20 are not. If half of each gender give it a positive review, that’s 40 positive reviews from men and 10 from women, for a total of 50% approval (or “Fresh”) rating. If half of men (40) and 100% of the women (20) gave it a positive review, that’s a 60% Fresh rating. If three-quarters of men (60) and 100% of the women (20) gave it a positive review, that’s an 80% Fresh rating.

Birds of Prey had an 80% Fresh rating. So even if every single female reviewer gave the film a positive review—which we know didn’t happen just from a glance at the reviews on RottenTomatoes—then that means that at least three out of four men gave it a positive review. Therefore it might statistically be true that “most of the negative reviews are from men,” but the exact opposite is also true: most of the positive reviews are from men, simply because there are more male reviewers. The article’s claim that “the bulk of the negative reviews come from male reviewers” is misleading at best and factually wrong at worst.

I also thought it strange that Pereira didn’t specify which male reviewers he was talking about. By “fools” did he mean professional film critics such as Richard Roeper and Richard Brody were overwhelmingly writing scathing reviews of the film? Or did he mean reviews from random male film fans? And if the latter, how did he determine the gender of the anonymous reviewers? It was possible that his statistic was correct, but readers would need much more information about where he got his numbers. Did he take the time to gather data, create a spreadsheet, and do some calculations? Did he skim the reviews for a minute and do a rough estimate? Did he just make it up?

I was wary of assuming the burden of proof regarding this claim. After all, Sergio Pereira was the one who claimed that most of the negative reviews were by men. The burden of proof is always on the person making the claim; it’s not up to me to show he’s wrong, but up to him to show he’s right. Presumably he got that number from somewhere—but where?

I contacted Pereira via Twitter and asked him how he arrived at the calculations. He did not reply, so I later contacted CBR directly, emailing the editors with a concise, polite note saying that the headline and articles seemed to be false, and asking them for clarification: “He offers no information at all about how he determined that, nor that less than 10% of the negative reviews are from women. The RottenTomatoes website doesn’t break reviewers down by gender (though named and photos offer a clue), so Pereira would have to go through one by one to verify each reviewer’s gender. It’s also not clear whether he’s referring to Top Reviewers or All Reviewers, which are of course different datasets. I spent about 20 minutes skimming the Birds of Prey reviews and didn’t see the large gender imbalance reported in your article (and didn’t have hours to spend verifying Pereira’s numbers, which I couldn’t do anyway without knowing what criterion he used). Any clarification about Pereira’s methodology would be appreciated, thank you.”  They also did not respond.

Since neither the writer nor the editors would respond, I resignedly took a stab at trying to figure out where Pereira got his numbers. I looked at the Top Critics and did a quick analysis. I found 41 of them whose reviews appeared at the time: 26 men and 15 women. As I suspected, men had indeed written the statistical majority of both the positive and negative reviews.

Reactions

On my friend’s Facebook page where I first saw the story being shared I posted a comment noting what seemed to be an error, and offering anyone an easy way to assess whether the headline was plausible: “A quick-and-dirty way to assess whether the headline is plausible is to note that 1) 80% of film critics are male, and that 2) Birds of Prey has a 80% Fresh rating, with 230 Fresh (positive) and 59 Rotten (negative). So just glancing at it, with 80% of reviewers male, how could the film possibly have such a high rating if most of the men gave it negative reviews?”

The reactions from women on the post were interesting—and gendered: one wrote, “did you read the actual article or just the headline? #wellactually,” and “haaaaard fucking eyeroll* oh look y’all. The dude who thinks he’s smarter than the author admits its maybe a little perhaps possible that women know what they’re talking about.”

The latter comment was puzzling, since Sergio Pereira is a man. It wasn’t a man questioning whether women knew what they were talking about; it was a man questioning whether another man’s harmful stereotypes about women highlighted in his online article were true. I was reminded of the quote attributed to Mark Twain: “It’s easier to fool people than to convince them they’ve been fooled.” Much of the reaction I got was critical of me for questioning the headline and the article. I got the odd impression that some thought I was somehow defending the supposed majority male film critics who didn’t like the film, which was absurd. I hadn’t (and haven’t) seen the film and have no opinion about it, and couldn’t care less whether most of the male critics liked or didn’t like the film. My interest is as a media literacy educator and someone who’s researched misleading statistics.

To scientists, journalists, and skeptics, asking for evidence is an integral part of the process of parsing fact from fiction, true claims from false ones. If you want me to believe a claim—any claim, from advertising claims to psychic powers, conspiracy theories to the validity of repressed memories—I’m going to ask for evidence. It doesn’t mean I think (or assume) you’re wrong or lying, it just means I want a reason to believe what you tell me. This is especially true for memes and factoids shared on social media and designed to elicit outrage or scorn.

The problem is when the person does occasionally encounter someone who is sincerely trying to understand an issue or get to the bottom of a question, their knee-jerk reaction is often to assume the worst about them. They are blinded by their own biases and they project those biases on others. This is especially true when the subject is controversial, such as with race, gender, or politics. To them, the only reason a person would question a claim is if they are trying to discredit that claim, or a larger narrative it’s being offered in support of.

Of course that’s not true; people should question all claims, and especially claims that conform to their pre-existing beliefs and assumptions; those are precisely the ones most likely to slip under the critical thinking radar and become incorporated into your beliefs and opinions. I question claims from across the spectrum, including those from sources I agree with. To my mind the other approach has it backwards: How do you know whether to believe a claim if you don’t question it?

If the reviews are attributable to sexism or misogyny due to feminist themes in the script—instead of, for example, lackluster acting, clunky dialogue, lack of star power, an unpopular title (the film was renamed during its release, a very unusual marketing move), or any number of other factors unrelated to its content—then presumably that same effect would be clear in other similar films.

Ironically, another article on CBR by Nicole Sobon published a day earlier—and linked to in Sergio Pereira’s piece—offers several reasons why Birds of Prey wasn’t doing as well at the box office, and misogyny was conspicuously not among them: “Despite its critical success, the film is struggling to take flight at the box office…. One of the biggest problems with Birds of Prey was its marketing. The trailers did a great job of reintroducing Robbie’s Quinn following 2017’s Suicide Squad, but they failed to highlight the actual Birds of Prey. Also working against it was the late review embargo. It’s widely believed that when a studio is confident in its product, it will hold critics screenings about two weeks before the film’s release, with reviews following shortly afterward to buoy audience anticipation and drive ticket sales. However, Birds of Prey reviews didn’t arrive until three days before the film’s release. And, while the marketing finally fully kicked into gear by that point, for the general audience, it might’ve been too late to care. Especially when Harley Quinn’s previous big-screen appearance was in a poorly received film.”

The Cinematic Gender Divide

A gender divide in positive versus negative reviews of ostensibly feminist films (however you may want to measure that, whether by the Bechdel Test or some other way—such as an all-female cast, or female writer/directors), is eminently provable. It’s not a subject that I’ve personally researched and quantified, but since Pereira didn’t reference any of this in his article, I did some research on it.

For example Salon did a piece on gender divisions in film criticism, though not necessarily finding that it was rooted in sexism or a reaction to feminist messages: “The recent Ghostbusters reboot, directed by Paul Feig, received significantly higher scores from female critics than their male counterparts. While 79.3 percent of women who reviewed the film gave it a positive review, just 70.8 percent of male critics agreed with them. That’s a difference of 8.5 percent… In total, 84 percent of the films surveyed received more positive reviews from female reviewers than from men. The movies that showed the greatest divide included A Walk to Remember, the Nicholas Sparks adaptation; Twilight, the 2008 vampire romance; P.S. I Love You, a melodrama about a woman (Hilary Swank) grieving the loss of her partner; Divergent, the teen dystopia; and Divine Secrets of the Ya-Ya Sisterhood… Men tended to dislike Young Adult literary adaptations and most films marketed to teenage girls. Pitch Perfect, which was liked by 93.8 percent of female critics, was rated much lower by men—just 76.9 percent of male reviewers liked it.” (There was nothing supporting Pereira’s assertion that critics, male or female, didn’t like films marketed to the opposite gender because of a perceived gap between what the reviewers expected from a film versus what the film delivered.)

The phrasing “just 76.9 percent of male reviewers liked Pitch Perfect” of course invites an ambiguous comparison (how many should have liked the film? 90%? 95%? 100%?). More to the point, if over three-quarters of men liked the obviously female-driven film Pitch Perfect, that rather contradicts Pereira’s thesis. In fact Pitch Perfect has an 80% Fresh rating on RottenTomatoes—exactly the same score that Birds of Prey did. We have two female-driven films with the majority of male reviewers giving both films a positive review—yet Pereira suggests that male reviewers pilloried Birds of Prey.

Perpetuating Harmful Stereotypes

Journalists making errors and writing clickbait headlines based on those errors is nothing new, of course. I’ve written dozens of media literacy articles about this sort of thing. As I’ve discussed before, the danger is that these articles mislead people, and reinforce harmful beliefs and stereotypes. In some cases I’ve researched, misleading polls and surveys create the false impression that most Americans are Holocaust deniers—a flatly false and highly toxic belief that can only fuel fears of anti-Semitism (and possibly comfort racists). In other cases these sorts of headlines exaggerate fear and hatred of the transgender community.

As noted, Pereira’s piece could have been titled, “Birds of Prey: Most of the Positive Reviews Are from Men.” That would have empowered and encouraged women—but gotten fewer outrage clicks.

In many cases what people think other people think about the world us just as important as what they personally think. This is due to what’s called the third-person effect, or pluralistic ignorance. People are of course intimately familiar with our own likes and desires—but where do we get our information about the 99.99% of the world we don’t and can’t directly experience or evaluate? When it comes to our understanding and assumptions about the rest of the world, our sources of information quickly dwindle. Outside of a small circle of friends and family, much of the information about what others in the world think and believe comes from the media, specifically social and news media. These sources often misrepresent the outside world. Instead they distort the real world in predictable and systemic ways, always highlighting the bad and the outraged. The media magnifies tragedy, exploitation, sensationalism and bad news, and thus we assume that others embody and endorse those traits.

We’re seeing this at the moment with shortages of toilet paper and bottled water in response to Covid-19 fears. Neither are key to keeping safe or preventing the spread of the virus, yet people are reacting because other people are reacting. There’s a shortage because people believe there’s a shortage—much in the way that the Kardashians are famous for being famous. In much the same way, when news and social media exaggerate (or in some cases fabricate) examples of toxic behavior, it creates the false perception that such behavior is more pervasive (and widely accepted) than it actually is. Whether or not male film reviewers mostly hated Birds of Prey as Pereira suggested—and they didn’t—the perception that they did can itself cause harm.

I don’t think it was done intentionally or with malice. But I hope Pereira’s piece doesn’t deter an aspiring female filmmaker who may read his widely-shared column and assume that no matter how great her work is, the male-dominated film critic field will just look for ways to shut her out and keep her down merely because of her gender.

There certainly are significant and well-documented gender disparities in the film industry, on both sides of the camera, from actor pay disparity to crew hiring. But misogynist men hating on Birds of Prey simply because it’s a female-led film isn’t an example of that. I note with some irony that Pereira’s article concludes by saying that “Birds of Prey was meant to be a celebration, but it sadly experienced the same thing as every other female-driven film: a host of negativity about nothing.” That “host of negativity” is not reflected in male film reviews but instead in Sergio Pereira’s piece. His CBR article is itself perpetuating harmful stereotypes about female-driven films, which is unfortunate given the marginalization of women and minorities in comic book and gaming circles.

You can find more on me and my work with a search for “Benjamin Radford” (not “Ben Radford”) on Vimeo, and please check out my podcast Squaring the Strange! 

A longer version of this article appeared on my Center for Inquiry blog; you can read it here.