(Note: Mark Harris noted that he purposely excluded the first 40ish years because “campaigning, calendar, rules were so different.” I don’t believe this changes the analysis too much - but it’s certainly worth noting.)

Recently, two writers (Mark Harris and David Poland) wrote opposing articles on the effect of the expansion of the Best Picture field to ten possible nominees – essentially, whether it has truly changed the variety of films nominated for Oscars.

Harris’ argument, put less elegantly, is that this expansion has forced Oscar voters to focus their watching (and ultimately, voting) on those most likely to be nominated for Best Picture, instead of a wider array of films. His deduction is that before the expansion, because voters could only nominate five films for Best Picture, it allowed for a larger variety of films to be nominated in all other categories.1 Ultimately, he concludes, this leaves films not in the Best Picture race to fall by the wayside – to him, a very bad thing, when “for movies, but also for the reputation of the Oscars, one major nomination truly is better than nothing.”

Poland argues that the expansion, at the very least, cannot be proven to be a bad thing. To him, nothing has truly changed for the worse – the last two years are simply anomalies, in that two films (Silver Linings Playbook and American Hustle) accounted for 8 acting nominations (and that Lincoln accounted for 3 last year). He says of the David O. Russell films, “just take away his two movies and the statistical analysis gets a lot harder to argue.” Moreover, “Not only are stats malleable, but things simply change.”

A fair point – statistics are absolutely malleable to those who want to use them to support their argument. Often, particular statistics are picked and chosen seemingly at random to support an already decided-upon opinion. And he’s even more right about his second point – things do change.

It’s a lot harder to bend statistics when looking at a full historical trend instead of picking the most supportive data points. Both of these articles miss this fuller historical context - is this really a large change? Have the past few years really seen an expansion in the number of Best Picture nominees, but a contraction in the total number of nominees? Are the last few years really outliers, and if so, how?

Let’s look at the data.2

The first chart shows the total number of nominated films in the top 8 categories for every year since the very first Academy Awards ceremony, as well as the 5-year moving average of that number.3

This gives some initial context around where these last few years really stand relative to history. Are they really that different than historically?

If you can’t tell from the chart – 2013 is number 1 (tied with 19814), and 2012 number 3, in terms of absolute lowest number of nominated films.

But that’s not telling the whole story. The five-year moving average – shown on the same graph – shows that the average of the past 5 years is only the 12th lowest of all time. While average is hardly a perfect measure to use, it at least allows for all of the years since the expansion to be considered. Notably, 2009 had 19 nominated films, and 2011 21 nominated films – above the historical average of 17.6. So this may really be a case of outliers to the historical trend, rather than a representation of the failure of the expansion.

That is, if we only consider the number of films. Importantly, the number of nominations has changed over time – having 18 nominated films in 2008 is actually quite different from having 19 in 2009. The entire point of this argument is due to the increased number of possible nominations. The second chart accounts for these changes. It shows the unique number of nominated films in a year, divided by the total number of nominations in that year. So the closer to 100%, the better – 100% would be a year in which every single nomination was given to a different movie. This is a bit more telling.

Here you’ll see that 2013 is, in fact, the lowest in history. The average of the last 5 years (i.e., since the expansion) is the 3rd lowest ever – behind 1981 and 1982.

The next chart shows the 5-year percentage change in the number of nominated films. So, a -10% change would mean that 10% fewer films were nominated this year than were five years ago.

2013 has the largest negative change over any 5-year period since 1934 – 36% fewer films were nominated this year than in 2009.

These are fairly convincing in singling out the last few years as nominating fewer films than ever. But they don’t get to one of Poland’s points – and, quite possibly, a fair one – that the last two years are, especially, an anomaly in that they each feature a films with 4 acting nominations each (and last year a film with 3). That must be unheard of, right?

Actually, not at all. The chart above shows the number of films per number of acting nominations each year. The orange is 4 – one each in 2012 and 2013, but also in 22 other years (1 out of every 4 years). In back-to-back years? 3 other times. 3 in a row? 1 additional time. 4 in a row? Another time – 1976 through 1979. For 2 films in one year? 4 separate times. 5 acting nominations? 9 times – though most recently for Network, in 1976.5

All of this to say that Silver Linings Playbook and American Hustle hardly make the last two years unique, at least in terms of films with a large number of acting nominations. This has been happening fairly consistently since the 1930s – but you might not know that when only looking at the past 30 years.6

But the last five years are undoubtedly unique in the trend toward fewer films being nominated overall. Whatever the cause – Harris’ theory is reasonable, but wholly unknowable with the limited set of data – it’s clear that it’s trending downward in the last two years.

But what’s also clear is that five years of data is hardly a sample size large enough from which to draw conclusions. Our memories are short term, and when it comes to film awards, they can be even worse (to say that the Academy doesn’t nominate blockbusters is a bit naïve – Lord of the Rings, Titanic, Raiders of the Lost Ark, Star Wars, Jaws, The Exorcist, even The Sound of Music and Gone with the Wind were nominated or won Best Picture, and are some of the highest grossers of all time). While this year and last may be unique, they don’t and can’t yet prove that the expansion is a failure. Should this trend continue for another 5 to 10 years – well, then it may be time to reconsider contracting the field.

Mark Harris - The Nolan Effect: Why the Larger Best Picture Pool Is Actually Shrinking the Number of Oscar Contenders

David Poland - 20W2O: 10 Best Picture Nominees – Part 2, Call & Response

Source of Nomination Data: Academy Awards Database

1 He focused on the eight “top” categories – Best Picture, Director, Actor in a Leading Role, Actress in a Leading Role, Actor in a Supporting Role, Actress in a Supporting Role, Adapted Screenplay, Original Screenplay.

2 One thing to keep in mind (which Harris mentions briefly) – the awards have changed several times over the years. In some of the first few years, there weren’t even, officially, nominees – though the Academy did keep the records of those “under consideration.” (The film years are also actually the span of two years – which is why I use “1927.5” in the graphs – to mean 1927/1928). By 1936, Supporting Actor and Actress had been added to the mix, and between the 1st ceremony and the 16th, as many as 12 films were nominated for Best Picture. But between 1944 and 2008, the top 8 categories remained relatively unchanged – save a few name changes – with 5 nominees for each category. 2009 saw the expansion of the Best Picture field to 10, and 2011 allowed for anywhere between 5 and 10 films to be nominated for that award.

3 The year signifies the year of the films nominated, not the year in which the ceremony is held.

4 In 1981, three films were nominated for the “Top 5” awards – Picture, Director, Actor, Actress, Screenplay – Reds, Atlantic City, and On Golden Pond. One has to think there had to be some outcry of lack of diversification that year – and it may be the best argument for these last two years being, very simply, outliers. And yet, in 1981, Chariots of Fire won Best Picture.

5 Other 5-acting-nominee films: Mrs. Miniver, All About Eve, From Here to Eternity, On the Waterfront, Peyton Place, Tom Jones, Bonnie and Clyde, The Godfather Part II. Chris Tucker and Jeremy Renner, it could’ve been you.

6 Last 3 films with 4 acting nominations (before 2012) – Doubt (2008), Chicago (2002), Terms of Endearment (1982). It could almost be argued that the period between 1982 and 2012 is, in fact, the anomaly.