Significantly Bad: Luck Does Not Explain Oakland A's Playoff Struggles

The A’s lost the 2018 American League Wild Card game to the Yankees yesterday, October 3rd 2018, in a game that favored the Yankees roughly 60-40 according to FiveThirtyEight’s win probabilities. The outcome on its own was not surprising, but it was another devastating loss for a long-term A’s fan such as myself. The A’s have trouble winning in the playoffs, particularly in games where they could advance to the next round.

Since 2000, the A’s are 15-24 in the playoffs for a winning percentage of 0.38.

They are 1-14 (0.06 winning percentage) in games where they could advance.

Those numbers don’t tell us how many games the A’s should have won. The A’s were favored to win in many of those games. How many games should we expect the A’s to have won? How unusual is their record in the playoffs?

It’s significantly bad.

Simulating the A’s Playoffs Since 2000

As far as statistics goes, this is a simple problem. I simply downloaded FiveThirtyEight’s game-by-game win probabilities. I extracted the “rating probabilities” which factor in home field advantage, pitchers, rest, and travel. Then I simulated each A’s playoff game as a Bernoulli trial using the probabilities as the p parameter. I ran 200,000 simulations for the A’s and Yankees playoff scenarios and 100,000 simulations for all other teams.

Since the A’s just faced the Yankees, I used them as a point of comparison.

playoff wins.png

Again, the A’s have won 15 of 35 playoff games since 2000. Of the 200,000 simulations, the A’s won 15 or fewer games only 2.8% of the time. This can be interpreted as a p-value. And the general rule of thumb places statistical significance on results with a p-value of 5% or lower. Hence, the A’s have been significantly bad in the playoffs. Based on the simulation, we would expect them to win 21 games, and they’ve won only 15.

The Yankees have won 70 out of 131 playoff games since 2000. Of the 200,000 simulations, the Yankees won 70 or fewer games 78.8% of the time. So they have gotten lucky, but not unusually lucky. We would expect them to win 66 of their playoff games.

Simulating A’s and Yankees Clinching Games Since 2000

A p-value of 2.8% indicates we’re seeing something unusual with the A’s playoff record. It’s unusual at the one in forty level. But with regards to clinching games (where they could have advanced with a win), the A’s are almost impossibly bad.

Of my 200,000 simulations, in only 0.008% of simulations did the A’s win either one or zero games. That’s a p-value very close to zero. Extremely significantly bad. Based on the simulations, we would expect them to have won 8 games. Not just one.

The Yankees performance in clinching games is very reasonable. They have won 16 out of 33 clinching games since 2000. In the simulations, they won 16 or fewer games 47.1% of the time. The Yankees expected wins for clinching games is 17.

clinching.png

The little yellow bar that would represent as bad or worse scenarios for the A’s is invisible in the plot because it is so rare. The A’s record in clinching games cannot be explained by luck. It’s too unusual. Too extreme.

This is very strong evidence that the win probabilities I used are wrong. The A’s must have had a greater chance of losing. Why?

I really don’t know. I’d like to look into it more. Some possibilities:

  • Youth: Younger teams can’t handle the pressure as well.

  • Lack of premier pitchers: Maybe the A’s haven’t had enough high velocity, premium pitchers.

  • Sports psychology: Some sort of small market team inferiority complex that inflicts the A’s band of misfit toys.

League Playoff Performance

I wasn’t able to look at clinching games across the league, but I did look at playoff performance for MLB since 2000. The A’s are not the worst. They have done slightly better than the Braves and Twins. The vertical lines indicate statistical significance. On the other end are the Giants and Royals. There is a less than 1% chance that the Giants or Royals would win more playoff games if we were to replay their postseasons and the probabilities are correct.

allteams.png

This plot is unusual on its own. Remember how I said the A’s playoff record is unusual at the 1 in 40 level? We have more teams underachieving and overachieving than we would expect. The teams should be bunched up in the middle of the plot.

This leaves me thinking that there is a different dynamic at play in the playoffs. FiveThirtyEight’s win probabilities are missing something.

Wouldn’t we all like to know what it is?