Thorns FC: A Tale of Two Keepers

By John Lawes Oct 27, 2017, 2:01pm PDT

Five players have won the NWSL Goalkeeper of the Year award handed out by the league at the end of the season.

A Portland Thorn has won it twice.

Michelle Betos was NWSL Goalkeeper of the Year for 2015.

Adrianna Franch is the NWSL Goalkeeper of the Year for 2017.

In my opinion, the two selections are very different; one troubling, the other nearly irrefutable. In this essay, I discuss the metrics I believe can be used to measure the quality of a soccer goalkeeper, and what they say about each.

Michelle Betos and the 2015 Goalkeeper of the Year

Michelle Betos may well be one of the best-beloved players ever to bear the Thorns FC crest, and she was one of - perhaps the best of - the few happy stories of the otherwise unhappy 2015 season.

I mean, look at her face as she received the Supporters’ Player of the Year Plaque. How could you not love that face? How could you not love the person whose blinding innocence to her own quality made her selection as the fans’ most valuable player a complete and utter shock to her?

And, of course, there was The Goal.

I loved Betos, too, both as a person and as a Thorn. But. I was a goalkeeper back when I played, and I’m a nitpicky sort of a fan, so something kept bugging me every time I read or heard about Betos’ post-season award.

I went and dug deep into the 2015 NWSL goalkeeper stats, and what I learned there convinced me Michelle Betos may well have been a great person, a wonderful teammate, and a credit to Thorns FC and the City of Portland.

What she was not was the best goalkeeper in the NWSL in 2015. In fact, in the metrics that measured pure goalkeeping skill, Betos was well behind many other starting keepers in the league that year.

The statistical record and how I use it

I’m going to use my statistics and explain the process I went through in 2015 to introduce how I’m going to look at the keepers who were eligible for the Goalkeeper of the Year in 2017.

I know that two seasons might as well be a sporting lifetime ago, but bear with me; I think it’s important to use that past season to understand my reasoning for this past season.

Here’s the table from 2015 showing my compilation. It’s limited to NWSL keepers who started more than eight matches that year (with the exception of Nadine Angerer because she was a Thorn and I was curious).

How did I assess pure technical quality in a goalkeeper?

I started with the idea that the most basic measure of quality for an individual goalkeeper is not how many goals the keeper concedes but how many of the shots she faces result in a goal. In other words, not the gross goals scored against her, but the net goals-per-shot-on-goal.

To assess that, the important numbers are:

Shots on goal (SOG, Column 3),
Goals conceded (Column 7), and
Percentage of SOG that result in a goal (Goals conceded %, Column 8)

Note: A fourth metric worth glancing at is the minutes the keeper played. Often, random chance means that a keeper who plays a very short time will concede fewer goals than a regular starter - unless the keeper is struggling.

Example; look at the ninth line down in the table above. Nadine Angerer in 2015? Struggling. Her 11 goals conceded in only 540 minutes makes her the concession rate (32.4% of the shots on target) the worst on the list. That’s probably why Angerer hung ‘em up that autumn. She was just barely hanging on that year.

Typically the best goalkeepers will have the lowest shot-to-concession ratio. Commonsense; if you face ten shots and save them all you’re doing better than if you let in 2 (20%) or 5 (50%).

Who were the top keepers in 2015, by these metrics?

In 2015 the top four starting keepers with the best shot-to-concession ratio were:

Leah Dalton of Chicago (11 goals conceded from 58 shots, so 19% of the SOG she faced went in),
Bianca Henninger of Houston (15 goals from 74 shots, 20.3%),
Hayley Kopmeyer of Seattle (11 goals from 52 shots, 21.2%), and
Nicole Barnhart of FCKC (16 goals from 70 shots, 22.9%)

Compared to the top four, how good was Betos?

Not that good. Betos conceded 18 goals from 63 shots, meaning that 28.6% - almost a third - of the shots she faced got past her.

That was poorer than eight other keepers and more than two and a half points below the league average of 26%.

But how can we tell the difference between good goalkeeping and a good defense out front of that keeper?

We’ve all seen how a good keeper can bail out a poor defense with great positioning and acrobatic saves, just like we’ve seen a great defense make a mediocre keeper look like Lev Yashin with timely blocks, clears, and tough tackles.

We can use two metrics to help us understand how much the field players are helping their goalkeeper.

One, obviously, is the raw number of SOG. The fewer the SOG the better the defense; they’re forcing the opponents to take shots from bad angles, blocking shots, or dispossessing opponents before they can shoot.

In order to measure this as a function of the keeper behind that defense, however, I believe using the SOG per keeper’s minutes played is more useful than just raw SOG or SOG per game.

No matter how good the defense in front of her, a keeper who starts and gets a lot of minutes is much more likely to face more SOG than a keeper whose minutes are limited. Calculating how often during the keeper’s time on the pitch she faces a shot helps reduce that sample size bias.

A brief note here on a link between SOG-to-minute and SOG-to-concession

Without access to the “expected goals” numbers we don’t have a good handle on the quality of those SOG. However, it seems reasonable to postulate that more frequent SOG are more likely to produce more frequent high-quality SOG.

So buried in the SOG-per-minute stat is the assumption that a goalkeeper that faces a lot of SOG-per-minute also has to make more difficult, technically demanding saves on high-quality shots. This, in turn, suggests that a goalkeeper whose defense allows a lot of SOG will have a higher SOG-to-concession ratio unless that goalkeeper is very good at shot-stopping.

To assess the defenses, the numbers to look at are:

Minutes played (Column 2),
Shots on goal (Column 3) and
Shots per minute (minutes/shot, Column 4)

By these statistics, how was the defending in front of the top keepers in 2015?

Now if you look at the table, look at those three columns, and look at the four top goalkeepers above do you notice anything? Anyone stick out? See it?

Henninger, right?

The SOG-per-minute numbers for Chicago, Seattle, and Kansas City defenders were so similar as to be statistically indistinguishable, less than one standard deviation.

Stat wonk intervention! What the heck is a “standard deviation” when it’s at home, I hear you ask?

Standard deviation, or σ, is a measure of the concentration of a dataset around the dataset average. Any set of real-world data will spread out between the highest and lowest points. “Standard deviation” is a statistic that expresses the distance from the average that data falls. A set with a small standard deviation is clustered around the average; a large σ means that the data is all over the place.

What that means for the three defenses listed above is that they were very close to the average.

The average interval between SOG was 18.6 minutes for Chicago, 19 minutes for Seattle, and 21.9 minutes for FCKC.

That, in turn, suggests that although the raw SOG numbers are very different for each keeper, ranging from only 28 for Kopmeyer to 74 for Barnhart, the differences reflect only the length of time each keeper spent between the sticks rather than the quality of their defenses.

But Henninger? Her defenders let their opponents take a shot at her roughly every 12.6 minutes. That’s terrible. That’s awful.

Only poor Naeher, whose Boston defense that year was a dumpster fire in a landfill inside a toxic waste facility, got shot at more often.

More to the point, it meant that Henninger’s defenders were only about 60% as good at keeping their opponents out of Henninger’s grille as Dalton’s, Kopmeyer’s, and Barnie’s were.

That suggests, in turn, we should consider that Henninger, despite getting shelled, managed to keep a very respectable number of shots out of her goal. And that despite playing for the fifth-place team that ended fourth in the league in goals conceded, Henninger was a better technical keeper than any of the other players who had better defending in front of them.

By this measure, how good were the Portland Thorns defenders in 2015?

At 20 minutes per SOG, they were slightly better than the league average, slightly weaker than Barnhart’s, and far better than Henninger’s.

So her field players’ defending was okay, just about average. Betos wasn’t getting hammered, like Henninger was. We can’t blame her defenders for allowing Betos to face more frequent SOG and more frequent high-quality chances that might help explain Betos’ poor shot-to-conversion ratio.

What does all this mean?

Looking purely at her technical quality in goal, Henninger might have been a better selection for Goalkeeper of the Year than Betos.

Barnhart’s quality, and her role in FCKC’s championship, suggest that Barnhart might have been an even better selection.

Given what the numbers say about Betos’ abilities compared to the players discussed above, Michelle Betos should not have been Goalkeeper of the Year for 2015.

Adrianna Franch and the 2017 Goalkeeper of the Year

Now let’s look at this year’s goalkeepers.

Here’s my raw data table for 2017. The numerical entries for each row below the keepers’ names are the opponents’ SOG for that match.

Note that where a team fielded more than one goalkeeper in the season the SOG number is color-keyed to that player. So North Carolina’s D’Angelo played Matchday 1 to Matchday 6 and Matchday 23; her SOG number is in black. Rowland, who played the bulk of the season, is shown in red.

I’m going to apologize to my fellow stats-geeks; there are errors and missing data all over the NWSL website. I’m not sure I have every single shot on goal for every match. In the end, I decided not to make the perfect the enemy of the good and did what I could to chase down the best numbers I could find.

From that raw data, I cooked everything down to a table similar to the one I worked up for 2015. This one sorts on minutes per SOG, so that puts the keepers in order of the quality of defense in front of each goalkeeper, with the poorest at the top.

First, let’s look at this to see if it tells us whether there was a keeper, or keepers, who benefited from playing behind an exceptionally good defense.

While we’re looking at this, I want to share something from this table I find intriguing.

Boston. What the hell was going on with Boston’s defense?

Look at Prudhomme’s min/SOG value for her five games: a SOG every 28.1 minutes, second best in the league in 2017.

Look at her counterpart, though. Smith? The number one keeper? 13.4 minutes per shot, the very worst in the league.

Did her teammates hate Smith? Or was it that they loved and trusted her so much that they figured they could just laze about texting each other while Smith played like a human neutron star sucking in every shot that approached her? Why the heck would you not play harder for your regular starting keeper?

I have no idea, but it’s a fascinatingly weird piece of data.

Based on minutes-per-SOG the standout defenses for 2017 were:

In front of Williams for Houston (surprising, and makes me think this might be some sort of artifact of her small sample size or the quality of the opponents she faced during that time),
Prudhomme in Boston (see the comment above), and
Rowland for North Carolina.

Everyone between D’Angelo and Kopmeyer played behind defenses that were within a couple points of the league average.

I’d love to dive into the data more deeply (and feel free to do so in the comments), but you have a life and I need to keep this essay down to a manageable size. As such, let’s just jump right to what we did with 2015; list the top four keepers in 2017 by shot-to-concession ratio, what I’m considering the metric of purely technical goalkeeping skill.

Prudhomme for Boston (3 goals from 16 shots, concession rate 18.8%),
Franch for Portland (20 goals from 100 shots, 20%),
Rowland for North Carolina (12 goals from 49 shots, 24.5%), and
Barnhart for FCKC (31 goals from 125 shots, 24.8%)

What would be the case for each of these as Goalkeeper of the Year? Is there any reason to suggest that the selection process picked the wrong keeper?

Prudhomme: At 450 minutes, she played the fewest of any of the keepers on this list outside Dalton. As a fill-in for Smith she played only 5 matches and went 1-1-3, even playing behind one of the toughest defenses in the league at the time.

You can’t really make a case for Prudhomme. Period.

Rowland: She was a critical player during North Carolina’s mid-season run, going 11-4-1 during that run including 8 clean sheets. That said, Rowland lost some important matches including to Portland, twice to the Red Stars, and once to Orlando. With the playoffs added, Rowland went 1-2 against Chicago, 0-1 against Orlando, and 0-2 against Portland, including losing the final. She benefited immensely from one of the best defenses in the league that cut the shots Rowland faced down to almost 2/3rd of the league average.

Rowland was clearly a very high-quality goalkeeper, but in my opinion her case falters compared to Franch on both concession rate, and from having the benefit of tougher defending in front of her. Although not a measure of technical quality, the two keepers are separated tactically by Rowland’s failing to beat Franch twice while Franch was keeping crucial clean sheets.

Barnhart Still one of the best goalkeepers in the league. Behind a slightly better than average defense Barnhart still faced a ton of shots and stopped more than ³⁄₄ of them. But her team couldn’t score and finished seventh, her record an undistinguished 8-9-7.

To make a case for Barnie as better than Franch, you’d pretty much have to insist that despite the statistics Barnhart lifted her team above where they should have finished, or that she was more important to her team’s success than Franch was to Portland, and I don’t think that’s a makeable case.

Franch Champion. 14-5-5 regular season record plus two playoff wins. Lowest concession-to-SOG rate, and a 0.83 goals-against-average that ties the regular season record set by Hope Solo in 2014. Set the new NWSL record of 11 clean sheets.

And, as amazing as this seems, she did all this behind a just-slightly-better-than-the-league-average defense, based on SOG-per-minute.

Before you scream, remember this; while the SOG-per-minute data suggests that while Portland’s field players weren’t the best in the league preventing SOG on Franch, the SOG-to-save ratio shows that Franch was the best in the league stopping those shots.

Put the two together and then you have one of the best defenses in the league, the kind of defense that wins championships.

Put it that way, and I think the Goalkeeper of the Year selection process got it just right this year.

Congratulations, Ms. Franch.

Thanks for all the roses.