Thursday, February 26, 2009

Judging the Judges

Since I started this blog in November, I've wobbled quite a bit on what topics I want to cover. I've tried to move away from posts I think are too "press-releasey" while still reporting on Indiana wine and wineries. Strangely enough, one thing I haven't been interested in covering is that of wine competitions.

Anyone who has been to a winery has seen them. Look up, because the bottles are usually placed on a high shelf, wearing three or four necklace-like medals the wine managed to win at some competition or another. What regulars at wineries tend to notice now are the wines that have no medals, sad and stooped like the child who received no valentines.

Those who wish to immerse themselves in local wine culture quickly learn that just about any winery can win a medal for a wine they made. Those who immerse themselves in local wine culture learn pretty soon after that that medals are not really indicative of quality.

Now, a study in the Journal of Wine Economics confirms what a lot of people already knew. As the abstract puts it:

Wine judge performance at a major wine competition has been analyzed from 2005 to 2008 using replicate samples. Each panel of four expert judges received a flight of 30 wines embedded with triplicate samples poured from the same bottle. Between 65 and 70 judges were tested each year. About 10 percent of the judges were able to replicate their score within a single medal group. Another 10 percent, on occasion, scored the same wine Bronze to Gold. Judges tend to be more consistent in what they don’t like than what they do. An analysis of variance covering every panel over the study period indicates only about half of the panels presented awards based solely on wine quality.

The article has lots more goodies and stats that show wine judges are all over the map in grading wines, which explains why just about any wine can win an award.

I understand medals are valuable PR tools. I would submit, however, they are only paid any mind by beginners. I think just about every wine fan can remember the first time they bought a wine because of the award it had "earned." This is usually followed shortly thereafter by said wine fan drinking said wine and realizing the medal should be melted down and sold for scrap.

Ignore those medals on the shelf. Try everything you can sip the next time you visit a winery. Expand your horizons, and you will be surprised at what you like.

Find more about this topic at the wonderful "sister" site Michigan Grapevine, as well an article in Wines and Vines.

PS: Think I have any chance of getting media passes to the Indy Wine Competition?

1 comment:

Anonymous said...

I also do feel that wine competitions and the awards given offer very limited help to a consumer seeking a wine they will enjoy. One key to this dilemma is that these competitions usually have just two purposes: (1) to make money for the organizing group and (2) to help market wine.

I have often compared choosing a wine for someone else to telling them what their favorite color is.

I have tasted gold medal wines that were well worth the medal they wore and I have tasted some that were just OK. Some wine enthusiasts submit that good wine is an absolute, we should all be able to agree on good-better-best.

But as with color, its fully and completely subjective to the person doing the tasting. And even that subjectiveness has many variables based on what the person has eaten, how they feel that day, etc.

I appreciate comments, suggestions, a little direction on a wine I might like (like this blog), but telling me that a particular wine is the best wine, period, seems arrogant.

rick