Friday, July 16, 2010

Why your product doesn't suck

This year, perhaps more than any other that I've been a part of, I've seen a great many publisher comments and questions. They're always polite, always good natured, and always reasonable. As someone who is sent a great many products at publisher expense every year, I can tell you they are always understandable as well.

Still, they usually come down to the same thing: "Why wasn't my product nominated." Publishers go through a lot of hassle to enter the awards, and with that hassle comes a not inconsiderable expense as well. This makes the question fairly natural.

So, here's my general answer:

If my little spreadsheet (see how we've come full circle? That's talent baby.) is to be believed, I received 301 entries for the awards this year. There are 20 award categories (now - the number has expanded as the awards have expended), each with six slots for recognition (counting Honorable Mentions); except for Product of the Year, which has 12. This year we also added five more slots to recognize great games, but we'll come back to that.

That's 126 slots available (131 if you'd rather count the Judges' Awards, which haven't been announced). By my count that means 60% - give or take a tad - will not see an award no matter how good they are. But of course there's far more to it.

Each category has 5 slots to fill with nominations. Your product isn't competing for 126 openings. You're competing 20 times for 5.

That's less than a 2% chance that you'll be recognized for anything.

So, if you don't get a nomination, does it mean that we didn't like your product, or that we think you need to change anything? No, it just means that 2% is a really small number, and there's not room for everything, no matter how much we like it.

I'm going to use Penny for My Thoughts as an example. I choose it because I'm familiar with the publisher and the author, and based on dealing with them in the past I trust them not to take my words the wrong way or find offense. The reason I have this trust is because it's come up with them in the past. Not that they questioned a nomination, but because my love of Don't Rest Your Head helped cost Spirit of the Century a much deserved nomination back in 2007. I've learned a lot since that first year, and it's a mistake I wouldn't make again, but their response to my admission was far more understanding than I deserved :)

So, Penny for My Thoughts was one of my favorite entries this year. I know it is because of those Judges' Awards. Each judge had to pick one game to give it to, and found myself torn between three, one of which was Penny. Its a fantastic game. It's well-written, fun, interesting, and different. Exactly the sort of thing I look for first in a nomination, and I couldn't recommend it more. Yet it didn't get a single nomination.

So, why is that?

Well, it's because it never quite seemed to make it. There's no doubting it's a good game, and worthy of recognition, but it always seemed to fall just short. Something always seemed to just edge it out. When it failed to make the final cut in Best Writing, the place I felt it best belonged, it became clear that the game, no matter how much I wanted it to, wasn't going to quite make it.

The fact disappoints me quite a lot, but I don't really think its the wrong choice. Finding a place to put it on the list means knocking out another game that deserves its place just as much.

Judges decide on nominations (at least in my experience) by mutual assent, not through arbitrary voting. If we don't all believe it needs to be on the list, it isn't, and by doing this we've created the most diverse list of nominations that I've seen in my time with the awards.

126 slots, 96 unique nominations. That's amazing to me, and I think it's a real testament to the work of the judges I worked with this year and the system we used. It's easy for one favorite product to completely dominate the awards, or for a handful to see nomination over and over again in a dozen categories. It's harder, and in my opinion better, for the judges to look at every product and honestly recognized their individual strengths in each category, rather than see a single much-loved product and nominate it over and over again until something sticks.

The downside of this, of course, is than many products that were really exceptional only received one - or a small handful - of nominations. That doesn't mean that the dominating products from previous years were better with their larger number of nominations, but is rather a reflection of the judges' dedication to putting the right product in the right place, and of the exceptional number of truly outstanding products that we received this year.

I want to thank everyone who gave me the opportunity to do this again this year, it's been quite an honor, and it's something I enjoy more than I can say. I hope this does something to explain why good games don't make the list. Choosing these games - making sure the right games make it into the right place - is harder than it seems. It's disappointing to see good games go without the recognition they deserve, as much to us as to the fans of the games that don't make it. As often as not, we're fans ourselves.

I think great strides are being made toward spreading the love around, and have absolute confidence that the awards' new leadership and new judges will be able to make use of all these new tools we've begun using to continue that trend. The Judges' Awards themselves are a great representation of this; a chance for each judge to pick just one thing that didn't quite make it, that they wish had.

I can't wait until they're announced so I can talk about them more.

----

Just to head off the questions, no - Penny wasn't my Judges' Award. I'm not allowed to say what that is until the ceremony. For that I was forced into a tie breaker, and I chose the game I had the most fun with in actual play. Sadly I didn't get a chance to playtest Penny, but the guys who taught me [my Judges' Pick] made it one of the best times I've had playing a game the entire year. Maybe it was the group, maybe it was the game facilitating the fun, but I had a blast, and I had to put my award where I found the fun first-hand.

6 comments:

Fred Hicks (Evil Hat Productions) said...

See, it's posts like this which are why you are always my go-to guy on the judge ballot. I'm a transparency addict, what can I say? :)

Thanks for insight here. Honestly we had no idea Penny had as many close calls as you suggest, here -- we knew we were headed into a year with stiff competition in all of its eligible categories.

The Hartleys said...

As always, Jeremy, thank you for your insight and transparency. I think this kind of information really goes a long ways towards not only assuaging hurt feelings, but also towards keeping the public (and industry) image of the ENnies as high as it is. Explanations breed understanding, opaqueness breeds assumptions.

Hurray for communication!

~jess hartley (One Geek To Another)

Steven D. Russell said...

Wow!

That really makes me feel good for Heroes of the Jade Oath and To Kill or Not to Kill,

It also makes me feel better for last years failure of A Witch's Choice which preceded 2KoN2K in the same series of adventures.

Thanks so much for a post like this.

Steve Russell
Rite Publishing

Jeramy Ware said...

Thanks everyone. I'm glad someone found some value in it.

I pretty much switched all of my ENnies commentary to Twitter since it seemed like I was reaching a larger audience there, but maybe there's some value in posts larger than 140 characters after all :)

Cthulhu's Librarian said...

I'm reading your last few posts about a week late, but I felt that they were still worth commenting on.

I was an ENnies judges for 3 terms, and worked on the board/admin team one year. When you came aboard (my year as an admin), I was hesitant about your ideas for opening the process up. I was very concerned about angering publishers, and didn't quite know what or how to open things up without doing something wrong. Looking back on the past few years, I'm very impressed with the strides the awards have made since then, and the transparency you've brought has only helped.

I wish that we had the tools, systems, and communication in place then that the judges are now using. I don't regret any of the choices made (and definitely do not think we made any bad choices), but getting to those choices and the final back and forth rounds was stressful and felt like we were going nowhere fast at times.

This post in particular illustrates the thing I hated most after the nominations were announced each year-the hurt and angry publishers. The lowest point I ever had as a judge came when I stopped at a booth to congratulate a publisher on their nominations, and the author of one of the entered books that didn't get a nomination was there. Instead of shaking the publishers hand and congratulating them, I ended up being shouted at by the author, accused of favoritism and told that his book was the most important rpg book published that year and how dare we skip it? How do you respond to that? I attempted to say that not everything can be nominated, but just got cursed at as he stormed off, leaving me feeling like crap as I had truly enjoyed the book (and still use it today), and would have happily bought another copy at the booth to have him sign.

I wish I had the numbers that you talk about here, and this post, to say "Look, here are the odds of a nomination, before taking anything else into account." We always tried to explain that there's no way to nominate everything, and obviously people are going to be disappointed if they aren't nominated. Not that it would have worked in this particular case, I fear, but being able to step back and say "Wait, here's how it works" in a clear manner would have helped to smooth over a lot of rough patches.

I guess this is all a long winded way of saying thanks, Jeramy, for what you've brought to the awards.

Jeramy Ware said...

Thanks, I really appreciate that more than you know. In fairness, this blog makes putting my thoughts out there a lot easier. Any time you respond to a publisher directly I worry that it comes off as pandering or disingenuous. Here I can put it out for everyone to see and hopefully see the truth of it.