Jump to content
Gwydion Stone

Using WS Absinthe Review Module

Recommended Posts

Overall

1. Unacceptable.

2. Barely acceptable, needs major improvement.

3. Acceptable, proper mid-market standard.

4. Above average, enticing, interesting, artisanal.

5. Uncommonly exceptional, distinctive, world-class, elite.

 

I would suggest removing the term "uncommonly" from 5, since "exceptional, distinctive, world-class, and elite" all imply that anyway.

 

I like the descriptors you propose, but if we change, all current reviews will be obsolete...we'd have to go back and fix them all. For me, that's no big deal, but poor Shabba!

Share this post


Link to post
Share on other sites
Bottom line is that the point of a review module is to help people in a general sense make good purchase decisions. So it is most important for the reviews to be articulate, understandable, well written, and clear in their reasoning. Sometimes it's easy to lose sight of that and try to create a "perfect" review system.

 

The point of a review module is not for seasoned reviewers to argue with each other over relatively minor nuances of reviewing, because that offers little help to those who actually need the reviews.

Makes sense that the first question should be, "is this/will this be useful to the reader of the reviews in making their choices." The finer points are interesting for us, but may be of limited use to the review-reader.

 

Eek! Is it time for another poll?!?! :biggrin:

Share this post


Link to post
Share on other sites

I don't mind going back and changing my reviews. There are very few that will be changed from a 4 to a 5 anyway.

 

As others have mentioned though, the most important thing is to make sure any changes will be useful to the consumer. That's where the comments section comes in. Clear, concise, to-the-point. People don't want to read a novel.

 

Just let me know if everyone agrees about the changes. I'll fix them as soon as we come to an agreement.

Share this post


Link to post
Share on other sites
Bottom line is that the point of a review module is to help people in a general sense make good purchase decisions. So it is most important for the reviews to be articulate, understandable, well written, and clear in their reasoning.

 

Agreed.

 

Makes sense that the first question should be, "is this/will this be useful to the reader of the reviews in making their choices." The finer points are interesting for us, but may be of limited use to the review-reader.

 

It makes more sense to me that the first question should be "Is this/will this be useful to the reviewer, in better guiding them to assign the most correct scores possible as they construct their review?" That's who these guidelines are written for. The "review-reader" is not necessarily aware of the guidelines that helped construct the review. If improved guidelines result in better and more accurate reviews, then the consumer is ultimately served.

 

The reason I approached this by rewriting the scoring criteria for scores of 2 through 5 in the category of "Overall", rather than just the criteria for a score of 5, is because, like Brian, I am also concerned about the possibility of overly high scores. It has been my opinion for some time that one of the reasons for some of the overly high scores here is that the scoring criteria in the previous sheets for all categories is derogatory or strongly critical fully through a score of 3. In fact, on the first sheet I used, there was a level of reservation in the criteria for "Color" fully through a score of 4. That only leaves scores of 4 or 5 in most categories to indicate "acceptable" or a compliment. I think that what happens as a result is that many reviewers, when reviewing something they think is "pretty good" default to the levels of scoring that are not derogatory, because the level that should indicate a solid mid-market offering (3), seems too critical. My approach here is to first define criteria for a level of 3 in each category, and then "write down" to scores of 2 and 1, and "write up" to scores of 4 and 5.

Share this post


Link to post
Share on other sites
Makes sense that the first question should be, "is this/will this be useful to the reader of the reviews in making their choices." The finer points are interesting for us, but may be of limited use to the review-reader.

 

It makes more sense to me that the first question should be "Is this/will this be useful to the reviewer, in better guiding them to assign the most correct scores possible as they construct their review?" That's who these guidelines are written for. The "review-reader" is not necessarily aware of the guidelines that helped construct the review. If improved guidelines result in better and more accurate reviews, the consumer is ultimately served.

Good point. I might have been thinking along those lines, though I can't quite recall! Substitute "limited use" in my sentence with "indirect use at best." Yeah, that's the ticket!

Guidelines->review writer->review reader. :cheers:

Share this post


Link to post
Share on other sites
Upon first reading, I like those proposed changes. Ruminations TK.

 

What is TK? Sorry, I'm an old guy!

 

Bottom line is that the point of a review module is to help people in a general sense make good purchase decisions. Sometimes it's easy to lose sight of that and try to create a "perfect" review system.

 

Not trying to make it "perfect", just improved.

 

Overall

1. Unacceptable.

2. Barely acceptable, needs major improvement.

3. Acceptable, proper mid-market standard.

4. Above average, enticing, interesting, artisanal.

5. Uncommonly exceptional, distinctive, world-class, elite.

 

I would suggest removing the term "uncommonly" from 5, since "exceptional, distinctive, world-class, and elite" all imply that anyway.

 

I agree about the implication, but some may not get it. I thought the same when I wrote it. I was just trying to "tighten" the criteria as much as possible so that the score of 5 in "Overall" would be used with extreme discretion.

Share this post


Link to post
Share on other sites
Upon first reading, I like those proposed changes. Ruminations TK.

 

What is TK? Sorry, I'm an old guy!

"To come." It's a journalism term, I believe. I first saw it as the title of a Breeders album, "Title TK." It may, in fact, only be used with 'title' in front of it, but whutevs.

Share this post


Link to post
Share on other sites

I'm sorry if I'm echoing anyone else's post, as I admittedly did not have time to read every single thought.

 

Regarding the newbie intimidation factor, the way I see it, it has more to do with the number of points involved for each criteria, not necessarily the total number of points. If you look at Fee Verte, it must be terrifying for a first-timer. "I have THIRTY points to work with for aroma? Why are there more points for aroma than taste? I have to rate the colour twice? Help me!" The WS system is very friendly to newbs, but at the expense of detailed scoring. Also, there is still one area of confusion with the two Overall ratings; one representing the "overall average of scores" and the other an "overall impression" so to speak. Personally, I like having as much info as possible, but a lot of people are going to want a more firm bottom line. A half-point system would not be nearly as intimidating as FV; aroma 19/30 or 3 1/2 stars? The latter sounds pretty starightforward and simple to grasp. Transitioning to this system would be a good balance of accessibility and accuracy in my view.

 

On a scale of 1 to 10/100-point system I do think a 10 or 100 rating can be referenced as an actual point, because your score can be more precise with more points to work with. 1 to 5? It is just too difficult not to make it a small range as opposed to a point, or else we should only be seeing a 5 once in a blue moon, especially considering Brian's welcome reminder about absinthe still improving as a whole. Absomphe's example was a very good illustration of this:

 

 

So (to use my favorite example) I gave Pacifique a 5 in every category but color, where I lowered it to a 4 because I thought it was just a wee bit on the pale side, and by your reckoning, I'm required to give it a 4 overall?

 

Given that it averaged out to a 4.8 in all the other categories, I think this is too rigid, and doesn't accurately reflect the overall quality of the product...a 4 is tantamount to an 80%, no matter how you slice it, and there is no way that I could justify giving Pacifique that kind of score on the whole...I'd sooner reinstate the original 5 I awarded it for color.

 

The big problem for me (particularly with the overall category) is that there is a tremendous disparity between a rating of 4 and a rating of 5. If I were able to give it a 4.8 overall, that would make far more sense to me. Perhaps the overall category should be broken down further to allow it to serve as a reflector, or average of all the other categories.

 

Thinking of WS review guide 10 years down the line, I'm sure we'll be glad the system was changed now rather than later. While it is unfortunate for the other reviews, I dont think they will be viewed as less accurate, just not as "specific".

 

Brian, I realize this is a very tough decision, and there will be someone unhappy with it no matter what you do. But you've read the arguments and have been mulling over this for months judging by the thread date. There is some passionate conversation here. Truly, this is what working together for a greater good is all about. That said, whatever is decided, we can at least say an improvement was made thanks to a collective effort. :cheers:

 

 

Funny how no one else on the review committee is adding any thoughts to this discussion. *AHEM*

 

Like I said, you're always on the front lines and on top of things, Brian. :thumbup: Maybe we could drop them a friendly reminder. :)

Share this post


Link to post
Share on other sites

And that's the major issue. This isn't MY decision at all. I just review the reviews. I don't have it within my power to just arbitrarily change the review system on a whim. It's not my place to do so, since the system was developed by Hiram with input from the entire advisory board and more.

 

An even bigger issue is that we only have a couple of people who voiced issues over the scoring system. We have 56 people who have written more than one review. 17 have written 10 or more. Of those 17, it looks like only a handful have weighed in on this topic. Of those, I haven't found one that is absolutely convinced that the changes need to be made. All of them seem open to the change, but none have called for the 'need' for change.

 

I do however, understand that the reason there is 17 people who've written 10 or more, as opposed to 117, could be due to the limitations. Can we have a legitimate show of hands to help us figure out how many people who haven't yet done many reviews would do so if the system were to make the aforementioned changes?

 

I like FPB's new classifications, including the term 'uncommonly', so I'm all for changing it to that.

 

I'm also open to discussion about whether we need to add half-stars to the categories. My only concern with that will be the description of those halves.

 

Now, let's here from the rest of the board as well as:

 

Buddha

Doc Love

Scuto

Accountant

pt447

Khiddy

TallyHo

Tits

T73

Abs

Marlow

Peridot

Anthonw

GreenBaron

PeterL

 

Sorry, I didn't make the conversions for people who have changed their names on the forums.

Share this post


Link to post
Share on other sites

I am not fond of any maths, yet if most of us vote for adding half stars or whatever it will be called, I will accept. I have just added 4 new reviews: (Brevans A. O. Spare, Wormwood Blanche, La Charlotte and Montmartre) using the old system and looking at the scores they seem right to me, exactly that way I see these products. For example both Brevans and WB should be given 4.5 to be more concise, yet the way I feel about them, Brevans is 4.4, so almost 4.5, but not 4.5, whilst WB is 4.2 and it describes it perfectly.

 

As a teacher when my student's year score is 4.3, I will give them 4.5, but students are not absinthes, so I am happy with the old system.

 

Back to reviewing!

Share this post


Link to post
Share on other sites

For the benefit of everyone here, I'm going to disregard Brian's list above, and, since I've done 9 reviews, include myself. Having said that, I agree with Ben completely, and feel rather strongly that the current system is too limiting. The lack of half-stars causes my otherwise limited brain power to completely choke up sometimes, so, hence, not as eager to review stuff as I once was. Just my two pfennig.

Share this post


Link to post
Share on other sites

I didn't want to make it seem like I'm limiting feedback to those people, it's just I wanted to make sure we heard from the most active reviewers. My appologies if it came off that way.

 

So can we get some input on what people's descriptions for the .5 star increments would be?

 

I'd love to take a stab at it myself, but it will have to be after Tales.

 

By the way, Gwydion, is it even possible to add half-stars if we all decide to go that route?

Share this post


Link to post
Share on other sites

I don't feel up to descriptions for half-star ratings at this very moment, but something struck me--When writing up my reviews, if I were to try and put a number on it, I felt maybe 20%-25% of my individual ratings would have benefited from having the half-star option. This leads me to a question that could be more trouble than it's worth: would certain individual categories benefit from having a half-star available more than others? Or just having half-stars for the user-rated "Overall" category (not the calculated one)?

 

Again, I could be making things overly complicated. Do tell me if it's true! :)

Share this post


Link to post
Share on other sites

I'm going to look in on this and get back to y'all. I just posted reviews tonight for WW and MOL and didn't feel limited in any way, but I'll go back and read what everyone else feels is limiting. Gotta leave the computer for a bit, though.

Share this post


Link to post
Share on other sites

I think you either get the half star, or you don't. To make it some yes and some no, well that's just piling on now. And, I would say that if you don't feel you need a half star to be precise enough in your judgements, well don't use 'em then.

Share this post


Link to post
Share on other sites
The argument now isn't about what it says. The discussion is about what it SHOULD say.

I've been thinking about it all day and this is tougher for me than an absinthe definition!

 

Just what you've seen from me in this one category was tough enough to concoct, as well. Try rewriting all six categories, which I've done. I've been at it, with almost daily review and tweaking since Brian agreed to consider some changes in this post.

 

It makes more sense to me that the first question should be "Is this/will this be useful to the reviewer, in better guiding them to assign the most correct scores possible as they construct their review?" That's who these guidelines are written for. The "review-reader" is not necessarily aware of the guidelines that helped construct the review. If improved guidelines result in better and more accurate reviews, then the consumer is ultimately served.

 

The reason I approached this by rewriting the scoring criteria for scores of 2 through 5 in the category of "Overall", rather than just the criteria for a score of 5, is because, like Brian, I am also concerned about the possibility of overly high scores. It has been my opinion for some time that one of the reasons for some of the overly high scores here is that the scoring criteria in the previous sheets for all categories is derogatory or strongly critical fully through a score of 3. In fact, on the first sheet I used, there was a level of reservation in the criteria for "Color" fully through a score of 4. That only leaves scores of 4 or 5 in most categories to indicate "acceptable" or a compliment. I think that what happens as a result is that many reviewers, when reviewing something they think is "pretty good" default to the levels of scoring that are not derogatory, because the level that should indicate a solid mid-market offering (3), seems too critical. My approach here is to first define criteria for a level of 3 in each category, and then "write down" to scores of 2 and 1, and "write up" to scores of 4 and 5.

 

The point I would like to make here is that not just the top level (5) of each category must be well defined, but that all levels (1-5) of each category must be very well defined. The descriptions for scoring need to be well defined, and speak to what should be expected at each level, in a matter of fact way.

 

When I rewrote the descriptors for the category of "Overall", I started with this standard, which I have applied to all categories;

 

Level 1. A completely unacceptable standard.

 

Level 2. A barely acceptable standard. Highly flawed.

 

Level 3. An acceptable standard. A mid-market offering, however not without some flaws.

 

Level 4. An elevated offering. Clearly above average.

 

Level 5. The very finest. Flawless or nearly flawless.

 

Of course from there, it becomes a simple task to write the descriptors. :twitchsmile:

 

So that is the logic I applied to this, and to the other categories.

 

Thinking of WS review guide 10 years down the line, I'm sure we'll be glad the system was changed now rather than later. While it is unfortunate for the other reviews, I dont think they will be viewed as less accurate, just not as "specific".

 

I have never been for adding levels (or points or half-points) to this system. I would much rather see the current system tweaked to make it better. I think a 5 point system is most useable by a wide spectrum of people, and that is what I think this site is all about. Also, my man J.L. said it best.

 

Joe Lagate May 14 2009, 09:52 AM

I've also wanted a .5 from time-to-time, myself. If it was difficult deciding between a 3 or a 4, a 4 or a 5, a .5 would have been a near-godsend.

 

But I'm not advocating the change, due in large because all the reviews to this point have been made without it. I'd hate to skew and screw-up all the work that's already been done.

 

I think we should take best efforts to maximize the effectiveness of the current 5 point system first before we go turning this whole thing on it's head.

 

And that's the major issue. This isn't MY decision at all. I just review the reviews. I don't have it within my power to just arbitrarily change the review system on a whim. It's not my place to do so, since the system was developed by Hiram with input from the entire advisory board and more.

 

I like FPB's new classifications, including the term 'uncommonly', so I'm all for changing it to that.

 

I'm also open to discussion about whether we need to add half-stars to the categories. My only concern with that will be the description of those halves.

 

Well, you've heard my opinion. If half-stars prevail, someone else can write it!

 

So can we get some input on what people's descriptions for the .5 star increments would be?

 

I'd love to take a stab at it myself, but it will have to be after Tales.

 

And I only responded now because of this post. I know Tales is the raison d'etre for many here right now. Also, I think this should be carefully considered, since it is so fundamental to the mission of the WS. We should take our time and proceed with all due care.

Share this post


Link to post
Share on other sites
IIf you look at Fee Verte, it must be terrifying for a first-timer. "I have THIRTY points to work with for aroma? Why are there more points for aroma than taste? I have to rate the colour twice? Help me!"

Not me. From the very beginning I liked having some room to breathe.

 

Overall, I'm fine with the system as is, but I'm not married to it. I don't mind changing the descriptors in the way that fingerpickinblue suggests. I'd rather the system were set up 1-10 from the beginning instead of 1-5, but I don't think it's smart to turn it on its head now.

 

The biggest thing that I do like is that I can go back and edit. I don't recall offhand how that's looked upon here, but my perfect review system would be a living one where everyone is not only able to but is encouraged to adjust their thoughts over time to ensure they are as accurate as possible.

Share this post


Link to post
Share on other sites
my perfect review system would be a living one where everyone is not only able to but is encouraged to adjust their thoughts over time to ensure they are as accurate as possible.

I completely agree. I highly encourage people to go back and update reviews as part of the process. Batches change, palates change, bottles age, etc.

 

One thing I recommend though is to reference the original review and explain why the changes were made.

Share this post


Link to post
Share on other sites

One of the main reasons I don't post reviews is that I've felt that once posted, they were pretty much cast in stone or not able to edit. I've learned better since.

 

Being able to go back and do the changes will make a world of difference. My opinions of different drinks are affected so much by my immediate surroundings that what I write is not close after rereading it a few days later.

 

After Tales I'll be adding to your review section. Although, my palate is still in it's infancy compared to some.

 

Have a grand time at Tales Brian. :cheers:

Share this post


Link to post
Share on other sites
IIf you look at Fee Verte, it must be terrifying for a first-timer. "I have THIRTY points to work with for aroma? Why are there more points for aroma than taste? I have to rate the colour twice? Help me!"

Not me. From the very beginning I liked having some room to breathe.

 

Overall, I'm fine with the system as is, but I'm not married to it. I don't mind changing the descriptors in the way that fingerpickinblue suggests. I'd rather the system were set up 1-10 from the beginning instead of 1-5, but I don't think it's smart to turn it on its head now.

 

The biggest thing that I do like is that I can go back and edit. I don't recall offhand how that's looked upon here, but my perfect review system would be a living one where everyone is not only able to but is encouraged to adjust their thoughts over time to ensure they are as accurate as possible.

 

 

In a nutshell, what he said.

 

I'm in favor of an optional half star system, as Speedle suggested.

 

 

Btw, Peridot, I loved your "oceanic" descriptor for Walton Waters. I didn't pick up the seaside saltiness for the first few glasses, but it's become a bit more noticeable with some breathing time. Not exactly the Islay of absinthes, but saltier than most. :)

Share this post


Link to post
Share on other sites
The biggest thing that I do like is that I can go back and edit. I don't recall offhand how that's looked upon here, but my perfect review system would be a living one where everyone is not only able to but is encouraged to adjust their thoughts over time to ensure they are as accurate as possible.

 

I completely agree. I highly encourage people to go back and update reviews as part of the process. Batches change, palates change, bottles age, etc.

 

One thing I recommend though is to reference the original review and explain why the changes were made.

 

This past weekend, I just updated my first 7 reviews posted on this site. I plan to get to all the others that are more than 90 days old this month. In each, I included a new paragraph of my current observations, and explanations of any changes, or not. In the future, I intend to replace that paragraph with one dated for my most recent evaluation. Of the 7, only 2 required score adjustments, to compensate for my then newbiness. I'm sleeping better at night already. :dry:

 

Btw, Peridot, I loved your "oceanic" descriptor for Walton Waters. I didn't pick up the seaside saltiness for the first few glasses, but it's become a bit more noticeable with some breathing time. Not exactly the Islay of absinthes, but saltier than most. :)

 

Damn you, Abs! I wasn't going to have an absinthe tonight. :dev-cheers:

Share this post


Link to post
Share on other sites

FPB, your post #136 makes a lot of sense to me; refinement, though it sometimes proves more difficult than outright change, appeals to me from reading it.

 

Again, I could be making things overly complicated. Do tell me if it's true! :)

 

I think you're just a hybrid kind of guy! ;)

Half sharkalligator half man. :shifty:

Share this post


Link to post
Share on other sites

Hey LDB, I just purchased a bottle of Le Clandestine myself today, so I hope to add my own review there next to yours by the end of the month. Congrats on getting your first one added to the site!

 

Oh, and comparing the flavor to "the caress of a cruel women" as you did only has me quivering in anticipation :cheers:

Share this post


Link to post
Share on other sites

Haha, trying to put style in my review, I'm not an expert in tasting or in alcool (outside of beer) but like to write so I try to compensate actual tasting talent by creativity :blush: . On this particular part, I had trouble finding my words in English to describe the fact that it "enrobe" your mouth but is a bit unpleasant and likeable at the same time.

 

Anyway, about the module : the only problem I found (which is part due to the smallness of the PC I use) was that I found the text box to show what you're writing in a really little character size. Don't know if anybody found that to, but that's my 2 cents!

Edited by Le Diable Bleu

Share this post


Link to post
Share on other sites

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now

×