Jump to content

 

Photo

Using WS Absinthe Review Module


  • Please log in to reply
190 replies to this topic

#61 Brian Robinson

Brian Robinson

    Shabba

  • Advisory Board
  • PipPipPipPipPipPipPipPipPip
  • 12,818 posts

Posted 13 May 2009 - 10:13 AM

Close those parentheses ya Sixer lookin' bastid. ;)

Regarding FV vs WS, I've had about equal amounts of people comment on both sides. I like the FV system too, but I find it entirely too open ended, for the reasons mentioned above.
Answers to common newcomer questions.

List of WS articles from across the web.


Help other absintheurs and newcomers by submitting a review. Click here to go to the main review page to submit your entry.

Rantings of a DC Gourmand.
WS on the Mutineer Blog!

#62 precenphix

precenphix

    Dreamer / Noise-maker

  • Member
  • PipPipPipPipPipPipPip
  • 1,984 posts

Posted 13 May 2009 - 10:28 AM

7 point system!

I see it both ways, actually. The 5 star system keeps things simple but perhaps a bit too simple. The 10 system might be too open. This is all subjective. I just somehow find it hard to be completely objective with any rating system. I'll stop speaking in truisms here until I have an actual suggestion to contribute...

Thinking...
Those with knowledge easily sense the truth of things. Those with egos built up on rumor and fancy, tend to maintain a hard line. - Tatan (Evan Camomile)

#63 Phoenix

Phoenix

    Advansed Member

  • Member
  • PipPipPipPipPipPipPip
  • 1,654 posts

Posted 13 May 2009 - 01:45 PM

Only if you think it needs a 4. Just look back through the reviews of other high rated absinthes and you'll find that many were given a 5 in the overall category.

Some could have been mistakenly done because the reviewer might not have understood the definition of a 5. But more often, those 5s were given before the criteria was changed to make it harder to score an absinthe an overall 5. I notice I still have a couple I haven't changed to reflect the new definition. I'll be doing that today.

If the board is concerned about an un-biased review database, I wouldn't think it's in their best interest to change the criteria and make scoring tougher after there own brand(s) have been released and reviewed using the old system and then keeping them side by side with brands reviewed off of a new, tougher system. Especially when there's nothing saying that such things are going on. That could be seen as dishonest/biased.

In the end, I think if someone is going to make a connection and say the review database is biased towards members, there's not much that could be done about it (I suspect it's going to happen no matter what happens). The members here that have gone commercial make fine products. They're passionate about the drink, they're not just someone who's trying to cash in on the newest thing. It shouldn't come as a surprise that their products are outstanding.

I also understand that we are dealing with a 5 point system, and that a prefect "5" score, at least compared to 100 point wine scoring systems, is the equivalent of a score of 90 - 100.

Wouldn't it be 80-100? What percentage of those "approximately 205,275" brands scored 80+?
"He's a politician. It's like being a hooker. You can't be one unless you can pretend to like people while you're f***ing them."

#64 Brian Robinson

Brian Robinson

    Shabba

  • Advisory Board
  • PipPipPipPipPipPipPipPipPip
  • 12,818 posts

Posted 13 May 2009 - 02:48 PM

I wouldn't think it's in their best interest to change the criteria and make scoring tougher after there own brand(s) have been released and reviewed using the old system and then keeping them side by side with brands reviewed off of a new, tougher system. Especially when there's nothing saying that such things are going on. That could be seen as dishonest/biased.

Well if you put it that way...

The criteria was changed before many of the WS brands came out, at least according to my records. The ones that I had previously given a 5 overall were not WS brands. They included Coquette, L'Artisanale, Bell Amie and a few others I don't remember. I think the only one that had been out at that point was Swiss Marteau, which didn't score a 5 anyway.

And there were several mentions that the criteria had changed, as far as I know.

As I mentioned before:

If you truly think that a brand cannot be improved upon, then 5 it is. It's your opinion, and it's perfectly valid and will be respected.


Answers to common newcomer questions.

List of WS articles from across the web.


Help other absintheurs and newcomers by submitting a review. Click here to go to the main review page to submit your entry.

Rantings of a DC Gourmand.
WS on the Mutineer Blog!

#65 Phoenix

Phoenix

    Advansed Member

  • Member
  • PipPipPipPipPipPipPip
  • 1,654 posts

Posted 13 May 2009 - 03:25 PM

My mistake. I figured the change was recent as I try to keep up with the absinthe world as best I can and didn't remember ever reading about it. I still stand behind the second part of what I said (which would include those same brand(s)).
"He's a politician. It's like being a hooker. You can't be one unless you can pretend to like people while you're f***ing them."

#66 Brian Robinson

Brian Robinson

    Shabba

  • Advisory Board
  • PipPipPipPipPipPipPipPipPip
  • 12,818 posts

Posted 13 May 2009 - 03:53 PM

There's no doubt about that.
Answers to common newcomer questions.

List of WS articles from across the web.


Help other absintheurs and newcomers by submitting a review. Click here to go to the main review page to submit your entry.

Rantings of a DC Gourmand.
WS on the Mutineer Blog!

#67 scuto

scuto

    Taste sensation

  • Member
  • PipPipPipPipPipPipPip
  • 1,908 posts

Posted 13 May 2009 - 04:49 PM

In the end, I think if someone is going to make a connection and say the review database is biased towards members, there's not much that could be done about it (I suspect it's going to happen no matter what happens).

Exactly--we can strive to keep the review section in as tip-top shape as we can, which includes having these sorts of discussions. If we keep our side of the street clean--which of course is an ongoing process--then if they perceive such things it's their issue because we know we've done the best we're able.

Precenphix thought he had cornered the market on truisms! :devil:
"The Saint when he is drinking/Is also pleasing God/As if he were praying and singing." - Angelus Silesius, quoted in Simmel's On Individuality and Social Forms, p.391. (Yay for classical sociology!)

"Full bottle in front of me/Time to roll up my sleeves and get to work/And after many glasses of work/I get paid in the brain" - They Might Be Giants "Your Own Worst Enemy."

"I've an absinthe factory in my head" (jcbphd, 2009). [Liberties taken. -ed.]

#68 precenphix

precenphix

    Dreamer / Noise-maker

  • Member
  • PipPipPipPipPipPipPip
  • 1,984 posts

Posted 13 May 2009 - 05:35 PM

Just trying to keep the peace and put things in perspective...and failing miserably on the later. :blush: I tend to blab a bit.
Those with knowledge easily sense the truth of things. Those with egos built up on rumor and fancy, tend to maintain a hard line. - Tatan (Evan Camomile)

#69 scuto

scuto

    Taste sensation

  • Member
  • PipPipPipPipPipPipPip
  • 1,908 posts

Posted 13 May 2009 - 06:18 PM

Oh! Sorry, I didn't mean it that way--I thought I was overusing truisms, so I perceived myself wresting that mantle from your paws. I mean headphones.

On topic--I've thought adding a .5 to the numerical rating system here, though that could make things more complicated. And I think it's already been covered. :blush:
"The Saint when he is drinking/Is also pleasing God/As if he were praying and singing." - Angelus Silesius, quoted in Simmel's On Individuality and Social Forms, p.391. (Yay for classical sociology!)

"Full bottle in front of me/Time to roll up my sleeves and get to work/And after many glasses of work/I get paid in the brain" - They Might Be Giants "Your Own Worst Enemy."

"I've an absinthe factory in my head" (jcbphd, 2009). [Liberties taken. -ed.]

#70 fingerpickinblue

fingerpickinblue

    Hallucinations Sold Separately

  • Content Team
  • PipPipPipPipPipPipPipPipPip
  • 3,364 posts

Posted 13 May 2009 - 07:38 PM

I also understand that we are dealing with a 5 point system, and that a prefect "5" score, at least compared to 100 point wine scoring systems, is the equivalent of a score of 90 - 100.

Wouldn't it be 80-100? What percentage of those "approximately 205,275" brands scored 80+?


No, it actually is 90 - 100 because most of the 100 point wine scoring systems are based on Robert Parker's system which is a system of 50 objective points stacked on top of a baseline of 50 points. A wine that received 0 points in each of Parker's four defined categories would have a score of 50 points. In other words, if a wine attains 80% of the objective points allowed by the system, that is a 90 point score.

I was happy when I discovered, the other day, that some descriptions of scoring within the categories on the WS evaluation sheet had been changed. I was especially happy that the description for a score of "5" in "Overall" had been better defined, since the former description "I'll have another, thank you!", was no objective criteria, and essentially useless.

Soon, I had planned to revisit each of my reviews, in part to make decisions about which ones were deserving of a score of "5" overall. Now I'm not sure what to do. It is my opinion that a "5" rating overall should be allowable if the reviewer judges the product to be in the very top echelon, not just when he judges it to be impossible to improve. I'm not sure how many people here are capable of making that judgement, certainly not me.

Initially, I saw the new description as very useful, but limiting. There is a difference between "Perfect. No improvements needed." and Perfect. No improvements possible.
blind man see her, dumb man call her name - Ed Bell

#71 Phoenix

Phoenix

    Advansed Member

  • Member
  • PipPipPipPipPipPipPip
  • 1,654 posts

Posted 13 May 2009 - 08:28 PM

Thanks for the reply on the wine rating thing. I had no idea that the lowest score a wine could get was a 50.

It is my opinion that a "5" rating overall should be allowable if the reviewer judges the product to be in the very top echelon, not just when he judges it to be impossible to improve.

I'd have to agree.
"He's a politician. It's like being a hooker. You can't be one unless you can pretend to like people while you're f***ing them."

#72 dakini_painter

dakini_painter

    Delaware Phoenix Distillery

  • Member
  • PipPipPipPipPipPipPipPipPip
  • 3,825 posts

Posted 14 May 2009 - 03:19 AM

I'd think that in a five star rating system you certainly should be able to use the fifth star, otherwise you just have a four star rating system. :fork:

Restaurants and hotels all have five star rating systems, and there are certainly establishments that receive all five stars.

"Good is the only investment that never fails." Thoreau.
"Don't you push me baby cause I'm holdin' low / and you know I'm only in it for the gold" Grateful Dead
Distiller and Proprietor, Delaware Phoenix Distillery, Walton, NY. DSP-NY-15019. www.delawarephoenix.com


#73 Brian Robinson

Brian Robinson

    Shabba

  • Advisory Board
  • PipPipPipPipPipPipPipPipPip
  • 12,818 posts

Posted 14 May 2009 - 03:29 AM

since the former description "I'll have another, thank you!", was no objective criteria, and essentially useless.

That's exactly why we changed it.

It is my opinion that a "5" rating overall should be allowable if the reviewer judges the product to be in the very top echelon, not just when he judges it to be impossible to improve.

We never said that we will prevent someone from using a 5 overall score. What I said was, use it at your discretion, but with trepidation and the knowledge that every 'really good' absinthe that comes out shouldn't be getting a 5. Just like every really good wine isn't getting a 100 score.
The problem there lies in future releases. Let's say a brand scores an overall 5 now. Speaking from experience over the past 10 years, production of absinthe will continue to improve. Does that mean, 4 or 5 years from now EVERY new absinthe that comes out will score an overall 5? When DPs product continues to be refined and she has the ability to age product, I'm sure it will get better (although it's great now). Scoring a 5 now means when the improved product comes out, it will score the same, and thus appear that it wasn't improved upon.

Initially, I saw the new description as very useful, but limiting. There is a difference between "Perfect. No improvements needed." and Perfect. No improvements possible.

So do we change it, or keep it how it is?
Answers to common newcomer questions.

List of WS articles from across the web.


Help other absintheurs and newcomers by submitting a review. Click here to go to the main review page to submit your entry.

Rantings of a DC Gourmand.
WS on the Mutineer Blog!

#74 Brian Robinson

Brian Robinson

    Shabba

  • Advisory Board
  • PipPipPipPipPipPipPipPipPip
  • 12,818 posts

Posted 14 May 2009 - 03:33 AM

I'd think that in a five star rating system you certainly should be able to use the fifth star, otherwise you just have a four star rating system. :fork:

Restaurants and hotels all have five star rating systems, and there are certainly establishments that receive all five stars.

But VERY VERY few do. And those scores are also reviewed yearly if not more. I don't see most reviewers on the page to have enough interest in it to go back and rescore their ratings every year.

As I mentioned before:

If you truly think that a brand cannot be improved upon, then 5 it is. It's your opinion, and it's perfectly valid and will be respected.


Answers to common newcomer questions.

List of WS articles from across the web.


Help other absintheurs and newcomers by submitting a review. Click here to go to the main review page to submit your entry.

Rantings of a DC Gourmand.
WS on the Mutineer Blog!

#75 Absomphe

Absomphe

    Krinkles the Clown™

  • Member
  • PipPipPipPipPipPipPipPipPip
  • 18,720 posts

Posted 14 May 2009 - 04:48 AM

Restaurants and hotels all have five star rating systems, and there are certainly establishments that receive all five stars.



Except for those Diners, Drive-Ins, and Dives that use the Michelin three star rating system, of course. ;)

Yes, I'm Krinkles the Clown on an absinthe a beer bender.

You got a problem with that?


#76 Brian Robinson

Brian Robinson

    Shabba

  • Advisory Board
  • PipPipPipPipPipPipPipPipPip
  • 12,818 posts

Posted 14 May 2009 - 04:52 AM

Which also is reviewed regularly and stars either given or taken away. For comparison's sake, roughly 0.0005% of restaurants in the world receive 3 Michelin stars, the equivalent of 5's across the board for a WS review.

0.28% of hotels and restaurants receive the coveted 5 Diamond award from AAA.

For some interesting insight from a wine guy about ratings, take a read here.

In my mind, giving all 5s for the qualities, then an overall of 4 gives it the equivalent of a Robert Parker wine score of 90-99, which he defines as:

equivalent to an A and is given only for an outstanding or special effort. Wines in this category are the very best produced of their type. There is a big difference between a 90 and 99, but both are top marks. As you will note through the text, there are few wines that actually make it into this top category because there are not many great wines.


A 5 in overall, would be the equivalent of giving Robert's wine a 100, which is almost as rare as the 3 Michelin star/5 Diamond award. Wine Spectator listed its Top 100 wines of 2008. Only 1 received a score of 100. Only 1 in 2007 and NONE in 2006.
Answers to common newcomer questions.

List of WS articles from across the web.


Help other absintheurs and newcomers by submitting a review. Click here to go to the main review page to submit your entry.

Rantings of a DC Gourmand.
WS on the Mutineer Blog!

#77 bksmithey

bksmithey

    Advanced Member

  • Member
  • PipPipPipPipPip
  • 890 posts

Posted 14 May 2009 - 05:05 AM

Ok, I'm still confused. Brian, you said ""If you truly think that a brand cannot be improved upon, then 5 it is." But as fpb pointed out, there is a difference between "no improvement needed" and "no improvement possible". The overall category in the WS review says the former, while your comment (cannot be improved upon) seems to imply the latter.

When I first read your comment, I thought "ok, that means if any category gets less than a 5, that really indicates that the absinthe can be improved, so the overall cannot be a 5". So I revised my MoL review, and changed the overall to a 4, and I had given a 4 to the aroma. But now, taking into consideration that the wording is "no improvement needed", I really feel that it can be a overall 5. (I'm going to let everything chill for a bit, give my rapidly eroding credibility a well-deserved rest, and maybe re-review if I have time over the weekend)

I really like fpb's suggestion of "very top echelon", but would that be considered too subjective? I'm actually very comfortable with such a notion; many years ago, when I was first getting into single malts, I had 5 groups that I'd lump them into after tasting. The ones that I had in my top group (my "very top echelon") were considered roughly of equal quality. I never considered whether or not they could possibly be improved, just that they were the best of the best. There weren't many that I put into this bucket (my poor memory tells me that the distribution was bell curve-ish), and occasionally, as I got more malts under my belt, some would drop out.

All my rambling aside:
  • When it gets to this top echelon, I'm not qualified (and probably never will be) to judge whether or not something CAN be improved. If that's the criterion for an overall 5, I can't make that judgement.
  • If the intent is really "no improvement needed" vs. "no improvement possible", we may want to update the description, as the distinction is subtle and was certainly lost on me. "Very top echelon" (or "in an elite class") actually suits me fine, but may be considered to be too subjective.


#78 Brian Robinson

Brian Robinson

    Shabba

  • Advisory Board
  • PipPipPipPipPipPipPipPipPip
  • 12,818 posts

Posted 14 May 2009 - 05:28 AM

Ok, I'm still confused. Brian, you said ""If you truly think that a brand cannot be improved upon, then 5 it is." But as fpb pointed out, there is a difference between "no improvement needed" and "no improvement possible". The overall category in the WS review says the former, while your comment (cannot be improved upon) seems to imply the latter.

This is the problem every review system gets into. Arguing about symantics.

I think I've explained pretty well what my thoughts are regarding the definition of a 5 overall score.

I'd like to get as many people as possible to weigh in on it. If the former definition is preferred, then that's what we'll go with. If the latter is preferred, same deal. I just want to make sure it's UNIFORM and that everyone understands the criteria.

Shall I set up a poll? Would that be the easiest way to get as many votes as possible in the shortest amount of time?
Answers to common newcomer questions.

List of WS articles from across the web.


Help other absintheurs and newcomers by submitting a review. Click here to go to the main review page to submit your entry.

Rantings of a DC Gourmand.
WS on the Mutineer Blog!

#79 Absomphe

Absomphe

    Krinkles the Clown™

  • Member
  • PipPipPipPipPipPipPipPipPip
  • 18,720 posts

Posted 14 May 2009 - 05:35 AM

"No improvement needed" sounds more like a four, to me (or somewhere between a four and a five, although I realize there's no room for such a breakdown under this scoring system), while "no improvement possible" has the ring of a five. Keeping in mind that it's still one's subjective idea of what couldn't be possible, but, in fact, might be.

Yes, I'm Krinkles the Clown on an absinthe a beer bender.

You got a problem with that?


#80 Phoenix

Phoenix

    Advansed Member

  • Member
  • PipPipPipPipPipPipPip
  • 1,654 posts

Posted 14 May 2009 - 05:51 AM

Do the powers that be want an absinthe review database made from only review experts, or by anyone who wishes to submit a review? If it's the former, then reviews should only be allowed to be submitted by a select few. If it's the latter, then the powers that be are going to have to deal with some reviews being filled out in ways in which they may not like or agree with. I don't think you can have it both ways.
"He's a politician. It's like being a hooker. You can't be one unless you can pretend to like people while you're f***ing them."

#81 Joe Legate

Joe Legate

    2 jobs. 0 sense.

  • Advisory Board
  • PipPipPipPipPipPipPipPipPip
  • 16,995 posts

Posted 14 May 2009 - 05:52 AM

I don't think there is any question, WS wants as many reviews as possible. I also don't think there's any question, we'll discuss what the numbers are suppose to mean, again and again. It is difficult to make subjective opinions objective but ultimately, that is the goal.

I've also wanted a .5 from time-to-time, myself. If it was difficult deciding between a 3 or a 4, a 4 or a 5, a .5 would have been a near-godsend.

But I'm not advocating the change, due in large because all the reviews to this point have been made without it. I'd hate to skew and screw-up all the work that's already been done.

This has been a good conversation. I think it's time for me to re-visit my reviews. :cheers:

#82 Brian Robinson

Brian Robinson

    Shabba

  • Advisory Board
  • PipPipPipPipPipPipPipPipPip
  • 12,818 posts

Posted 14 May 2009 - 06:02 AM

If it's the latter, then the powers that be are going to have to deal with some reviews being filled out in ways in which they may not like or agree with. I don't think you can have it both ways.

The latter of course. But that is also why they asked to have a Review Editor.

If something looks out of the ordinary, additional information is gathered to see if 1) The review was submitted by a brand rep, 2) The review was submitted with a misunderstanding of criteria, 3) The review was submitted by someone trying to intentionally make a brand look bad, or 4) The review was submitted appropriately.

We have tried to make the criteria as easy as possible to submit a correct score without any way of needing to argue about what the numbers mean. Does it look natural? Is the louche appropriate? Does it taste basic, or is it well formed and complex? Is it appropriate for it's classification?


The numbers should be objective based on the stated criteria. The comments should be subjective.

As mentioned ad nauseam, there will always be outliers both on the high end and low end of ratings for a particular brand. The more reviews that are posted, the less impact those outliers will have on the overall score.
Answers to common newcomer questions.

List of WS articles from across the web.


Help other absintheurs and newcomers by submitting a review. Click here to go to the main review page to submit your entry.

Rantings of a DC Gourmand.
WS on the Mutineer Blog!

#83 fingerpickinblue

fingerpickinblue

    Hallucinations Sold Separately

  • Content Team
  • PipPipPipPipPipPipPipPipPip
  • 3,364 posts

Posted 14 May 2009 - 07:21 AM

It is my opinion that a "5" rating overall should be allowable if the reviewer judges the product to be in the very top echelon, not just when he judges it to be impossible to improve.

What I said was, use it at your discretion, but with trepidation and the knowledge that every 'really good' absinthe that comes out shouldn't be getting a 5. Just like every really good wine isn't getting a 100 score.


Agreed

When DPs product continues to be refined and she has the ability to age product, I'm sure it will get better (although it's great now). Scoring a 5 now means when the improved product comes out, it will score the same, and thus appear that it wasn't improved upon.


It is possible that a product is being improved or changed, but within the strata of the very finest of its type. I don't think little tweaks within such a narrow range will ever be accounted for adequately with a 5 point rating system.

Initially, I saw the new description as very useful, but limiting. There is a difference between "Perfect. No improvements needed." and Perfect. No improvements possible.

So do we change it, or keep it how it is?


Not sure yet, but I think this conversation is helpful to guide us.

Which also is reviewed regularly and stars either given or taken away.

roughly 0.0005% of restaurants in the world receive 3 Michelin stars, the equivalent of 5's across the board for a WS review.
0.28% of hotels and restaurants receive the coveted 5 Diamond award from AAA.


I think we're loosing track of the fact that we're not talking about 5s across the board, we're talking about 1 component of the final score. I agree that 5s across the board is the equivalent of a Parker 100. The final score is the final score. A final score of 4.5, to my way of thinking, would be the equivalent of Parker's 90 points, and there are certianly currently produced absinthes that are deserving of that.

Right now, we almost seem to be making the mistake, in my opinion, of treating the category of "Overall" as if it is the final score. The implication is that an overall impression of 5 is only deserving when a product receives all 5s in the other categories. I think it should be treated as the reviewers overall blended impression of the product they are reviewing.

A 5 in overall, would be the equivalent of giving Robert's wine a 100...


I don't think so, see above.
blind man see her, dumb man call her name - Ed Bell

#84 Brian Robinson

Brian Robinson

    Shabba

  • Advisory Board
  • PipPipPipPipPipPipPipPipPip
  • 12,818 posts

Posted 14 May 2009 - 07:26 AM

Right now, we almost seem to be making the mistake, in my opinion, of treating the category of "Overall" as if it is the final score. The implication is that an overall impression of 5 is only deserving when a product receives all 5s in the other categories. I think it should be treated as the reviewers overall blended impression of the product they are reviewing.

A 5 in overall, would be the equivalent of giving Robert's wine a 100...


I don't think so, see above.

But the only time one would even be able to consider giving an overall score of 5 is if all other criteria score a 5.

One can score 5's in all areas, but still a 4 in 'Overall', but based on the criteria, there isn't a way were a brand could score less than 5 in all areas, but then a 5 in 'Overall'.

Anything less than 5 in any of the any other criteria indicates that there can be an improvement made.
Answers to common newcomer questions.

List of WS articles from across the web.


Help other absintheurs and newcomers by submitting a review. Click here to go to the main review page to submit your entry.

Rantings of a DC Gourmand.
WS on the Mutineer Blog!

#85 Absomphe

Absomphe

    Krinkles the Clown™

  • Member
  • PipPipPipPipPipPipPipPipPip
  • 18,720 posts

Posted 14 May 2009 - 07:57 AM

So (to use my favorite example) I gave Pacifique a 5 in every category but color, where I lowered it to a 4 because I thought it was just a wee bit on the pale side, and by your reckoning, I'm required to give it a 4 overall?

Given that it averaged out to a 4.8 in all the other categories, I think this is too rigid, and doesn't accurately reflect the overall quality of the product...a 4 is tantamount to an 80%, no matter how you slice it, and there is no way that I could justify giving Pacifique that kind of score on the whole...I'd sooner reinstate the original 5 I awarded it for color.

The big problem for me (particularly with the overall category) is that there is a tremendous disparity between a rating of 4 and a rating of 5. If I were able to give it a 4.8 overall, that would make far more sense to me. Perhaps the overall category should be broken down further to allow it to serve as a reflector, or average of all the other categories.

Just my .02.

Rant off, btw. :)

Yes, I'm Krinkles the Clown on an absinthe a beer bender.

You got a problem with that?


#86 Brian Robinson

Brian Robinson

    Shabba

  • Advisory Board
  • PipPipPipPipPipPipPipPipPip
  • 12,818 posts

Posted 14 May 2009 - 08:15 AM

If we go by either definition above about what a 5 overall signifies, then yes, if you give a 4 to color, then overall shouldn't be 5, since you readily admit you think the color can be improved upon.

You can note in the comments your thoughts about the scoring and its limitation.
Answers to common newcomer questions.

List of WS articles from across the web.


Help other absintheurs and newcomers by submitting a review. Click here to go to the main review page to submit your entry.

Rantings of a DC Gourmand.
WS on the Mutineer Blog!

#87 Absomphe

Absomphe

    Krinkles the Clown™

  • Member
  • PipPipPipPipPipPipPipPipPip
  • 18,720 posts

Posted 14 May 2009 - 08:41 AM

I think I'll simply take the other option, since I really do find Pacifique's color to be quite appealing. I only quibbled with it as excuse not to give Pacifique a perfect score, however, the longer I enjoy it, the more I realize that it's as perfect for me as it gets.

Yes, I'm Krinkles the Clown on an absinthe a beer bender.

You got a problem with that?


#88 fingerpickinblue

fingerpickinblue

    Hallucinations Sold Separately

  • Content Team
  • PipPipPipPipPipPipPipPipPip
  • 3,364 posts

Posted 14 May 2009 - 06:04 PM

I don't think there is any question, WS wants as many reviews as possible.

But I'm not advocating the change, due in large because all the reviews to this point have been made without it. I'd hate to skew and screw-up all the work that's already been done.


Agreed.

...that is also why they asked to have a Review Editor.

If something looks out of the ordinary, additional information is gathered to see if 1) The review was submitted by a brand rep, 2) The review was submitted with a misunderstanding of criteria, 3) The review was submitted by someone trying to intentionally make a brand look bad, or 4) The review was submitted appropriately.


And I think this is necessary... And I'm glad it's you, not me, Brian! :dev-cheers:
blind man see her, dumb man call her name - Ed Bell

#89 fingerpickinblue

fingerpickinblue

    Hallucinations Sold Separately

  • Content Team
  • PipPipPipPipPipPipPipPipPip
  • 3,364 posts

Posted 14 May 2009 - 06:51 PM

Right now, we almost seem to be making the mistake, in my opinion, of treating the category of "Overall" as if it is the final score. The implication is that an overall impression of 5 is only deserving when a product receives all 5s in the other categories. I think it should be treated as the reviewers overall blended impression of the product they are reviewing.

A 5 in overall, would be the equivalent of giving Robert's wine a 100...


I don't think so, see above.

But the only time one would even be able to consider giving an overall score of 5 is if all other criteria score a 5.

One can score 5's in all areas, but still a 4 in 'Overall', but based on the criteria, there isn't a way were a brand could score less than 5 in all areas, but then a 5 in 'Overall'.

Anything less than 5 in any of the any other criteria indicates that there can be an improvement made.


Then the definition of what attributes qualify for a rating of "5" overall need to be changed again, since it removes one fifth of one of the building blocks for a final score, except in rare instances.

Also, and I quote from the WS review guidelines;

"OVERALL IMPRESSION
This is your overall feeling about this absinthe. You can equalize any deficits you feel may have resulted from the scorings above. While an absinthe might technically score lower on the above criteria, it may have some other quality that mitigates that deficit."

I have never been comfortable with the idea of "perfection" being the standard for the rating of "5" in any category. Extreme excellence should be the standard.

Imagine this;

0-----1---------2--------3---------4---------5----P


That's the way I see it.

Brian, it seems you see it;

0-----1---------2--------3-----------4-----------5


And that is what effectively happens when you define a rating as perfection, let alone how you ever get an effective number of people in the user base here to understand "perfection".

In this scoring system;

1 implies a range.
2 implies a range.
3 implies a range.
4 implies a range.
5 at least at this point is... exactly that. An exact point. How do we know? As long as perfection is the criteria, a rating of "5" in any category is as useless as the former definition for "Overall".

5 should also imply a range. It should be narrow, elite, and well defined.
blind man see her, dumb man call her name - Ed Bell

#90 fingerpickinblue

fingerpickinblue

    Hallucinations Sold Separately

  • Content Team
  • PipPipPipPipPipPipPipPipPip
  • 3,364 posts

Posted 14 May 2009 - 07:21 PM

If we go by either definition above about what a 5 overall signifies, then yes, if you give a 4 to color, then overall shouldn't be 5, since you readily admit you think the color can be improved upon.

You can note in the comments your thoughts about the scoring and its limitation.


In my case, my reviews can become even wordier than they already are! :tongue:

I think I'll simply take the other option, since I really do find Pacifique's color to be quite appealing. I only quibbled with it as excuse not to give Pacifique a perfect score, however, the longer I enjoy it, the more I realize that it's as perfect for me as it gets.


And that is why some range in each rating in each category has to be part of the system.

I stated earlier that it was my plan to revisit each of my reviews in July, in part, to decide which were truly deserving of ratings of "5" in overall. While it is not done, my current inclination would be to give a "5" overall to;

Jade PF1901
Jade Edouard
MAdlBE
Pacifique

and... drum roll......................

MOL... Yeah, It blows me away that much! Hip, Contemporary, Creative, Respectful of Tradition, Well Executed, and Very Artistic.

Now at first glance, while 5 of the presumably 22 - 24 reviews I will have posted at that time may seem like a high percentage, keep in mind that the absinthes I've reviewed are, for the most part, very top tier offerings, thanx in large part to this site and this forum. If I had the time, money, and inclination to review another 40 offerings of far more pedestrian quality, the number would be right down under the 10% range that I think might comprise the very top echelon, to which I refer, in these posts.
blind man see her, dumb man call her name - Ed Bell


0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users

Copyright © 2014 The Wormwood Society Absinthe Association