YouTube     Facebook     Twitter

Tuesday, September 7, 2010

The problem with Yelp

As I was taking a break over Labor Day weekend SliceTruck got Yelp bombed.

Yelp bombed? I don't know what to call it exactly but I saw an article on someone getting Google bombed today and this seemed similar. Anywise...

We got three yelp reviews on Labor Day (all negative). This is peculiar because the truck hasn't been out selling pizza for 3 days and typically when we have a fully booked weekend and serve thousands of people we might get just 1-3 reviews at the most.

Its clear the reason is because we reviewed a Yelp review by Randi and it turns out the Yelp community did not care for this. They started by leaving anonymous comments at the bottom of the post and then moved on to posting Yelp reviews aimed at defending Randi and her review.



The other two reviews were not as obvious as this one but both tied in to the comments made on slicetruck.com and are a clear reaction to us reviewing Yelp reviews and particularly the one by Randi.

Now you might say, "Why would you even provoke these people? Its the biggest review site on the web and you want these people to be your friends. You're only hurting your business. People like positive and you're being negative etc. etc."

To this I say you're probably right and I'm sure it does appear to be a dumb business move, but we can't dwell on the past and the point I'm trying to make here is that Yelp is fatally flawed. Its a review site that can very easily be gamed or controlled by a mob. A group can decided they want to make a point and post a bunch of negative reviews or, more likely, business owners have all their friends post glowing reviews giving their business a false baseline to absorb any negative reviews. The result of this ends up providing very little help to the person who visits the site trying to find good restaurants in their area.

Another problem is that Yelpers tend to associate different values to their stars, three of five means good from some who think they're giving out Michelin stars, where three of five means average and forgettable to others.

For example if I do a search on yelp using my zip code "90291" for restaurants and sort to most reviewed and then look at the first 100 results 95% are between 3 and 4 stars. What does this mean to me? It means 4 is good and 3 is bad.

When I look at this chart that I pulled from Yelp it actually tells a similar yet different story. If I average it all out this chart tells me that 3.77 is the average rating given on Yelp. Formula:(.32*5 + .35*4 + .18*3 + .08*2 + .07*1).

Now if we use the classic letter grading system where a C is average and an average C is 75% we can call 3.77 stars average.

We're going to go off this for the letter grading:
A= 90%+
B= 80-89%
C= 70-79%
D= 60-69%
F= less than 59%

Converting this to Yelp stars I get:

A= 4.46 Yelp stars and above
B= 3.96-4.45 Yelp stars
C= 3.46-3.95 Yelp stars
D= 2.97-3.45 Yelp stars
F= 2.96 Yelp stars and below

So basically on Yelp 3 stars is a D- or an F+. I don't think most Yelpers realize this. It doesn't matter though because the system is flawed and its too late to fix it.

Yelp is a great place for people to play restaurant critic and become virtual friends with other food writers and food picture takers, but the sum result of these peoples' activity provides little help for someone who's looking for a recommendation.

16 comments:

  1. Thank you for writing this ...

    ReplyDelete
  2. The rating is irrelevant.. it's just a place where people can gather to give information to other people about local businesses. If I'm looking for a Greek restaurant, sure, I'll check out Yelp, but I'm not going to base my decision on the "star rating".. I'm going to go through and read what specifically people are saying about their own experiences. And give people a little more credit... it's easy to tell when reviews are planted by angry mobs, and which reviews are genuine. I agree that the site is heavily flawed, and for what ever reason, people need "ratings" on things because they dont want to take the time to read facts, but aside from the mathematical woes, Yelp is a forum for people to share experiences. If someone had a bad experience at your truck, own up to it, and become better because of it.

    ReplyDelete
  3. I agree, Yelp is flawed... but as a person who often finds restaurants through Yelp, I consider 3 Stars as pretty good. It isn't until I see an average rating of 2 that I start to shy away from a place. I know it seems skewed that most restaurants are in the 3 and 4 range, but that doesn't mean that low threes/upper 2s are fails. I think most restaurants just fall into the "Average to Above Average" rating or else they wouldn't be open anymore. For what it's worth, my girlfriend and I consider your pizza among the best (and we're both from NY, and we've been to a LOT of pizza places out here just trying to find a DECENT slice, and yours rocks regardless of which coast its from). I don't have a yelp account, but she does. If she hasn't reviewed you guys yet, I'll make sure she gives you the glowing review you deserve.

    ReplyDelete
  4. This argument is extremely flawed because you're making your assumptions based on your own mathematical system of equating 1-5 stars to school letter grades. (as opposed to the LA County's Health Dept Letter grades, which are truly on a different scale, but I digress...)

    Yelp already has their own system on the site, and clearly identifies what each star rating means. So the fact you arbitrarily change the meaning on a whim, makes you look that much sillier.

    You also assume that there's a mathematical equivalent to star ratings in the first place that correlates to how the users interpret the ratings.

    Ex: User A sees that X restaurant has an average 3.5 star rating. Do they automatically assume that X restaurant is somehow worse than one with an average 4-star rating?

    Yelp's strength is the reviews that have content. Not "this place got an average 2 stars from 8 people so I'm not going." The more well-written the reviews, no matter good/bad, the more serious people take that review. This goes across multiple mediums. One's opinion/review is only as good as the evidence given to support it.

    If someone gives 5 stars to every restaurant they go to without providing much detail, that review is completely useless to the site's population and only slightly skews the star rating up/down.

    The only times restaurants should truly take umbrage with yelp reviews are when there are misrepresentation of facts posted. Like if someone hated your pizza because they said each slice cost $15... Fire away.

    But otherwise, this misguided campaign of yours against Yelp reviewers and ranting about what other people's ratings mean, is only hurting yourselves.

    ReplyDelete
  5. How about reviewing events before they happen and give them five stars...yes it is flawed

    ReplyDelete
  6. While you may think that Yelp may be fatally flawed, This doesn't mean that two negatives equals a positive.

    Your customer service must rise above that, and as I pointed out in the past, if you reach out to those who did not leave satisfied then you'll get a better rep for good customer service.

    You attract more bees with honey...

    I completely agree with you about Clarissa's review. And if you flag it, they'll remove it since she states right there that she never has been to your truck. That alone is reason enough for that review not to exist.

    But you see that you could have avoided that by not blasting reviews that were fairly positive. Again, a 3 star is not the end of the world and should be considered an average rating.

    Your hard line of looking at reviews as black/white is still off. 4=good 3=bad is a wrong way to look at it.

    3 is average, Again, much like Yelp's own reviewing meaning indicates "A-ok".

    You're setting a definition using a school grade when you shouldn't. This isn't grading on a curve because everyone's curve will be different.

    There's some who give 5 stars out like candy while others seldom do so. Maybe this should just ENCOURAGE you more to not just blast customers who were dissatisfied and actually do something to improve their potential future business with them.

    ReplyDelete
  7. your rating system is the flawed one, yelp makes it pretty damn clear what their ratings mean and its your own personal interpretation that has you hating on yelpers - if you want to see harsh yelp reviews look at the ones up in SF where they *really* take that rating system seriously, lots more 2 and 3 star reviews running around.

    as far as the negative yelp reviews you received from people who didnt even eat at your truck, go flag them. I already did. there *is* a yelp community that *does* mediate itself. its not perfect, but it certainly does pretty damn well. I personally find yelp very helpful in finding restaurants and recommendations.

    And while you're over there on the hated http://www.yelp.com , why not just get a business account with yelp so you can blast all those reviews right there *on yelp* instead of over here on your lame blog that no one reads and that you have to tweet about your yelp bashing for anyone to care about.

    ReplyDelete
  8. First off, if I had the chance to NOT rate you guys at all based on the PERSONAL service you have provided to -me-, and the food itself, I would have not chosen a star or several at all!

    But the fact is, I had no choice. But in my mind I felt it was necessary to personally show people how you've treated Randi - a girl who I don't know at all. And this would have gone for anyone else.. Knowing them or not was/is not a factor of the 'review' I made.


    You say you do not dwell yet you 'picture' my post on your site and continue to talk about how I "defended Randi". I didn't defend her. I said nothing about Slice Truck treating her unfairly. I simply SHOWED them what you guys have posted. And now they will see this too. And look at the people who have responded, for the most part a lot of these people aren't really even agreeing with you.

    So sorry for 'lowering' your rating on Yelp. I like to stick with the truth, and that text was profoundly shown from your hands.

    Good luck!

    - Clarissa ( From Yelp )

    ReplyDelete
  9. Yelp is absolutely flawed, and I base this on the quality of reviews. I like it for the photo archives and for addresses/photos of the restaurant, but the reviews overall tend to be really thin and uninspired, and I would say that only about 30% of my Yelp finds have been good ones. I've mostly stopped using Yelp because of this. Other sites, like Chowhound, are less flashy but the posters go in to much greater depth and I'd say about 90% of my Chowhound finds have been successful.

    ReplyDelete
  10. I understand your rationalization, but do not agree that it is a flaw. While placing it into the context that you have, it appears that using the 1-5 star system is not fair to the business, however to the yelper it makes perfect sense. You have to add the "common sense" factor which is 100% qualitative and cannot be rationalized using calculations to understand.
    To illustrate this, I will use my own experience with your truck as an example:
    I ate at the slice truck and found the pizza to be mediocre, however the service was above average and the nifty factor of eating from a truck would cause me to give you five stars. Given the fact that yelp is a qualitative system, I WILL factor in my personal feelings about your posting here WHICH ARE NEGATIVE because I am a yelper. This brings my rating down to three stars. Ultimately your rating hurts because of having strong personal feelings which in business you just cannot voice when they conflict with a large group of your customer base!

    ReplyDelete
  11. I have a different perspective for the Slice Truck. I haven't eaten from your truck. I haven't seen the Yelp reviews or read any other blog articles/comments here. I am on the UCLA campus. Several of my colleagues got pizza from your truck on August 25th. Their review was unanimous and echoed here by chickeninthemist – mediocre. BUT I am on your site today because your truck is here on campus again today and I felt like pizza. I take others opinions with a grain of salt so I was looking for your menu before heading out to the trucks. Guess what? Now I have my own opinion that the truck management’s response to problems is to blame others. I think I’ll pass simply based on your blog post. That my friend is how to lose customers before you even have them.

    ReplyDelete
  12. Thank you so much for writing this. You are correct. I believe that Yelp is flawed. Sure it promotes connections and trying to help people out, but there are wrongs with this site that just doesn't appeal to me at all. For instance, the "Was this review...? Useful, Cool, Funny". Are you serious? Sure it promotes positivity, but what about realism? There is one particular reviewer that I despise due to his lack of understanding and ethics.

    Which also brings another flaw. Although someone giving a reasoned negative for a site may prove helpful, some people criticize is in other aspects that aren't related to the business itself. In fact, people who have the same competing business will give a "lengthy" negative review to pose as a legitimate review.

    And as for the review itself, everything can be described. It's just a phrase to say that the food was indescribable. But your reviews are providing no substance by saying such a phrase. In fact, please describe it. How did it taste? How was the texture? etc. You can explain how it tastes!

    Although there are a few users who's reviews are legitimate and provide reason and evidence, it's like finding a needle in a haystack.

    ReplyDelete
  13. You have it dead on, slice. Yelp has created a juggernaut of dumbsourcing that no longer provides useful or discerning information. Hopefully someone will move in with an improved system soon.

    ReplyDelete
  14. Thank you for your blog. Unfortunately, Yelp and Yelpers are still up to the same tricks years later. Don't pay for Yelp's advertising, and guess what...negative reviews posted, positive reviews filtered.

    Also, I don't quite know how that person's review you posted ended up on your Yelp Page. According to Yelp's Review Guidelines, they clearly state that the review has to be of first hand knowledge, not hearsay. The reviewer even stated they hadn't frequented your business.

    Yelp is a fraud and a joke. If anyone is interested, join me and others in our fight against yelp at http://www.facebook.com/yelpisafraud. Post your own stories, chat with others, spread awareness about Yelp's questionable business practices, etc.

    ReplyDelete
  15. .Check Footsie in Montana Avenue and I truly believe Yelp is totally flawed. Yelp fiiltered all the negative reviews and based on the filtered reviews this one lady owner shop needs to close down!

    ReplyDelete