Are Online Reviews Broken?
10 comments
Caveat emptor applies today as always.
In college we learned how to evaluate sources. It was called "The CRAAP Test".
https://researchguides.ben.edu/source-evaluation
It behooves every consumer to make use of available sources of information, but to weight them appropriately and consider the components of "CRAAP" to evaluate them.
There is a lot of good signal that can be gleaned, even from a fake or negative review. Do not discount the inherent subjectivity that posters introduce.
I don't weight reviews heavily, unless some aspect is a real standout. The preponderance of reviews can sometimes tip the balance in my decision to buy/not buy, but they're often more helpful in learning comparative features/quality of similar items. If I'm shopping in unfamiliar territory, the knowledge of reviewers is invaluable to help me know more about such a product line.
Often, if enough reviewers complain about a specific defect or flaw, it will not hinder me from a purchase, but I'll be aware and informed that this product may be problematic in that specific way, and if reviewers have focused on a cosmetic or non-essential defect, I count it as an advantage that they didn't find any "showstoppers".
In college we learned how to evaluate sources. It was called "The CRAAP Test".
https://researchguides.ben.edu/source-evaluation
It behooves every consumer to make use of available sources of information, but to weight them appropriately and consider the components of "CRAAP" to evaluate them.
There is a lot of good signal that can be gleaned, even from a fake or negative review. Do not discount the inherent subjectivity that posters introduce.
I don't weight reviews heavily, unless some aspect is a real standout. The preponderance of reviews can sometimes tip the balance in my decision to buy/not buy, but they're often more helpful in learning comparative features/quality of similar items. If I'm shopping in unfamiliar territory, the knowledge of reviewers is invaluable to help me know more about such a product line.
Often, if enough reviewers complain about a specific defect or flaw, it will not hinder me from a purchase, but I'll be aware and informed that this product may be problematic in that specific way, and if reviewers have focused on a cosmetic or non-essential defect, I count it as an advantage that they didn't find any "showstoppers".
Had never seen this before, a documented and logical approach to evaluating information. Wonder if a review platform could be made where they summarize all of the review information using the CRAAP filter.
The best way I have found to evaluate online reviews has been to focus on the 2-3 star ratings that have actual rationale and logic behind their negative review.
Then evaluate if the reason that the reviewer did not enjoy the product, service, or experience is a deal-breaker for me or if it’s a tradeoff that I can live with
You see, what might be a turn-off for someone else might actually a plus for me.
The best way I have found to evaluate online reviews has been to focus on the 2-3 star ratings that have actual rationale and logic behind their negative review.
Then evaluate if the reason that the reviewer did not enjoy the product, service, or experience is a deal-breaker for me or if it’s a tradeoff that I can live with
You see, what might be a turn-off for someone else might actually a plus for me.
A voting system based on Web-of-trust minimizes the risk of spam or bots posting 5-star reviews. Building your own WOT however requires work. Lots of work in this direction are taking place on the relatively new Nostr protocol. But it’s not ready for full scale usage today.
how does a web-of-trust work?
I'd also be curious if there's any research on a new ranking system that's not only based on 5 stars. Something where it compares your reviews to other reviews you have made in the past and get some type of relative score?
When buying stuff, I use oldskool information like Consumer Report, the German TÜV, or the NYT WireCutter to inform myself.
The thing is that this just covers a subsection of online purchases, you can't use it to value a hotel for example right?
The concept though is 1000% valid, we need a review platform that somewhat guarantees, verifies and vets all reviews that it displays.
Maybe, instead of anybody publishing a review, it's this central entity that has a due diligence team that crafts a review. And instead of making money from the affiliate links to the products sites, they make money from people paying to read the verified reviews.
The concept though is 1000% valid, we need a review platform that somewhat guarantees, verifies and vets all reviews that it displays.
Maybe, instead of anybody publishing a review, it's this central entity that has a due diligence team that crafts a review. And instead of making money from the affiliate links to the products sites, they make money from people paying to read the verified reviews.
That's already how Consumer Reports works, but they don't have the capital and reach to really make much of a difference. Their reviews are also kind of gamble in terms of reliability. The negative reviews from a mass of real buyers will often find things Consumer Reports did not. It's a lot easier to post a negative review directly than to go through the reporting process of Consumer Reports.
IMHO the problem with this is that in a business environment where you need growth, eventually you run out of revenue you can extract from subscribers. I'm skeptical that this kind of service can be sustainably provided by any profit seeking company, the incentive gradient towards enshittification is very steep.
How do you build a business / culture that resists being "corrupted" (i.e. getting easy revenue and growth that's not in the customer's interest)? I believe this is a genuinely hard problem.
How do you build a business / culture that resists being "corrupted" (i.e. getting easy revenue and growth that's not in the customer's interest)? I believe this is a genuinely hard problem.
[deleted]
It’s impossible. You spend hours of time and hundreds or even thousands of dollars on something to end up feeling ripped off.
The system is fundamentally broken.