BI Buyers Guide Taps Real Customer Reviews - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management // Big Data Analytics
12:40 PM
Connect Directly

BI Buyers Guide Taps Real Customer Reviews

TrustRadius releases a business intelligence buyer's guide based on 532 verified customer reviews. SAS, BIRST, GoodData, and Microsoft stand out.

Cloud Storage: 8 Ways You're Wasting Money
Cloud Storage: 8 Ways You're Wasting Money
(Click image for larger view and slideshow.)

Would you recommend your current business intelligence product to others? That's a core question behind a new BI Buyer's Guide released on Tuesday that offers ratings based on customer reviews.

The "Buyer's Guide to Business Intelligence" is based on 532 customer reviews that have been screened and authenticated by TrustRadius, a two-year-old startup that bills itself as a cross between a Yelp- or TripAdvisor-style customer review site and analyst businesses like Gartner and Forrester Research.

TrustRadius isn't alone in its Yelp-for-IT aspirations. G2 Crowd and others are compiling and publishing customer product reviews. TrustRadius says it stands apart in that it validates that the reviews it publishes, which average 400 words but can be as long as 3,000 words, are written by actual customers. The company says it has ways to screen out glowing reviews written by vendor proxies and bad reviews written by those seeking to trash rivals.

[Want more on BI product reviews? Read Gartner BI Magic Quadrant: Winners & Losers.]

For starters, TrustRadius recruits at least 80% of its reviews from previous users of its site or from people with LinkedIn profiles that cite experience and expertise with specific products. TrustRadius researchers verify that profiles weren't created yesterday, and they read each and every review to spot those that don't seem legitimate.

"There are signals that we look for when we read a review that tip off that it's a puff piece or a competitor, and we end up rejecting about 3% to 5% of the reviews that are submitted," Vinay Bhagat, founder and CEO of TrustRadius, said in an interview with InformationWeek. "We ask reviewers about pros and cons, for example, and we see patterns such as extreme perspectives or cons that don't seem very genuine."

TrustRadius also works with vendors, which recommend the other 15-20% of customers and offer comments and perspectives on what customers have to say about their products. TrustRadius serves in an analyst role in that it curates customer comments, summarizes use cases and deployment patterns associated with each product, and writes explanatory and how-to content on the topics it covers.

Launched in 2013, TrustRadius has released buyer's guides and detailed reports on digital analytics, marketing automation, social media management, and, now, business intelligence vendors. The guides are freely available for download. More in-depth, product-specific reviews are also free, but only to "Premium" users who are registered and agree to allow their contact information to be shared with vendors.

TrustRadius would not identify the vendors that work with it, and it insisted that vendor involvement doesn't influence its curation of comments or its "TrustMap" rankings, which are based on average user ratings and the number of page views tied to individual product reports.

"Much like consumer review sites like TripAdvisor, which make money from referrals and bookings, our reviews are 100% driven by user perspectives," Bertrand Hazard, TrustRadius vice president of marketing, said in a followup email interview. "We create the questions that are asked about all products, but have no influence over responses or ratings. We source and publish reviews for all products irrespective of whether vendors have a revenue relationship with us."

As seen above, the standouts in the first TrustMap user rankings of full-stake BI vendors are SAS, BIRST, GoodData, and Microsoft, in that order, all of which were well above the median user rating. A separate TrustMap on Data Discovery and Visualization vendors puts Tableau Software ahead of Qlik, Tibco Spotfire, and Entrinsik. A separate Advanced Analytics TrustMap is in the planning stage, according to the company.

TrustRadius says the products included in its TrustMap rankings have at least 20-30 validated reviews. A visit to the site reveals that some products have more than 30, while others (not included in the TrustMap) have far fewer. Even 20-30 is not a big sample. Nonetheless, TrustRadius research director Alan Cooke insists it's enough to yield reliable insight.

"Once you get up to 20 reviews, you start to see clear patterns, and we have fairly high confidence after that point," Cooke told InformationWeek.

The user ratings and numbers of page views on reports on each vendor hint at user satisfaction and interest in each product. TrustRadius says it gets more than 130,000 visitors to its site each month, but that's across all report categories. The built-in sample bias here is that the user and interest rankings are limited to people who have visited and interacted with the TrustRadius site. It's a bit surprising, for example, that BIRST and GoodData received more product evaluations than Oracle, maker of a top BI product (OBIEE) that has been growing in recent years.

It can't hurt to consider these screened and curated comments and added perspectives from TrustRadius, so long as you don't mind being contacted by vendors. What this report needs most is more reviews and more visits to the site to ensure a more statistically valid take on vendors. That and full disclosure of vendor relationships would bring more trust to this new, Yelp-like source of product reviews.

Apply now for the 2015 InformationWeek Elite 100, which recognizes the most innovative users of technology to advance a company's business goals. Winners will be recognized at the InformationWeek Conference, April 27-28, 2015, at the Mandalay Bay in Las Vegas. Application period ends Jan. 16, 2015.

Doug Henschen is Executive Editor of InformationWeek, where he covers the intersection of enterprise applications with information management, business intelligence, big data and analytics. He previously served as editor in chief of Intelligent Enterprise, editor in chief of ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Oldest First  |  Newest First  |  Threaded View
D. Henschen
D. Henschen,
User Rank: Author
12/9/2014 | 1:18:10 PM
TrustRadius is a lead-generation and vendor-matchmaking business
TrustRadius gets paid for lead generation. When you register, you'll get matched with vendors who choose to participate in the program, according to TrustRadius. Company execs didn't say whether you're matched based on the fit with your expressed interest and desired deployment approach (departmental vs. enterprisewide, cloud vs. on-premises, etc.) or how much vendors are willing to pay for a BI-shopper lead.

In my book, vendor involvement tends to compromise things. The BARC BI study, which costs thousands of dollars, is based on thousands of customer reviews, but this publisher also has relationships with vendors whereby they can boost customer participation.

Cindi Howson's BI Scorecard is one example of an vendor-independent review source, but you have to pay for the her expert reviews and her experience. It's a small sum to pay when you consider the cost of making a bad software choice.
User Rank: Apprentice
12/11/2014 | 11:01:37 AM
Re: TrustRadius is a lead-generation and vendor-matchmaking business

I appreciate your coverage of our BI guide but wanted to respond to your comment.

1. Our focus is helping business software buyers make better decisions. We do this through sourcing and publishing first-hand perspectives in the form of detailed reviews from real users, dynamically creating comparisons from reviews, and summarizing user insights into guides. Who pays us does not influence which products are covered, nor how they are covered. We source and publish reviews for all significant products in a category irrespective of whether vendors have a revenue relationship with us. We are an open directory - any product can be listed and reviewed. Company ratings are based solely on user ratings, and as you stated in your article, we strive to ensure the ratings are as representative as possible through independent sourcing.

2. People who register on our site are not automatically matched with vendors. They have to opt-in to be matched, and do so in return for accessing one of our premium guides at no cost. As you pointed out, other guides typically cost thousands of dollars.

3. With respect to buyers who opt-in for matching via downloading premium content, we endeavor to match them to vendors that appear to be the best fit based upon their company size, geography, the products in which their browsing history indicates they have the most interest, and where possible talking to them. Fees do not vary significantly per vendor. While we do not disclose the vendors we work with, I can state that the majority of "top rated" BI vendors currently have no business relationship with us.

The integrity of our content, and the value delivered to our members is our true north.

Vinay Bhagat

Founder & CEO, TrustRadius

10 Cyberattacks on the Rise During the Pandemic
Cynthia Harvey, Freelance Journalist, InformationWeek,  6/24/2020
IT Trade Shows Go Virtual: Your 2020 List of Events
Jessica Davis, Senior Editor, Enterprise Apps,  5/29/2020
Study: Cloud Migration Gaining Momentum
John Edwards, Technology Journalist & Author,  6/22/2020
White Papers
Register for InformationWeek Newsletters
Current Issue
Key to Cloud Success: The Right Management
This IT Trend highlights some of the steps IT teams can take to keep their cloud environments running in a safe, efficient manner.
Flash Poll