BI Buyers Guide Taps Real Customer Reviews

Cloud Storage: 8 Ways You're Wasting Money

Would you recommend your current business intelligence product to others? That's a core question behind a new BI Buyer's Guide released on Tuesday that offers ratings based on customer reviews.

The "Buyer's Guide to Business Intelligence" is based on 532 customer reviews that have been screened and authenticated by TrustRadius, a two-year-old startup that bills itself as a cross between a Yelp- or TripAdvisor-style customer review site and analyst businesses like Gartner and Forrester Research.

TrustRadius isn't alone in its Yelp-for-IT aspirations. G2 Crowd and others are compiling and publishing customer product reviews. TrustRadius says it stands apart in that it validates that the reviews it publishes, which average 400 words but can be as long as 3,000 words, are written by actual customers. The company says it has ways to screen out glowing reviews written by vendor proxies and bad reviews written by those seeking to trash rivals.

[Want more on BI product reviews? Read Gartner BI Magic Quadrant: Winners & Losers.]

For starters, TrustRadius recruits at least 80% of its reviews from previous users of its site or from people with LinkedIn profiles that cite experience and expertise with specific products. TrustRadius researchers verify that profiles weren't created yesterday, and they read each and every review to spot those that don't seem legitimate.

"There are signals that we look for when we read a review that tip off that it's a puff piece or a competitor, and we end up rejecting about 3% to 5% of the reviews that are submitted," Vinay Bhagat, founder and CEO of TrustRadius, said in an interview with InformationWeek. "We ask reviewers about pros and cons, for example, and we see patterns such as extreme perspectives or cons that don't seem very genuine."

TrustRadius also works with vendors, which recommend the other 15-20% of customers and offer comments and perspectives on what customers have to say about their products. TrustRadius serves in an analyst role in that it curates customer comments, summarizes use cases and deployment patterns associated with each product, and writes explanatory and how-to content on the topics it covers.

Launched in 2013, TrustRadius has released buyer's guides and detailed reports on digital analytics, marketing automation, social media management, and, now, business intelligence vendors. The guides are freely available for download. More in-depth, product-specific reviews are also free, but only to "Premium" users who are registered and agree to allow their contact information to be shared with vendors.

TrustRadius would not identify the vendors that work with it, and it insisted that vendor involvement doesn't influence its curation of comments or its "TrustMap" rankings, which are based on average user ratings and the number of page views tied to individual product reports.

"Much like consumer review sites like TripAdvisor, which make money from referrals and bookings, our reviews are 100% driven by user perspectives," Bertrand Hazard, TrustRadius vice president of marketing, said in a followup email interview. "We create the questions that are asked about all products, but have no influence over responses or ratings. We source and publish reviews for all products irrespective of whether vendors have a revenue relationship with us."

As seen above, the standouts in the first TrustMap user rankings of full-stake BI vendors are SAS, BIRST, GoodData, and Microsoft, in that order, all of which were well above the median user rating. A separate TrustMap on Data Discovery and Visualization vendors puts Tableau Software ahead of Qlik, Tibco Spotfire, and Entrinsik. A separate Advanced Analytics TrustMap is in the planning stage, according to the company.

TrustRadius says the products included in its TrustMap rankings have at least 20-30 validated reviews. A visit to the site reveals that some products have more than 30, while others (not included in the TrustMap) have far fewer. Even 20-30 is not a big sample. Nonetheless, TrustRadius research director Alan Cooke insists it's enough to yield reliable insight.

"Once you get up to 20 reviews, you start to see clear patterns, and we have fairly high confidence after that point," Cooke told InformationWeek.

The user ratings and numbers of page views on reports on each vendor hint at user satisfaction and interest in each product. TrustRadius says it gets more than 130,000 visitors to its site each month, but that's across all report categories. The built-in sample bias here is that the user and interest rankings are limited to people who have visited and interacted with the TrustRadius site. It's a bit surprising, for example, that BIRST and GoodData received more product evaluations than Oracle, maker of a top BI product (OBIEE) that has been growing in recent years.

It can't hurt to consider these screened and curated comments and added perspectives from TrustRadius, so long as you don't mind being contacted by vendors. What this report needs most is more reviews and more visits to the site to ensure a more statistically valid take on vendors. That and full disclosure of vendor relationships would bring more trust to this new, Yelp-like source of product reviews.

Apply now for the 2015 InformationWeek Elite 100, which recognizes the most innovative users of technology to advance a company's business goals. Winners will be recognized at the InformationWeek Conference, April 27-28, 2015, at the Mandalay Bay in Las Vegas. Application period ends Jan. 16, 2015.