How We Built Our Own BI: When Off-The-Shelf Apps Won't Do - InformationWeek
IoT
IoT
Data Management // Big Data Analytics
Commentary
10/5/2016
11:06 AM
Sagi Bakshi
Sagi Bakshi
Commentary
50%
50%

How We Built Our Own BI: When Off-The-Shelf Apps Won't Do

Sagi Bakshi, GM of Digital Solutions for ironSource, needed a business intelligence application to meet all of his organization's unique needs. When off-the-shelf options proved inadequate, his team decided to build its own layer on top of an existing system. Here's how it did it -- and the role IT played in the process.

InformationWeek 2016 Salary Survey: 10 Best-Paying IT Leadership Jobs
InformationWeek 2016 Salary Survey: 10 Best-Paying IT Leadership Jobs
(Click image for larger view and slideshow.)

In our industry, a sturdy business intelligence solution is key. It helps us power our data-driven platforms, analyze valuable consumer insights, and track our performance -- all in near real-time. This last bit is especially important, considering that at ironSource -- an online software distribution and monetization company with a complete ecosystem for downloadable applications -- we process billions of data points monthly.

Our customer success managers -- who, in a way, were our first customers -- wanted to see how campaigns were performing in real-time, so they could easily adjust and optimize them on the fly.

Naturally, we looked into several major BI providers, hoping one of them would be able to address our strategic needs. BIScience, GoodData, and Qlik were among those we evaluated.

None seemed to quite get the job done. There were a few critical metrics and KPIs missing from each.

[Don't do it like this. Read 8 Ways to Fail at DevOps.]

So, we turned instead to building an additional layer on top of one of the existing BI platforms, and then adjusting it so it could support the essential business units and features we needed.

What We Did

It took a while (six months to be exact) until we finally developed the perfect BI solution for us. Because we built it ourselves, we were able to add useful and industry-specific features. More on that later.

First, let's talk about how, during the course of this project, our small BI team acted as salespeople, customer support, Q/A, and -- of course -- developers. At ironSource, it's typical for our teams to wear many hats, so we were prepared.

In the beginning, our BI team presented our customer success managers with a trial version and asked them to use it and offer feedback. It was a struggle at first to get the customer success managers to adopt the new tool.

It was an amusing role reversal -- usually they're the ones nudging external leads to use ironSource solutions. This time it was up to us, a group of developers, to get our internal clients to integrate our portal into their everyday routines.

We sent constant email reminders with functionality updates, animations, funny gifs -- anything to catch their attention. We even enlisted the help of our in-house studio to design appealing graphics.

(Image: ismagilov/iStockphoto)

(Image: ismagilov/iStockphoto)

Truthfully, what really pushed the customer success managers to adopt the platform was showing them that our upper management team was using it, too. Our leadership demonstrated how important and necessary it was for them to start utilizing our BI portal.

Specifically, I'd send screen shots pulling numbers and statistics from the BI platform to the customer success managers and ask them to check up on certain campaigns. This way, they'd need to go into the portal in order to respond to and access the data.

This helped us create a shared language for discussing performance, results, and expectations with clients, which can be key to both successful client management and company-wide unification. Since ironSource is made up of many divisions, this is a critical challenge for us.

Customer Support

Failure was a given. But as the saying goes, if at first you don't succeed, try, try again. In the beginning, as with most new platforms, there were discrepancies between the data our portal was presenting, and the actual data. Really, the only way we could fix this particular mistake was with the help of the customer success managers, who consistently flooded our email inbox with bugs and requests.

In fact, within a few days, we set up a WhatsApp group for support, so users could direct all their concerns to it, instead of emailing people individually. Customer support was critical. If it was 3:00 a.m., my team was up and ready to fix bugs ASAP. After all, ad campaigns don't sleep. If something was off with our numbers, it couldn't wait until morning. It wasn't uncommon for our team to stay in the office until midnight during those days.

Quality Assurance

Unsurprisingly, our initial version needed quite a bit of work. In the WhatsApp group, the customer success managers outlined three major flaws:

  • The user interface was confusing
  • The platform needed a faster response time
  • The platform also lacked some key metrics

First we tackled the UI, dividing the various different dashboards into logical groups and adding intuitive icons. Our design studio was a great help here, working with us on logos, fonts, and general design. They're UI experts, so, luckily for us, they took on most of the burden on this front.

Next, we tweaked the response time. Our platform holds millions of data points, so it needs to work fast, otherwise it becomes unhelpful, and the customer success managers fall back on using spreadsheets.

Last, we enlisted the help of ironSource's analytics team. Its members helped us calculate the complicated metrics our customer success managers requested. For them we developed hourly and monthly campaign tracking.

Sometimes customer success managers need to create monthly reports to demonstrate progress over time. To accomplish the task, we used Pentaho for ETL, which allowed us to aggregate millions of "by the minute" records into "by the month" ones.

In addition, because of our volume, we knew we needed to compare hour-to-hour results. If there's a 30% drop within an hour, the solution alerts the team so we can react quickly.

Calibrating the alerting is a fine art. On one hand, we don't want to send too many alerts (and cause alert blindness). On the other hand, we don't want to send too few (and risk missing important events). In fact, the KPIs and features we've added to our BI solution for internal use are now being used many of our external customers as well.

Of course, this is an ongoing process. We develop a new version once every two weeks based on feature requests by our own customer success managers and our external customers. Or, if a dashboard isn't being used, we remove it, try to understand why it was underutilized, and learn from it.

Where Does IT Fit in?

Our IT team was in charge of preparing the information security architecture for our BI portal. At an early stage, we had to decide if we wanted the BI portal to be available outside the office. It made the most sense to do so. After all, our customer success managers need to have access from home, because data is in real-time.

In addition, once we extended it to our external customers, they needed to be able to access the portal from all parts of the world, and in all time zones. Of course, with this decision came many security concerns.

The IT team developed a secure system to allow external access to the portal. Once it was built, they ran stress tests, vulnerability tests, and penetration tests to make sure absolutely nothing could attack and harm the system.

In addition to security management, the IT team was vital in designing and assessing the hardware requirements of the machine that runs the portal's servers.

Results

In time, we became a well-oiled machine, consistently receiving feedback from the customer success managers, assistance for implementation from the analytics team, and security maintenance reports from the IT team.

Usage of the BI platform jumped from 200 visits a day to 1,500 in only six months. (There are about 250 users on the BI platform from the business, finance, and R&D departments at ironSource.) Today, the average customer success manager accesses the platform 15 times a day, even on weekends.

In the end, building our own BI solution was perhaps the best decision we could have made. Not only did it benefit us and improve the way we tracked our own performance, it also gave rise to a new solution for our external customers. This way, they're able to use the same high-level measurements and metrics we're using, all while being able to easily interpret and understand complicated numbers on an easy-to-use dashboard.

What's your BI story? Have you attempted to build your own? Are off-the-shelf solutions meeting your needs? What role is IT playing in your BI efforts? Does adoption of a new BI solution always require a push by leadership? Tell me all about your experiences in the comments section below.

With over a decade of experience in the industry, Sagi Bakshi is the GM of ironSource Digital Solutions. He has been with the company from its earliest days. Sagi built and nurtured ironSource's platform for software developers, transforming it into ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Milady
50%
50%
Milady,
User Rank: Apprentice
10/6/2016 | 4:08:33 AM
Great
It could be really effective in a few year I hope.
TerryB
50%
50%
TerryB,
User Rank: Ninja
10/5/2016 | 1:44:33 PM
IT Defintion
I got a kick out of part that "IT" contributed: infrastructure and security. I would argue your entire company is IT, with exception of certain back office jobs like A/P, accounting, etc. 

Whether you realize it or not, much of that entire project falls under IT, from any traditional definition in a manufacturing type environment. When you are a tech company, I don't think it redefines what IT is. Just means the "business" employees are extremely tech savvy, maybe even having Comp Sci backgrounds. 

For example, I'm a developer at small business unit. I have to manage the entire spectrum of a project like that. I probably would have asked 3rd party experts to assist with security and infrastructure stuff at the scale you are operating. But that doesn't make them "IT" and me not. 
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
IT Strategies to Conquer the Cloud
Chances are your organization is adopting cloud computing in one way or another -- or in multiple ways. Understanding the skills you need and how cloud affects IT operations and networking will help you adapt.
Video
Slideshows
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll