Trulia Pursues Data-Driven Transformation, Cloud Migration - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management
09:00 AM
Connect Directly

Trulia Pursues Data-Driven Transformation, Cloud Migration

Trulia transformed its data stack to accommodate real-time high-volume data collection, and to provide customers with recommendations. Now, to gain greater reliability and elasticity, the organization plans to migrate its data operations to AWS.

Digital transformations are at top of mind for many traditional enterprises today as they look to replicate some of the practices that have made upstarts like Uber and Netflix successful.

Real estate web site Trulia isn't that old by corporate standards. Founded in 2004, Trulia could be considered a digital native, since its primary presence has always been on the web. But a lot has changed since 2004. For instance, the iPhone was introduced just three years after Trulia was founded. And the widespread use of mobile phones, apps, and the consumer behavioral information they generate, has created both headaches and opportunities for data engineers, IT organizations, and business analysts.

(Image: Andy Dean Photography/Shutterstock)

(Image: Andy Dean Photography/Shutterstock)

So when Deep Varma joined Trulia as VP of Data Engineering about 2 and a half years ago (the same year that Zillow announced the acquisition of Trulia) Varma's charter was to transform Trulia to be more data-driven -- to use that data to be proactive rather than defensive.

"Our goal remains the same -- how do we provide an amazing experience to our consumer," he said. "With more consumer growth and more engagement of consumers, we were collecting so much more data." Varma brought years of experience at companies like Yahoo, IBM, and a host of startups, to the job.

The ingredients to make that data-driven transformation were already in place, Varma told InformationWeek in an interview. The company had started using big data technology about 6 years ago, including Hadoop and Java.

[Trulia parent Zillow explained its data stack at a recent Strata + Hadoop event. Read Zillow Uses Analytics, Machine Learning To Disrupt With Data.]

But Trulia's new strategy to use that data proactively meant going beyond just providing a real estate search service to consumers. Varma's team wanted to provide more of a recommendation engine, giving consumers the personalized results they wanted before those consumers had even searched for them.

"We have built our own recommender system which surfaces listings to the consumer," Varma told InformationWeek. "We have built click-through models, which helps us measure their success."

Specifically, Trulia is looking to leverage AI and computer vision "to provide unique insights to consumers at their fingertips," Varma said.

In this case, computer vision refers to the effort to train computers to think and act like human beings when it comes to visual information, Varma said. It includes image recommendation systems. To help get Trulia to this corporate vision, Varma augmented Trulia's existing stack of big data technologies.

Since Varma joined Trulia, his team of over 70 data engineers, data scientists, software engineers, and DevOps pros have introduced the use of Apache Kafka, Apache Spark, and microbatching for real-time processing. The organization also uses noSQL databases such as Redis and Apache Solr for search. The team has also transitioned from Python to Cython, which enables writing C extensions for Python. The team also implemented SQL search engine, Presto, and has migrated processing from CPUs (central processing units) to GPUs (graphics processing units), Varma said.

Around the time when Varma joined, Trulia created its own collocated data center. But in 2016, the company embarked on another big change -- migrating to AWS. Varma said Trulia is looking to gain the scalability, reliability, elasticity and innovation that comes with a cloud-based system.  That means moving the operation of the data center away from Trulia's internal IT and into an infrastructure-as-a-service outsourcing scenario.

"We want to keep innovating faster rather than depending on operational people," Varma said.  The goal is to eventually move the entire data engineering operation to Amazon's cloud, but that will take time, he said.

"I believe for a while we will have a hybrid solution," he said. "These are not simple systems when you have millions of consumers."

The biggest challenges in transforming to a data-driven business has been scaling the systems, Varma said. Other challenges have been aligning the teams to move in the right direction, building the personalization platform, and thinking from a consumer point of view.

"I think 2016 has been an amazing foundational year," Varma said. "Our challenges will be greater in 2017 when we are going to make sure our personalization platform extends our footprint across all product development."

Varma's plan for the future of the product includes augmented reality, too, all part of the goal of "making our consumer's experience amazing."

Jessica Davis has spent a career covering the intersection of business and technology at titles including IDG's Infoworld, Ziff Davis Enterprise's eWeek and Channel Insider, and Penton Technology's MSPmentor. She's passionate about the practical use of business intelligence, ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
What Digital Transformation Is (And Isn't)
Cynthia Harvey, Freelance Journalist, InformationWeek,  12/4/2019
Watch Out for New Barriers to Faster Software Development
Lisa Morgan, Freelance Writer,  12/3/2019
If DevOps Is So Awesome, Why Is Your Initiative Failing?
Guest Commentary, Guest Commentary,  12/2/2019
White Papers
Register for InformationWeek Newsletters
State of the Cloud
State of the Cloud
Cloud has drastically changed how IT organizations consume and deploy services in the digital age. This research report will delve into public, private and hybrid cloud adoption trends, with a special focus on infrastructure as a service and its role in the enterprise. Find out the challenges organizations are experiencing, and the technologies and strategies they are using to manage and mitigate those challenges today.
Current Issue
Getting Started With Emerging Technologies
Looking to help your enterprise IT team ease the stress of putting new/emerging technologies such as AI, machine learning and IoT to work for their organizations? There are a few ways to get off on the right foot. In this report we share some expert advice on how to approach some of these seemingly daunting tech challenges.
Flash Poll