The Digital Analytics Divergence (The Great Divide)

There has always been a separation between those that HAVE and those that don’t. In the fast paced digital data/analytics world this is becoming an increasing wide separation.

Speaking from the retailing perspective, there have always been those power houses that could manage their own data storage, data analytics, data handling and architecture. Walmart, Target Stores, Kroger (albeit a Grocer), Home Depot, Lowe's, and do not forget Amazon. These organizations can afford and manage complex contracts with external suppliers to assist them with massive programs. These programs are handled internally with the use of middleware and cloud services; but always managed by internal resources. These companies want IT and CAO/CIO to help drive the internal resources to keep this in-house.

Now the divide, there are many more small to moderate-sized retailers that need to compete in this new digital data/analytics world. You could say, with the emergence of the new shopping generations (millennial’s, post-millennials) keeping up with the trends is more important than ever. The big box stores are designing new software to handle digital shelf and space management programs (In theory these keep the online presentation similar to in-store), which could help in in-stock/replenishment and basket trends analysis. The smaller chains could take advantage of these types of programs too, but do not have the internal resources or budget to maintain an in-house team. You’re talking space and hardware, loading and refreshes, EDW support for production utilities as well as analytics and marketing.

So, as this divide continues to widen as the big shops continue to break new ground with strong partnerships with IBM, Teradata, Apache Hadoop, SAP, and SAS to name a few. Many of these partnerships are enterprisewide and are scaled to grow with at an increasingly fast pace.

The smaller shops have not been left out by any means, but the selection of a good partner can be tougher. These partners have to carry much more of the burden in support of the retailer, such as technology, version updates, security (data and transactions), and data storage. Many of the smaller retailers can't just bring in the middleware (data handling, data mining, data merging) and stack it on top of their data architecture. That can defeat the purpose of building a partnership with a nimble third party. Most companies are moving in the direction of the cloud, where their data is stored outside the traditional framework, where the power of the third-party software can really be utilized.

Both of these situations have pros and cons as you would suspect. Building and storing/maintaining software that continually needs to be updated is difficult even with a substantial partner; not to mention expensive. Outsourcing the technology storage/maintenance and analytics even if it is just the software (where you do the analytics internal off their software) can be hugely advantageous for smaller companies. The responsibility for version control is on the vendor, both backward migration and forward strategies.

I have found that smaller technology partners can be more nimble than some of the larger organizations. They can also accommodate more individual client nuances, as some retailers put more emphasis on replenishment while others tackle merchandise cross selling (market basket optimization). Larger companies seem to have a solution that works great for A or B, but very little tailoring is possible.

One drawback to the Software-as-a-Service solution, is that as these packages become more and more complex very specialized training can be required. So a specialized role needs to be created, which can be troublesome in the best of cases.

Now, everyone has their own war stories with pros and cons. I was not attempting to cover every case imaginable. I’ve worked with very large organizations that used these methods to their advantage. I’ve worked with smaller retailers that had no idea where to start, but clearly knew where they wanted to be in five years. I’ve seen the good, the bad and the ugly of both methods; but in every case the retailer came out ahead.

So, whichever solution you choose to take, please take advantage of the data at your disposal. It is time that we begin harnessing big data/big analytics and see a clear ROI on the other side. It is a great time to be in analytics!