May 25, 2016

Big Data in Retail

Will Big Data help unfold the next retail revolution?

Big Data in Retail

Big Data — a simple word that is used to represent the complex explosion of data available today. There are many definitions and interpretations of what Big Data is, but on a basic level, it is any data that cannot be tamed by conventional data management techniques.


Data has always been available and companies have been curating, analyzing and using it for many years. The only difference now is that there are multiple ways to collect, store, order and make meaningful decisions from all the structured and unstructured data available. The internet and the use of smart technologies like mobile phones, tablets & wearables have made more volume and variety of information available — what we as customers like, what we buy and how we act. This is a treasure trove of customer data for any company or sector — like Retail. Big Data can provide valuable insights about customer habits which enables retailers to fine-tune their business processes (like products, supply chain, pricing, store designs, marketing, etc.) making them more efficient and profitable.

Retail Industry landscape


"Retail is going to change more in the next 5 years than the last 50 — FITCH

The Retail industry is one of the biggest industries in the US with two-thirds of the US economy depending on the retail sector. The retail giant Walmart is the world's largest company by revenue and retail sector is the second largest employer in the US after healthcare. Global retail sales are expected to grow by 3.2% in 2016 and up to 3.4% by 2018.

How useful is Big Data for a retailer?

L arge and small retailers alike have been collecting and using structured data to predict customer demands and trends for many years. However with vast improvements in technology and customers embracing mobile technology into their day to day lives, a new kind of data has emerged — known as unstructured data. All retail and e-commerce stores now have huge volume and variety of this unstructured customer data — tweets, photos, videos, location data, customer feedback and more available to them. This structured and unstructured data together commonly known as Big Data, when analyzed properly using the right tools can provide very valuable insights for a retailer. When properly analyzed, this Big Data can also help make informed decisions in real time making a retailer more competitive and in trend with customer needs.


Drivers of Big Data in Retail

T here are two major drivers of Big Data in retail today — Technology and Social media/networks.


  • Technology has emerged as the major platform using which Big Data is collected and processed. Retailers willing to embrace technology and innovate using it have seen more success and increase in revenue and sales. The move to interconnected and smarter devices have been a potential boon to retailers and have helped collect unstructured data in a big way. Since every interaction with a smart device leaves a digital trace, retailers are able to understand and collect data about the customer by following their paths digitally.

  • Social Media is another major driver of Big Data. There are hundreds of thousands of tweets, photos and messages posted every minute on social media, generating huge volume of valuable data from social sites alone.

  • Leveraging customer data from social networks and e-commerce site activity helps retailers find real time trends and customer sentiments about products and services. Retailers have also seen great success by engaging with their customers through social channels. "43% percent of social media users have purchased a product after sharing or liking it on Pinterest, Facebook, or Twitter."

Data is available to retailers, but how advantageous is it?

Predict trends and Forecast demand : With so much data available now, it is easier than ever for retailers to use this data to predict future trends and the demand associated with them. Retailers do not have to speculate about customer demands anymore — rather they use the data available to them to make informed decisions and improve their decision making. Some major advantages are:

  • Makes the retailer leaner with easier supply chain management — by understanding what products people want and buy, the retailer can forecast demand, stock merchandise accordingly and also stock the right number of products at the right channels.

  • Helps stay competitive and up to date with the latest trends in the industry — becoming an industry leader who brought the trend to the market and not a follower who is scrambling to meet the demand created. Also, by analyzing past purchases using data metrics and profiling customers, relevant recommendations can be made to the customer which helps cut down on time wasted in product discovery. This keeps the retailer competitive and profitable.
  • Spot issues with proper data analysis — identify problem areas and the costs associated with them, discover customer pain points and ways to resolve it.

Personalized customer experience : One of the biggest challenges for a retailer to stay competitive is to create a more personalized experience for the customer. Customers now crave products and experiences tailored to their individual needs. They expect better buying experiences and retailers who can provide this have an edge over their competition. Relevance is the highest currency for a customer today.


Most retailers have an online presence along with their brick and mortar stores. Using the combined data from both these sources, loyalty programs, real time analytics and in — store technologies like beacons, WiFi, smart shelves, store apps, mobile payment options, etc., the retailer can take action in minutes and provide products tailored to the customer's needs, price optimization options, inventory details about the product the customer wants and the best and fastest channel to receive it.


Extreme personalization and contextual awareness is being demanded by consumers now. Retailers who can adopt and adapt faster to the disrupting forces can only stay competitive. For example, the technology company Aagnaa has made shopping an experience for the customer with interactive mirrors which show you how you would look in a particular dress you want without physically trying it on.


Adoption of new payment technologies like paying with apps, wearables, PayPal, etc., provide convenience and cost benefits to customers and increases engagement and operational efficiency of retailers. Most major retailers now allow customers to pay using Apple Pay or Android Pay, enhancing the shopping experience. For example, Starbucks allows customers to pay for their purchases using their app and also sends promotions and discounts. One of the major retailers — Kohls, uses its app to keep a customer's promotions, coupons and discounts so they don't have to carry physical paper coupons around. It also allows the customer to search and track the product they want when it is not available in store.


Role of Technology


Using the insights gathered from Big Data, retailers can then use various technology tools to create unique and personalized shopping experiences for the customer. Technology tools like beacons, WiFi, Augmented reality (AR) and Virtual Reality (VR), interactive screens, in-store kiosks, etc., are now being used by various retailers.



  • Retail chain Nordstrom encourages its customers to pin their favorite products on Pinterest and displays the most popular items in its brick and mortar store — this shows the customers shopping in store what the most popular items are and helps the store associates to stock inventory based on popular demand.

  • IKEA- one of the top furniture retailers, uses image recognition and AR on their mobile catalog. Customers can see 360-view displays of furniture and even virtually place furniture in their homes and choose the products that work best without going into a store. It now has a virtual reality app — IKEA VR experience that provides a virtual reality kitchen experience on Valve.

  • The luxury retailer Rebecca Minkoff is well known for its adoption of latest technology trends to enhance the customer experience. The Rebecca Minkoff official cardboard pop-up VR viewer allows the customer to view the designer's runway collection using VR. The retailer uses beacons, RFID technology and interactive displays to improve the customer experience at its brick and mortar stores.
  • Pharmacy retail chain Walgreens is testing Google's Project Tango 3D for AR-enabled maps in store so a customer can easily navigate the aisles and find what they need in half the time.

  • Beauty and makeup retailer Sephora uses AR and facial recognition on its app to allow customers to choose the right makeup which will suit them. It also uses beacons in store to send personalized promotions and offers to its customers.

  • BMW created an interactive window display in New York to show the customers their vision of clean and environment friendly BMW i cars for the future.

  • British retail chain TopShop piloted virtual fitting rooms in some of their stores using which customers could choose the garments they liked and see how they would look wearing them.

  • Kate Spade — an American fashion and accessory brand uses social media websites like Pinterest, Tumblr, Facebook and Twitter to find the trending and most liked pieces of their brand to improve their design and provide new and fashionable products every time. The Kate Spade Pop up Stores featuring interactive displays allowed customers to choose the products they wanted, pay from their mobile device and delivered to them within an hour.

Our Take

Despite the huge advantages a retailer has to gain from Big Data and proper data management, many retailers are still missing out on converting this opportunity to their benefit.


An estimated 80% of data from in-store transactions, customer & product information and e-commerce are not considered for analysis.

With the volume and variability of data available to retailers today, they face immense challenges in storing, processing and making meaningful decisions based on that data. Only by understanding each customer in depth using data analytics, retailers can provide a personalized & creative experience for the customer — resulting in increased sales and revenue.


The data collected by retailers has to be done in a fair and transparent manner and support the data privacy needs required by regulations and customers. Nowadays, the government and privacy advocates are making attempts to regulate the way a person's personally identifiable information (PII) is collected and used by retailers. Innovation in technology payments have led to increased security concerns prompting retailers to invest in security solutions. Around 10% of customers stop shopping and 36% shop less frequently after a data breach — it therefore becomes important for retailers to integrate security & privacy processes and concepts within their data infrastructure.


Retailers who will ultimately win will be those who embrace the changing dynamics of their customers, blend the physical and digital and use the technology available to them to re-invent and innovate the shopping experience of a customer. All this is only possible by translating meaning from the Big Data available and using it to improve decision making.