Due to phenomenal growth in the usage of smart devices, all carriers are experiencing Data Tsunami, whereby, the data is growing by over 20TB per week. Also, due to location intelligence, real time ad targeting is required. Thus an inexpensive solution was to be architected to avoid performing entire business analytics on Oracle and expensive storage like EMC. Real Time Analytics was also required for location based recommendation
SOLUTION
Enterprise Data Warehouse, Clickstream Data Analysis, Real Time Analytics. Architected the data warehouse staging on Hadoop/MapReduce using 100 node inexpensive Linux servers to process 2B transactions per day and 100TB of data.
Used Hive to provide batch reports, thereby, reducing load on the Oracle DW production instance.
Transformation and Cleaning services were designed to be invoked before the data was stored in Hadoop/MapReduce.
The solution helped in integrating data from various sources like Venturi video streaming, Techtronix probes, Call data record, Clickstream data.
SUCCESS CRITERIA & BUSINESS VALUE
TCO was reduced by 40% by providing a hybrid solution using Hadoop / MapReduce. Java recommendation engine was developed to provide real-time recommendations to mobile users providing a new revenue stream.