BUSINESS OBJECTIVES
Customer wanted to reduce the total cost ownership for the data warehouse environment by migrating to open source Hadoop Big Data architecture
SOLUTION
Architected the Hadoop infrastructure, capacity, security and data strategy. Configured Hive, Scoop and other administration tools. Each existing ETL stored procedure touching the raw LZ Omniture tables were converted to a series of Map-Reduce jobs. Raw Omniture files were stored in Hadoop HDFS in a directory structure organized by TPID, Year, Month and Day – to enable processing / re-processing for different periods of time. Output of Omniture ETL Workflows were exported from Hadoop back into existing DB2 ADS and DataMarts .An export utility was built for R1.0 to bring selective datasets from Hadoop into DB2 scratchpad for ad hoc analysis, and for advanced analytics using traditional SQL methods
SUCCESS CRITERIA & BUSINESS VALUE
Infometry team worked closely with an Online travel industry company in helping them to solve Big Data Analytics problem which involves designing and architecting the Hadoop/MapReduce solution to replace their existing DB2 based data warehouse platform which resulted in the savings of 8.5 million over 2.5 yrs and also enable customer to analyze large volume data leveraging Hive/Hue.