LogicMatter offers a low cost, flexible service to design, host, and manage Big Data Analytics platform and solutions.
We combine the use of traditional (e.g. EDW, ODS) and emerging (e.g. Hadoop MapReduce, Tableau Visualization) analytical tools to operate on big data. The data platform is designed to be cloud-native and build on the powerful, flexible Amazon AWS Cloud platform. It uses Hadoop technologies with the LogicMatter-designed Analytical Data Store (ADS) to capture, store, process, map, transform, and cleanse data. This data discovery platform uses the file-based storage from Hadoop for fast, flexible data processing and combine it with the powerful SQL-based relational integrated data warehouse, the ADS, to enable low-latency, iterative analysis. The big data analytics platform enables the delivery of continuous analytics and visualizations, both real-time and historical, via the increasingly popular Tableau.
The Big Data Analytics platform and solutions can be built specifically to solve complex customer problems as varied as video analytics, clickstream analytics, fraud detection, sales performance analysis, and financial analytics.
Data Source Integration – This platform enables collection, processing, storage, and transformation of both structured and unstructured data exclusively for analytical purposes. It can quickly process a wide variety of unstructured data including documents, text, Excel, XML, weblogs, video, audio, call logs, machine logs (devices, sensors, RFID), clickstream (e.g. video manipulation, website activity), and event data. On the other hand, it can simultaneously process structured data from familiar enterprise data sources such as ERP, CRM, and SQL Databases.
The data collection process is decoupled from transformation and analysis. It allows one to easily add data sources of known and unknown kind without impacting the analysis, a big challenge with today’s analytics solutions. Data transformation is delayed until you need to do the analysis reducing upfront costs and wastage.
Data Storage Platforms – The data platform consists of two primary components – the Hadoop Cluster and the LogicMatter-designed ADS (Analytical Data Service). The flexibility and scalability of Hadoop technology is used to collect both structured and unstructured data as they become available. Very little upfront design is needed. Once the data is collected, it is integrated, pre-processed (as necessary), and stored. The flat file-based storage allows you to scale quickly and handle large amounts of known and unknown data. Hadoop is an integrated, intermediate data source and acts as a feeder to the ADS.
The data from Hadoop is mapped, transformed, and finally cleansed to develop a data model. This model built iteratively and stored in the ADS; forms the basis for powerful analytics. The ADS uses traditional data warehouse technology – ODS, Cubes, and OLAP. Hence, it supports all the powerful, traditional analytical techniques (reports, dashboards, scorecards, ad-hoc queries, etc.).
This combination of flat-file based (Hadoop) storage and relational SQL-based query decouples data collection from analysis. The data platform is flexible to the extent that you can easily connect any of your favorite visualization tools (such as Excel, PowerPivot, Qlikview). However, we recommend Tableau or Microsoft Power BI
One of the key design tenets of LogicMatter’s Big Data Analytics services is to enable continuous analytics and iterative data discovery, both real-time and historical. With an integrated data discovery platform, you can now connect a visualization tool directly to either Hadoop or ADS to develop the analytics. You can run ad-hoc queries against Hadoop for exploratory analytics and immediate access to data. You can also run ad-hoc queries against the ADS, which has a clean data model to work with. For the standard, canned reports and dashboards, you connect to the ADS to gain historical perspective.