Big Data Analytics

big data analytics

Big Data Services

LogicMatter offers a low cost, flexible service to design, host, and manage Big Data Analytics platform and solutions.

We combine the use of traditional (e.g. EDW, ODS) and emerging (e.g. Hadoop MapReduce, Tableau Visualization) analytical tools to operate on big data. The data platform is designed to be cloud-native and build on the powerful, flexible Amazon AWS Cloud platform. It uses Hadoop technologies with the LogicMatter-designed Analytical Data Store (ADS) to capture, store, process, map, transform, and cleanse data. This data discovery platform uses the file-based storage from Hadoop for fast, flexible data processing and combine it with the powerful SQL-based relational integrated data warehouse, the ADS, to enable low-latency, iterative analysis. The big data analytics platform enables the delivery of continuous analytics and visualizations, both real-time and historical, via the increasingly popular Tableau.

The Big Data Analytics platform and solutions can be built specifically to solve complex customer problems as varied as video analytics, clickstream analytics, fraud detection, sales performance analysis, and financial analytics.

Data Source Integration – This platform enables collection, processing, storage, and transformation of both structured and unstructured data exclusively for analytical purposes. It can quickly process a wide variety of unstructured data including documents, text, Excel, XML, weblogs, video, audio, call logs, machine logs (devices, sensors, RFID), clickstream (e.g. video manipulation, website activity), and event data. On the other hand, it can simultaneously process structured data from familiar enterprise data sources such as ERP, CRM, and SQL Databases.

The data collection process is decoupled from transformation and analysis. It allows one to easily add data sources of known and unknown kind without impacting the analysis, a big challenge with today’s analytics solutions. Data transformation is delayed until you need to do the analysis reducing upfront costs and wastage.

Data Storage Platforms – The data platform consists of two primary components – the Hadoop Cluster and the LogicMatter-designed ADS (Analytical Data Service). The flexibility and scalability of Hadoop technology is used to collect both structured and unstructured data as they become available. Very little upfront design is needed. Once the data is collected, it is integrated, pre-processed (as necessary), and stored. The flat file-based storage allows you to scale quickly and handle large amounts of known and unknown data. Hadoop is an integrated, intermediate data source and acts as a feeder to the ADS.

The data from Hadoop is mapped, transformed, and finally cleansed to develop a data model. This model built iteratively and stored in the ADS; forms the basis for powerful analytics. The ADS uses traditional data warehouse technology – ODS, Cubes, and OLAP. Hence, it supports all the powerful, traditional analytical techniques  (reports, dashboards, scorecards, ad-hoc queries, etc.).

This combination of flat-file based (Hadoop) storage and relational SQL-based query decouples data collection from analysis. The data platform is flexible to the extent that you can easily connect any of your favorite visualization tools (such as Excel, PowerPivot, Qlikview). However, we recommend Tableau or Microsoft Power BI

Big Data Solutions

One of the key design tenets of LogicMatter’s Big Data Analytics services is to enable continuous analytics and iterative data discovery, both real-time and historical. With an integrated data discovery platform, you can now connect a visualization tool directly to either Hadoop or ADS to develop the analytics. You can run ad-hoc queries against Hadoop for exploratory analytics and immediate access to data. You can also run ad-hoc queries against the ADS, which has a clean data model to work with. For the standard, canned reports and dashboards, you connect to the ADS to gain historical perspective.

  1. Data Warehouse Augmentation – We help IT/Engineering enhance their traditional data warehouse (DW)  to meet the needs of their stakeholders.  Implement big data operational data stores (ODS) to speed up analytics through data aggregation. Expand DW capacity  big data technologies like Redshift and Hadoop to offload less frequently used data and improve DW performance.
  2. Data Refinery / Hub – We use big data platforms to be the operational data stores (ODS) for landing and processing zone for data from many diverse sources to create a  “data hub”, before it is pushed downstream for low-latency analytics (most likely to an analytical datastore (ADS)).  ELT pattern delivers huge cost savings and scales up the enterprise analytics process.
  3. Consumer/Customer 360 Degree View – The 360 Degree View blends a variety of streaming, operational and transactional data sources to create an on-demand analytical view across multitude of dimensions in a consumer life cycle management system.
  4. Monetize from data feeds – We use big data platform to enrich datasets and create mashups of datasets that are needed to support downstream applications and/or outside partner systems. Deliver Data as a Service (DaaS) to 3rd party systems that need to leverage powerful data processing and embedded analytics to generate a new revenue stream for the enterprise.
  5. Harnessing Machine and Sensor Data – Collect multi-device trend and alarm history data to provide multi-dimensional views of the environment/things from high volume data using devices like sensors, routers, and set-top boxes.
  6. Big Data Lake – We help companies who need to dump massive data into big data stores, with an intent to use them at a later point or make available for data mining or machine learning systems.
  7.  Analytics HubBig data offers a new set of tools for optimizing analytics and using machine-learning algorithms for predictive analysis.
  8. Big Data As A Service – As companies use SaaS, IaaS, IoT and Legacy data for their decision making. The trend to use new asynchronous integration methods that would need data hubs for the next generation mobile, tablet and web applications rises. Enterprises are tapping into big data platform as a shared database service, to be provisioned across a number of application development teams for data ingestion and access.