Master Data Management (MDM), the distributor of Datameer in southern Africa, has announced the introduction of Datameer 5.0 with Smart Execution.
Datameer 5.0 with Smart Execution is a new patent-pending technology that intelligently and dynamically selects the best-of-breed compute framework at each step in the big data analytics process.
Datameer is transforming the analytics process with its new Smart Execution engine by examining dataset characteristics, analytics tasks and available system resources to determine the most efficient compute framework for the task at hand.
A new series of computation frameworks complementary to MapReduce have joined the Hadoop ecosystem to address the full range of data analytics needs. This proliferation of tools creates complexity, as most frameworks are optimised for either small or large data, or for particular analytics tasks, meaning end-to-end data pipelines require switching between multiple solutions and computation frameworks. Often, organisations try to integrate these solutions themselves, resulting in home-grown analytics systems that are difficult to maintain and require extensive IT resources.
“Datameer 5.0 with Smart Execution provides more intelligence and delivers more than one way to action big data within an organisation. Furthermore, this solution future-proofs organisations against changes and new technology. As Hadoop evolves, Datameer will ensure optimal performance by selecting the best method to run each query,” says Gary Allemann, MD of Master Data Management.
Datameer Smart Execution leverages and simplifies the rapidly evolving Hadoop ecosystem by intelligently and dynamically selecting the best compute framework for each step of an analytics job, orchestrating multiple compute frameworks as necessary. Once that selection is made, Smart Execution uses a snapshot of total and currently available system resources to intelligently allocate and run each concurrent workload for maximum performance.
“Datameer is addressing an emerging bottleneck with big data computing,” says Tony Baer, principle analyst at Ovum. “As Hadoop becomes multi-purpose, it is drawing a wider variety of workloads. Some utilise large clusters more efficiently than others. With Smart Execution, Datameer is introducing optimisation that caches and directs the right workloads to the right compute framework.”
Smart Execution will execute analyses on large data in a Hadoop cluster using Apache Tez, an optimised form of MapReduce, while small data analysis will be executed on a single Hadoop node or using in-memory technology. This selection is completely transparent to the end user and does not require IT assistance or extra hardware or software. Most importantly, Smart Execution can easily incorporate new advances in the Hadoop ecosystem as they become available, including technologies such as Spark, as they become enterprise ready.
“We are thrilled that Smart Execution has broadened access to new Hadoop ecosystem technologies like Apache Tez and YARN,” says John Kreisa, vice president of strategic marketing at Hortonworks. “Datameer’s solution will be the first in its class to fully leverage Tez’s unique capability of supporting mixed workloads with enterprise grade performance, and will free users from having to determine what compute framework is best for their big data analytics jobs.”