Over the past few years, there has been a rise in database systems and their tools owing to the fact that big data and machine learning fields are growing parallelly. There is no dearth in the variety of tools available to users to handle data systems. In addition, the progress of distributed file systems and cloud computing have made an impact on the way database systems work.
Platforms such as Apache Hadoop and Apache MapReduce have witnessed stellar developments in the recent years, to effectively meet the demand of computing enormous amounts of data. In fact, Hadoop has grown so big that the framework itself is designed into a software library that offers a host of database tools. The applications of these tools span from cloud computing to data mining, and now has made its way into ML.
In this article, we discuss two extensions of Hadoop known as Cassandra and Hive, and look at how their functions help with ML.
Apache Cassandra
Cassandra is a distributed database management system developed by Apache Software Foundation in 2008. It uses techniques based on NoSQL and is an open source software. The key features of the software are:
- Decentralised system
- Distributed deployment
- High application scalability
- Fault tolerance.
- Tunable consistency
- MapReduce Support
- A separate query language called Cassandra Query Language
It manages data in the form of clusters which are interconnected to thousands of nodes spread across data centres. It is also known as ‘column-oriented database’ in NoSQL, wherein the data is stored in a column-by-column fashion in contrast to the row-based approach in traditional database systems. This is the reason it has lesser I/O operations for storing data.
Cassandra has mainly been used in big data applications which use real-time data such as those from sensor components or from social networking websites. In addition, Cassandra has a decentralised architecture, which means function modules such as data partitioning, replication, scaling and failure handling are present separately, and work in tandem. This means any node can take up any data processing operation.
Cassandra’s key advantage lies in its ability to run on less powerful hardware. The tool performs read/write functions quickly on hundreds of gigabytes of data. The architecture behind Cassandra is loosely based on Amazon’s Dynamo, which implements a key-value database system. Since ML involves iterative tasks with significantly large data, Cassandra can be the perfect tool for executing large datasets with good throughput.
Apache Hive
Hive is primarily a data warehousing tool which is based and built on the features of Hadoop. It uses a SQL-like syntax for queries in managing data to and fro from the database. The first official and stable version of the software was released in 2010 by Apache. Mainly used for data analysis, Hive supports functions such as data summarisation and ad-hoc querying conveniently. Hive has the following features:
- Easy data access through SQL
- Support for a variety of data formats
- Distributed file storage system
- Query execution through data processing tools
- Query retrieval
Originally developed as a translation layer for Hadoop MapReduce, it uses its SQL-like language to interpret direct acyclic graphs in MapReduce therefore reducing the burden of writing long codes to handle data in the storage systems. Furthermore, Hive supports popular programming languages such as Java, Python, C++ and PHP.
Hive is not exactly a database system, and so it is generally not used in critical systems which involve real-time transactions such as bank transactions or online ticketing.
One-On-One Comparison
| Cassandra | Hive | |
| Function | Distributed database system that has data stored in clusters. | Data warehousing tool which relies on features of Hadoop |
| Website | http://cassandra.apache.org/ | https://hive.apache.org/ |
| Current Stable Release | 3.11.2 / February 19, 2018 | 2.3.0 / July 19, 2017 |
| Written in | Java | Java |
| Supported Operating Systems (OS) | Windows, OSX, Linux | Almost all OS |
| Open source availability | Yes | Yes |
| Supported Programming Languages | Java, JavaScript, Python, Perl, Ruby, Scala, C++, Haskell | Java, Python, PHP, C++ |
| MapReduce Support | Yes | Yes |
| Query Language | CQL | Specific SQL statements |
| API Support | Through CQL | Through JDBC, ODBC |
Comments:
Since both Cassandra and Hive take on huge amounts of data, both of them look ideal for ML applications. ML algorithms are usually iterative in function. These iterative computations demand higher power as well as quick data handling capabilities. Also, before using these software, care should be taken that the data is relevant as well as of top quality for the ML project.
It should be noted Cassandra and Hive are specifically used in big data applications. Therefore, ML must deal with ramifications involved in big data carefully without compromising user experience. Contrastingly, for ML, more data means better output that gives useful insights into the problem.
The post Why Cassandra And Hive Are The Best Prospective Big Data Tools For ML appeared first on Analytics India Magazine.