Apache Sqoop has graduated as a Top-Level Project of the ASF
I received a message now from Sally Khudairi, in behalf of the Apache Software Foundation announcing that Apache Sqoop is officially a Top-Level Project :
Apache Sqoop is an Open Source big data tool used for efficient bulk transfer between Apache Hadoop and structured datastores.
Forest Hill, MD — The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of nearly 150 Open Source projects and initiatives, today announced that Apache Sqoop has graduated from the Apache Incubator to become a Top-Level Project (TLP), signifying that the Project’s community and products have been well-governed under the ASF’s meritocratic process and principles.
Designed to efficiently transfer bulk data between Apache Hadoop and structured datastores such as relational databases, Apache Sqoop allows the import of data from external datastores and enterprise data warehouses into Hadoop Distributed File System or related systems like Apache Hive and HBase.
“The Sqoop Project has demonstrated its maturity by graduating from the Apache Incubator,” explained Arvind Prabhakar, Vice President of Apache Sqoop. “With jobs transferring data on the order of billions of rows, Sqoop is proving its value as a critical component of production environments.”
Building on the Hadoop infrastructure, Sqoop parallelizes data transfer for fast performance and best utilization of system and network resources. In addition, Sqoop allows fast copying of data from external systems to Hadoop to make data analysis more efficient and mitigates the risk of excessive load to external systems.
“Connectivity to other databases and warehouses is a critical component for the evolution of Hadoop as an enterprise solution, and that’s where Sqoop plays a very important role” said Deepak Reddy, Hadoop Manager at Coupons.com. “We use Sqoop extensively to store and exchange data between Hadoop and other warehouses like Netezza. The power of Sqoop also comes in the ability to write free-form queries against structured databases and pull that data into Hadoop.”
“Sqoop has been an integral part of our production data pipeline” said Bohan Chen, Director of the Hadoop Development and Operations team at Apollo Group. “It provides a reliable and scalable way to import data from relational databases and export the aggregation results to relational databases.”
Since entering the Apache Incubator in June 2011, Sqoop was quickly embraced as an ideal SQL-to-Hadoop data transfer solution. The Project provides connectors for popular systems such as MySQL, PostgreSQL, Oracle, SQL Server and DB2, and also allows for the development of drop-in connectors that provide high speed connectivity with specialized systems like enterprise data warehouses.
Craig Ling, Director of Business Systems at Tsavo Media, said “We adopted the use of Sqoop to transfer data into and out of Hadoop with our other systems over a year ago. It is straight forward and easy to use, which has opened the door to allow team members to start consuming data autonomously, maximizing the analytical value of our data repositories.”
Availability and Oversight
Apache Sqoop software is released under the Apache License v2.0, and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project’s day-to-day operations, including community development and product releases. Apache Sqoop source code, documentation, mailing lists, and related resources are available at http://sqoop.apache.org/.
Final Thoughts
So, if you are a Hadoop user, you should consider to include to Sqoop in your current or in your infrastructure, because for its own features, it will be a critical part of the architecture. So, download it, install it and test it.
Happy Hacking !!!
- Marcos Ortiz Valmaseda
- Data Engineer at UCI
- Linkedin’s profile
- @marcosluis2186
- Big fan of Big Data, NoSQL and PostgreSQL
10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS…
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci
