News
- Sqoop successfully graduated from the Incubator in March of 2012 and is now a Top-Level Apache project: https://blogs.apache.org/sqoop/entry/apache_sqoop_graduates_from_incubator
- Press release: https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces25
What is Apache Sqoop?
Apache Sqoop is a tool designed for efficiently transferring bulk data between Apache Hadoop and structured datastores such as relational databases. You can use Sqoop to import data from external structured datastores into Hadoop Distributed File System or related systems like Hive and HBase. Conversely, Sqoop can be used to extract data from Hadoop and export it to external structured datastores such as relational databases and enterprise data warehouses.
Sqoop provides a pluggable connector mechanism for optimal connectivity to external systems. The Sqoop extension API provides a convenient framework for building new connectors. New connectors can be dropped into Sqoop installations to provide connectivity to various systems. Sqoop itself comes bundled with various connectors that can be used for popular database and data warehousing systems.
Getting started
Presentations
- 13 June 2012, A New Generation of Data Transfer Tools for Hadoop: Sqoop 2, Link to Hadoop Summit video by Bilung Lee and Kathleen Ting at Hadoop Summit 2012, San Jose Convention Center, CA.
- 4 April 2012, Sqoop: The Early Days by Aaron Kimball at the Sqoop Meetup, Cloudera, Palo Alto, CA.
- 4 April 2012, Highlights of Sqoop 2 by Arvind Prabhakar at the Sqoop Meetup, Cloudera, Palo Alto, CA.
- 8 November 2011, Integrating Hadoop with Enterprise RDBS by Arvind Prabhakar at Hadoop World 2011, Sheraton NYC.
- 7 November 2011, Habits of Effective Sqoop Users by Kathleen Ting at the Sqoop Meetup, Sheraton NYC.
- 7 November 2011, Sqooping 50 Million Rows a Day from MySQL by Eric Hernandez at the Sqoop Meetup, Sheraton NYC.
- 7 November 2011, Scratching Your Own Itch by Joey Echeverria at the Sqoop Meetup, Sheraton NYC.
- 7 November 2011, Inaugural Sqoop Meetup Minutes from the Sqoop Meetup, Sheraton NYC.
- 21 September 2011, Apache Sqoop: A Data Transfer Tool for Hadoop by Arvind Prabhakar at the Bay Area Hadoop User Group, Yahoo!, Sunnyvale, CA.
Release 1.4.1-incubating
Resources
Community
Developers
- Environment Notes
- Proposed Design
- Sqoop 2 - Proposed Design
- Sqoop 2 Highlights - Presentation
- Sqoop 2 Highlights - Blog post
- Sqoop 2 JIRAs