This document details the internals of how the sqoop-server works.
This document is relevant to the release 1.99.5. Further changes can happen in future releases
Sqoop Tomcat Server
- Sqoop-server uses tomcat web server, it is very bare bones.
The main entry point is the TomcatToolRunner, it bootstraps the tomcat and loads all the sqoop related classes into its class path. It is invoked from the bash script .
- The main hook for the sqoop server to start is this entry in the web.xml. Tomcat invokes it callbacks as it bootstraps and we use the contextInitialized callback to initialize all the related code.
- The sqoop server is represented by the java class
- SqoopServer.initialize() is the called from the
- SqoopServer.destroy() is called when the tomcat server is shutdown
- We have servlets for each Sqoop Entity that is exposed via the REST API in web.xml. Refer to Sqoop 2 (1.99.4) Entity Nomenclature and Relationships for more details on the supported Sqoop Entities.
- They receive the requests from the web or the Sqoop Client and process the request.
- They delegate most of the business logic to their corresponding
All the rest APIs supported via the sqoop-server are documented here: http://sqoop.apache.org/docs/1.99.4/RESTAPI.html#id1
- There is an authentication filter to authenticate all request.
- There are two authentication mode supported: simple and Kerberos, which could be set in the sqoop.properties.
Sqoop Request Handlers
Each Sqoop Servlet has its corresponding handler class that handles the request for that servlet. It then internally calls the internal sqoop core/ common code.
- Sqoop Client is represented by the java class
- It has wrapper
ResourceRequestclasses for each sqoop entity, they encapsulate the request/postBody parameters to be sent in the request. Refer to Sqoop 2 (1.99.4) Entity Nomenclature and Relationships for more details on the supported Sqoop Entities.
It used the bare bones
HttpURLConnectionobject to make requests to the Sqoop-server.
SqoopClient used to use jersey REST client for making tomcat requests. Recently it was switched to Hadoop-auth/SPENGO for adding Kerberos support that are documented here
Run command to start Sqoop Client.
- In Kerberos Authentication mode. Kinit is required to set Kerberos environment.