Note

The following information is relevant ONLY for releases before 1.99.4

 

Sub-Systems

  • Configuration: Manage system configuration
  • Repository Manager:** Create Repository Schema** Create or edit connectors** Create or edit Connections
    • Create or edit Jobs
  • Connector Manager** Verify connectors** Register new connectors** Edit connectors (enable/disable)
  • Job Manager:** Submit new jobs to MR** Get Progress of Jobs from MR** Kill specific job when requested

Initialization

  • Configuration** Initialize configuration system
    • Initialize logging infrastructure
  • Repository Manager** Create repository Schema
    • Verify if the schema is accurate
  • Connector Manager** Register connectors
    • Verify that all connector information is accurate
  • Job Manager** Verify the status of jobs that were running at the time of last shutdown

Metadata

Component

Resource

Metadata

Connector

sc2, sc3

- Information needed to create connection
- Information needed to create job

Connection

sx2, sx3

- Which connector backs this connection
- Connection identifier

Job

sj2, sj3

- Which connection the job is associated with
- Connection metadata
- Job configuration metadata

Connector Metadata

  • Information needed by the connector to create a connection
  • Information needed by the connector to create a job, given a connection

Resources

Green resources are currently implemented in current (2012-10-17) version of sqoop2.

Id

Resource

Method

Action/Parameters

sv1

/version

GET

Return a list of supported protocol versions. Current list includes “1” only.

sc1

/v1/connector

GET

Return a brief list of available connectors.

sc2

/v1/connector/all

GET

Return a list of connectors with each connector’s metadata.

sc3

/v1/connector/cid

GET

Return the metadata of the connector identified by cid

sx1

/v1/connection

GET

Return a brief list of available connections.

sx2

/v1/connection/all

GET

Return a list of available connections with metadata about each.

sx3

/v1/connection/xid

GET

Return the metadata about connection identified by xid

sx4

/v1/connection

POST

Create new connection.

sx6

/v1/connection/xid

PUT

Update connection identified by xid.

sx5

/v1/connection/xid

DELETE

Delete connection identified by xid.

sj1

/v1/job

GET

Return a brief list of available jobs.

sj2

/v1/job/all

GET

Return a list of available jobs with their configuration metadata.

sj3

/v1/job/jid

GET

Return the metadata about job identified by jid

sj4

/v1/job

POST

Create new job.

sj5

/v1/job/jid

PUT

Update job identified by jid.

sj6

/v1/job/jid

DELETE

Delete job identified by jid.

ss1

/v1/status

GET

List of all jobs that are running

ss2

/v1/status/all

GET

List of all jobs that are running with details about each job’s progress

ss3

/v1/status/jid

GET

Details of th job identified by jid

st1

/v1/jobcontrol

POST

Starts/Stop/Kill a job. “JOBID” → Sqoop specific Job ID

sn1

/v1/notification

POST

Notification support for job progress

  • No labels