Child pages
  • Sqoop 2 Configurable Entity
Skip to end of metadata
Go to start of metadata

SQOOP-1557

Summary

We currently have two entities in sqoop that expose configs, connector and driver ( after it was renamed from framework-driver  in SQOOP-1498 ). Driver is not persisted in the repository like the various connectors. The proposal is to change this and treat both of them as configurable entities in the sqoop repository. 

Requirements

  • Do not use "null" as an primary key indicator to identify Driver
  • Connectors and Drivers expose configs and hence both should be in one table and they can uniquely identified by the primary key
  • Make sure the driver and its config upgrade works
  • Remove DIRECTION field  out of SQ_CONFIG, since it is not applicable to all configurable.  For instance Driver has no DIRECTION. Connectors have DIRECTION and it should be only relevant to connectors. 

Design & Implementation Details

Note: The work related to removing DIRECTION field out of config was done in a separate ticket after this proposal.

Repository Schema changes

  • Rename the SQ_CONNECTOR to SQ_CONFIGURABLE
  • Rename relevant FK fields to reflect the SQ_CONFIGURABLE
  • Driver is now added a first class row in the SQ_CONFIGURABLE, and has a real persistence ID, instead of a fake Id/ null

Repository Schema upgrade changes

  • Upgrade code in derby added for renames of table and field names



  • Force register the driver as an entity for the "Driver" upgrade path to work across release

Repository API changes

  • registerDriver method, makes a actual call to the repository and persists and entry into the SQ_CONFIGURABLE
  • findDriver similarly fetches from the repository 
  • No labels