We currently have two entities in sqoop that expose configs, connector and driver ( after it was renamed from framework-driver in SQOOP-1498 ). Driver is not persisted in the repository like the various connectors. The proposal is to change this and treat both of them as configurable entities in the sqoop repository.
- Do not use "null" as an primary key indicator to identify Driver
- Connectors and Drivers expose configs and hence both should be in one table and they can uniquely identified by the primary key
- Make sure the driver and its config upgrade works
- Remove DIRECTION field out of SQ_CONFIG, since it is not applicable to all configurable. For instance Driver has no DIRECTION. Connectors have DIRECTION and it should be only relevant to connectors.
Design & Implementation Details
Note: The work related to removing DIRECTION field out of config was done in a separate ticket after this proposal.
Repository Schema changes
- Rename the SQ_CONNECTOR to SQ_CONFIGURABLE
- Rename relevant FK fields to reflect the SQ_CONFIGURABLE
- Driver is now added a first class row in the SQ_CONFIGURABLE, and has a real persistence ID, instead of a fake Id/ null
Repository Schema upgrade changes
Upgrade code in derby added for renames of table and field names
- Force register the driver as an entity for the "Driver" upgrade path to work across release
Repository API changes
registerDrivermethod, makes a actual call to the repository and persists and entry into the SQ_CONFIGURABLE
findDriversimilarly fetches from the repository