Page History

Overview

There are presently three ways to issue HCatalog DDL commands:

Command line interface
REST APIs (upcoming)
HiveMetaStore Client

Presently, java developers go through the Hive meta store (HMS) client interface to issue HCatalog DDl commands. Though the HMS client interface is public, it is not intended for public users. According to the hive user mailing list, the HMS client is not a public API and is subject to change in the future. So, it will be a good idea to have a java APIs in HCatalog which will provide a protect users from the changes made to the hive meta store client. Also, the under the covers either the Rest APIS or the hive metastore client can be used to provide end users with the required data.

Design

Image Added

New Classes

– This document is a work in progress.

Overview

Templeton provides a REST-like web API for HCatalog and related Hadoop components. Developers can make HTTP requests to the Templeton web server to execute HCatalog DDL commands. With the REST APIs in place for HCatalog DDL commands, it is desirable to have a JAVA APIs in HCAT which can help end users to execute DDL commands without using CLI.

Design Image Removed

...

HCatClient

The HCatClient is an interface abstract class containing all the APIs permitted HCatalog DDL commands.

Code Block


 package org.apache.hcatalog.api;

import java.util.List;

import org.apache.hcatalog.common.HCatException;
import org.apache.hcatalog.mapreduce.HCatDatabaseInfo;
import org.apache.hcatalog.mapreduce.HCatPartitionInfo;
import org.apache.hcatalog.mapreduce.HCatTableInfo;

/**
 * The Interface HCatClient containing APIs for HCatalog DDL commands.
 */
public interface HCatClient {

    /**
     * Gets the database like.
     *
     * @param regex The regular expression. Providing "*" would retrieve all the names
     *              of the databases.
     * @return The list of all the database names.
     * @throws HCatException
     */
    public List<String> getDatabaseLike(String regex) throws HCatException;

    /**
     * Gets the database.
     *
     * @param dbName The name of the database.
     * @return An instance of HCatDatabaseInfo.
     * @throws HCatException
     */
    public HCatDatabaseInfo getDatabase(String dbName) throws HCatException;

    /**
     * Creates the database.
     *
     * @param dbInfo An instance of HCatCreateDBDesc.
     * @return true, if successful
     * @throws HCatException
     */
    public boolean createDatabase(HCatCreateDBDesc dbInfo)
            throws HCatException;

    /**
     * Deletes a database.
     *
     * @param dbName The name of the database to delete.
     * @param ifExists Hive returns an error if the database specified does not exist,
     *                 unless ifExists is set to true.
     * @param mode This is set to either "restrict" or "cascade". Restrict will
     *             remove the schema if all the tables are empty. Cascade removes
     *             everything including data and definitions.
     * @param userGroup The user group to use
     * @param permissions The permissions string to use. The format is "rwxrw-r-x".
     * @return true, if successful
     * @throws HCatException
     */
    public boolean deleteDatabase(String dbName, boolean ifExists, String mode,
            String userGroup, String permissions) throws HCatException;

    /**
     * Gets the tables like a pattern specified.
     *
     * @param dbName The name of the database.
     * @param regex The regular expression. Providing "*" would retrieve all the names
     *              of  the table.
     * @return A list of all table names matching the specified pattern.
     * @throws HCatException
     */
    public List<String> getTablesLike(String dbName, String regex)
            throws HCatException;

    /**
     * Gets the table.
     *
     * @param dbName The name of the database.
     * @param tableName The name of the table.
     * @return An instance of HCatTableInfo.
     * @throws HCatException
     */
    public HCatTableInfo getTable(String dbName, String tableName)
            throws HCatException;

    /**
     * Creates the table.
     *
     * @param createTableDesc An instance of HCatCreateTableDesc class.
     * @return true, if successful.
     * @throws HCatException the h cat exception
     */
    public boolean createTable(HCatCreateTableDesc createTableDesc)
            throws HCatException;

    /**
     * Creates the table like an existing table.
     *
     * @param dbName The name of the database.
     * @param existingTblName The name of the existing table.
     * @param newTableName The name of the new table.
     * @param ifExists the if exists
     * @param isExternal Set to "true", if table has be created at a different
     *                   location other than default.
     * @param location The location for the table.
     * @return true, if successful
     * @throws HCatException
     */
    public boolean createTableLike(String dbName, String existingTblName,
            String newTableName, boolean ifExists, boolean isExternal,
            String location) throws HCatException;

    /**
     * Delete a table.
     *
     * @param dbName The name of the database.
     * @param tableName The name of the table.
     * @param ifExists Hive returns an error if the database specified does not exist,
     *                 unless ifExists is set to true.
     * @param userGroup The user group to use.
     * @param permissions The permissions string to use. The format is "rwxrw-r-x".
     * @return true, if successful
     * @throws HCatException
     */
    public boolean deleteTable(String dbName, String tableName,
            boolean ifExists, String userGroup, String permissions)
            throws HCatException;

    /**
     * Renames a table.
     *
     * @param dbName The name of the database.
     * @param oldName The name of the table to be renamed.
     * @param newName The new name of the table.
     * @param userGroup The user group to use.
     * @param permissions The permissions string to use. The format is "rwxrw-r-x".
     * @return true, if successful
     * @throws HCatException
     */
    public boolean renameTable(String dbName, String oldName, String newName,
            String userGroup, String permissions) throws HCatException;

    /**
     * Gets all the partitions.
     *
     * @param dbName The name of the database.
     * @param tblName The name of the table.
     * @return A list of partition names.
     * @throws HCatException the h cat exception
     */
    public List<HCatPartitionInfo> getPartitions(String dbName, String tblName)
            throws HCatException;

    /**
     * Gets the partition.
     *
     * @param dbName The database name.
     * @param tableName The table name.
     * @param partitionName The partition name, Comma separated list of col_name='value'.
     * @return An instance of HCatPartitionInfo.
     * @throws HCatException
     */
    public HCatPartitionInfo getPartition(String dbName, String tableName,
            String partitionName) throws HCatException;

    /**
     * Adds the partition.
     *
     * @param partInfo An instance of HCatAddPartitionDesc.
     * @return true, if successful
     * @throws HCatException the h cat exception
     */
    public boolean addPartition(HCatAddPartitionDesc partInfo) throws HCatException;

    /**
     * Deletes partition.
     *
     * @param dbName The database name.
     * @param tableName The table name.
     * @param partitionName The partition name, Comma separated list of col_name='value'.
     * @param ifExists Hive returns an error if the partition specified does not exist, unless ifExists is set to true.
     * @param userGroup The user group to use.
     * @param permissions The permissions string to use. The format is "rwxrw-r-x".
     * @return true, if successful
     * @throws HCatException
     */
    public boolean deletePartition(String dbName, String tableName,
            String partitionName, boolean ifExists, String userGroup,
            String permissions) throws HCatException;

    /**
     * List partitions by filter.
     *
     * @param dbName The database name.
     * @param tblName The table name.
     * @param filter The filter string,
     *    for example "part1 = \"p1_abc\" and part2 <= "\p2_test\"". Filtering can
     *    be done only on string partition keys.
     * @return list of partitions
     * @throws HCatException the h cat exception
     */
    public List<HCatPartitionInfo> listPartitionsByFilter(String dbName, String tblName,
            String filter) throws HCatException;

}

HCatTempletonClient

This class implements HCatClient interface.

HCatTempletonDriver

This class implements Hive's CommandProcessorResponse interface.

Code Block
public interface CommandProcessor { public void init(); public CommandProcessorResponse run(String command) throws CommandNeedRetryException; }

The "run" method will consume the curl command as an input parameter and return the response.

HCatCommandDesc

This is an abstract class that helps in validating user input, building valid command descriptors and queries.

Code Block


/**
 * The Class HCatCommandDesc contains methods which help in validating,
 * building command descriptors and queries.
 */
public abstract class HCatCommandDesc{

    public abstract void validateCommandDesc() throws HCatException;
    abstract String buildQuery() throws HCatException;
    abstract boolean isValidationComplete();

}

HCatCreateTableDesc

This class is a sub class of HCatCommandDesc and will be used by the users to create descriptor and validate it for the "create table" command.

Code Block


public class HCatCreateDBDesc extends HCatCommandDesc {

    private String databaseName;
    private String locationUri;
    private String comment;
    private Map<String, String> dbProperties;

    /**
     * Gets the database properties.
     *
     * @return the database properties
     */
    Map<String, String> getDatabaseProperties() {
        return this.dbProperties;
    }

    /**
     * Sets the database properties.
     *
     * @param dbProps the db props
     */
    public void setDatabaseProperties(Map<String, String> dbProps) {
        this.dbProperties = dbProps;
    }

    /**
     * Gets the name.
     *
     * @return the name
     */
    String getName() {
        return this.databaseName;
    }

    /**
     * Sets the name.
     *
     * @param databaseName the new name
     */
    public void setName(String databaseName) {
        this.databaseName = databaseName;
    }

    /**
     * Gets the comment.
     *
     * @return the comment
     */
    String getComment() {
        return this.comment;
    }

    /**
     * Sets the comment.
     *
     * @param comment the new comment
     */
    public void setComment(String comment) {
        this.comment = comment;
    }

    /**
     * Gets the location.
     *
     * @return the location
     */
    String getLocation() {
        return this.locationUri;
    }

    /**
     * Sets the location.
     *
     * @param location the new location
     */
    public void setLocation(String location) {
        this.locationUri = location;
    }

    /* @return
    /* @throws HCatException
     * @see org.apache.hcatalog.api.HCatCommandDescBuilder#buildCommandDesc()
     */
    @Override
    public void validateCommandDesc()
            throws HCatException {

    }

    /* @param desc
    /* @return
    /* @throws HCatException
     * @see org.apache.hcatalog.api.HCatCommandDescBuilder#buildQuery(java.lang.Class)
     */
    @Override
    String buildQuery() throws HCatException {
        // TODO Auto-generated method stub
        return null;
    }

    /* @return
     * @see org.apache.hcatalog.api.HCatCommandDesc#isValidationComplete()
     */
    @Override
    boolean isValidationComplete() {
        // TODO Auto-generated method stub
        return false;
    }

}

HCatCreateDBDesc

This class is a sub class of HCatCommandDesc and will be used by the users to create descriptos and validate it for the "create database" command.

The implementation class will be provided as a configuration property, which will be used by the
"create" method. In this way, the implementation details will be masked to the users.

Code Block


public abstract class HCatClient {

    /**
     * Creates an instance of HCatClient.
     *
     * @param conf An instance of configuration.
     * @return An instance of HCatClient.
     * @throws IOException
     */
    public static HCatClient create(Configuration conf) throws IOException{
      HCatClient client = HCatUtil.getHCatClient(conf);
        if(client != null){
            client.initialize(conf);
        }
        return client;
    }

    abstract void initialize(Configuration conf) throws HCatException;

    /**
     * Get all existing databases that match the given
     * pattern. The matching occurs as per Java regular expressions
     *
     * @param databasePattern
     *          java re pattern
     * @return list of database names
     * @throws HCatException
     */
    public abstract List<String> listDatabaseNamesByPattern(String pattern) throws HCatException;

    /**
     * Gets the database.
     *
     * @param dbName The name of the database.
     * @return An instance of HCatDatabaseInfo.
     * @throws HCatException
     */
    public abstract HCatDatabase getDatabase(String dbName) throws HCatException;

    /**
     * Creates the database.
     *
     * @param dbInfo An instance of HCatCreateDBDesc.
     * @throws HCatException
     */
    public abstract void createDatabase(HCatCreateDBDesc dbInfo)
            throws HCatException;

    /**
     * Drops a database.
     *
     * @param dbName The name of the database to delete.
     * @param ifExists Hive returns an error if the database specified does not exist,
     *                 unless ifExists is set to true.
     * @param mode This is set to either "restrict" or "cascade". Restrict will
     *             remove the schema if all the tables are empty. Cascade removes
     *             everything including data and definitions.
     * @throws HCatException
     */
    public abstract void dropDatabase(String dbName, boolean ifExists, String mode) throws HCatException;

    /**
     * Returns all existing tables from the specified database which match the given
     * pattern. The matching occurs as per Java regular expressions.
     * @param dbName
     * @param tablePattern
     * @return list of table names
     * @throws HCatException
     */
    public abstract List<String> listTableNamesByPattern(String dbName, String tablePattern)
            throws HCatException;

    /**
     * Gets the table.
     *
     * @param dbName The name of the database.
     * @param tableName The name of the table.
     * @return An instance of HCatTableInfo.
     * @throws HCatException
     */
    public abstract HCatTable getTable(String dbName, String tableName)
            throws HCatException;

    /**
     * Creates the table.
     *
     * @param createTableDesc An instance of HCatCreateTableDesc class.
     * @throws HCatException the h cat exception
     */
    public abstract void createTable(HCatCreateTableDesc createTableDesc)
            throws HCatException;

    /**
     * Creates the table like an existing table.
     *
     * @param dbName The name of the database.
     * @param existingTblName The name of the existing table.
     * @param newTableName The name of the new table.
     * @param ifNotExists If true, then error related to already table existing is skipped.
     * @param isExternal Set to "true", if table has be created at a different
     *                   location other than default.
     * @param location The location for the table.
     * @throws HCatException
     */
    public abstract void createTableLike(String dbName, String existingTblName,
            String newTableName, boolean ifNotExists, boolean isExternal,
            String location) throws HCatException;

    /**
     * Drop table.
     *
     * @param dbName The name of the database.
     * @param tableName The name of the table.
     * @param ifExists Hive returns an error if the database specified does not exist,
     *                 unless ifExists is set to true.
     * @throws HCatException
     */
    public abstract void dropTable(String dbName, String tableName,
            boolean ifExists) throws HCatException;

    /**
     * Renames a table.
     *
     * @param dbName The name of the database.
     * @param oldName The name of the table to be renamed.
     * @param newName The new name of the table.
     * @throws HCatException

Code Block


public class HCatCreateTableDesc extends HCatCommandDesc{

    private String tableName;
    private boolean isExternal;
    private ArrayList<HCatFieldSchema> cols;
    private ArrayList<HCatFieldSchema> partCols;
    private ArrayList<String> bucketCols;
    private ArrayList<Order> sortCols;
    private int numBuckets;
    private String dbName;
    private String comment;
    private String fileFormat;
    private String location;
    private String storageHandler;
    private Map<String, String> tblProps;
    private boolean ifNotExists;

    ArrayList<HCatFieldSchema> getColsString() {
      return this.getCols();
    }

    boolean getIfNotExists() {
        return this.ifNotExists;
    }

    /**
     * Sets the if not exists.
     * If true, the user will not receive an error if the table already exists.
     * @param ifNotExists the new if not exists
     */
    public void setIfNotExists(boolean ifNotExists) {
      this.ifNotExists = ifNotExists;
    }

   String getTableName() {
        return this.tableName;
    }

    String getDatabaseName(){
        return this.dbName;
    }

    /**
     * Sets the database name.
     *
     * @param dbName the new database name
     */
    public void setDatabaseName(String dbName){
        this.dbName = dbName;
    }

    /**
     * Sets the table name.
     *
     * @param tableName the new table name
     */
    public void setTableName(String tableName) {
      this.tableName = tableName;
    }

    ArrayList<HCatFieldSchema> getCols() {
       // ArrayList<FieldSchema> cols = this.tableDesc.getCols();
        return null;
    }

    /**
     * Sets the table columns.
     *
     * @param cols List of columns.
     */
    public void setCols(ArrayList<HCatFieldSchema> cols) {
        //convert and set.
      this.cols = null;
    }

    ArrayList<HCatFieldSchema> getPartCols() {
        return null;
    }

    /**
     * Sets the part cols.
     *
     * @param partCols List of partition columns.
     */
    public void setPartCols(ArrayList<HCatFieldSchema> partCols) {
      //this.partCols = partCols;
    }

    ArrayList<String> getBucketCols() {
        return this.bucketCols;
    }

    /**
     * Sets the bucket cols.
     *
     * @param bucketCols The list of columns to be used for clustering.
     */
    public void setBucketCols(ArrayList<String> bucketCols) {
      this.bucketCols = bucketCols;
    }

    int getNumBuckets() {
        return this.numBuckets;
    }

    /**
     * Sets the num buckets.
     *
     * @param numBuckets The number of buckets.
     */
    public void setNumBuckets(int numBuckets) {
      this.numBuckets = numBuckets;
    }

    String getComment() {
        return this.comment;
    }

    /**
     * Sets the comment.
     *
     * @param comment The comment for the table.
     */
    public void setComment(String comment) {
      this.comment = comment;
    }


    String getStorageHandler() {
        return this.storageHandler;
    }

    /**
     * Sets the storage handler.
     *
     * @param storageHandler the new storage handler
     */
    public void setStorageHandler(String storageHandler) {
      this.storageHandler = storageHandler;
    }

    String getLocation() {
        return this.location;
    }

    /**
     * Sets the location.
     *
     * @param location the new location
     */
    public void setLocation(String location) {
      this.location = location;
    }

    boolean getExternal() {
        return this.isExternal;
    }

    /**
     * Sets the external.
     *
     * @param isExternal True/False, indicating if the table is an external table.
     */
    public abstract void setExternal(boolean isExternal) {
      this.isExternal = isExternal;
    }

    ArrayList<Order> getSortCols() {
        return this.sortCols;renameTable(String dbName, String oldName, String newName) throws HCatException;

    /**
     * Gets all the partitions.
     *
     * @param dbName The name of the database.
    }

 * @param tblName /**
The name of the table.
  * Sets the sort* cols.
@return A list of partition *names.
     * @param@throws sortColsHCatException the sortColsh tocat setexception
     */
    public voidabstract List<HCatPartition> setSortCols(ArrayList<Order> sortCols) {
getPartitions(String dbName, String tblName)
       this.sortCols = sortCols;
   throws }HCatException;

    /**
     * @return the table properties
     */
    Map<String, String> getTblProps() {
     * Gets the return thispartition.tblProps;
    }

    /**
     * @param tblPropsdbName The database name.
     * @param tableName The table name.
     the* @param tablepartitionName propertiesThe topartition set
name, Comma separated list  */of col_name='value'.
    public * void setTblProps(Map<String, String> tblProps) {@return An instance of HCatPartitionInfo.
     * this.tblProps = tblProps;
@throws HCatException
     }*/

    /**
public abstract HCatPartition getPartition(String dbName, *String SetstableName,
 the file format.
     *
    String *partitionName) @param format the new file formatthrows HCatException;

    /**
     */ Adds the partition.
    public void*
 setFileFormat(String format){
   * @param partInfo An instance of thisHCatAddPartitionDesc.fileFormat
 = format;
   * }

@throws HCatException the h String getFileFormat(){cat exception
     */
   return this.fileFormat;
    } public abstract void addPartition(HCatAddPartitionDesc partInfo) throws HCatException;

    /**
 @return
    /* @throwsDrops HCatExceptionpartition.
     * @see org.apache.hcatalog.api.HCatCommandDescBuilder#buildCommandDesc() *
     */
 @param dbName The database @Overridename.
    public * void@param validateCommandDesc()tableName throwsThe HCatExceptiontable {name.
     * @param partitionName //The TODO Auto-generated method stub
    }


partition name, Comma separated list of col_name='value'.
     /* @param desc
    /* @return
    /* @throws HCatException ifExists Hive returns an error if the partition specified does not exist, unless ifExists is set to true.
     * @see org.apache.hcatalog.api.HCatCommandDescBuilder#buildQuery(org.apache.hcatalog.api.HCatCommandDescBuilder)@throws HCatException
     */
    @Override
public abstract void  dropPartition(String buildQuery() throws HCatException {
        // TODO Auto-generated method stub
dbName, String tableName,
            return null;
    }


String partitionName, boolean ifExists) throws HCatException;

    /* @return*
     * @see org.apache.hcatalog.api.HCatCommandDesc#isValidationComplete()List partitions by filter.
     */
     * @param dbName The database @Overridename.
     * boolean isValidationComplete() {@param tblName The table name.
     * @param  // TODO Auto-generated method stub
 filter The filter string,
     *    for example "part1 return false;
    }

HCatAddPartitionDesc

This class is a sub class of HCatCommandDesc and will be used by the users to create descriptos and validate it for the "add partition" command.

Code Block


public class HCatAddPartitionDesc extends HCatCommandDesc {

    //private AddPartitionDesc addPartDesc;
    private String dbName;
    private String tableName;= \"p1_abc\" and part2 <= "\p2_test\"". Filtering can
     *    be done only on string partition keys.
     * @return list of partitions
    private String* location;
@throws HCatException the h private LinkedHashMap<String,String> partSpec;

cat exception
     */**
    public abstract  * @return database name
List<HCatPartition> listPartitionsByFilter(String dbName, String tblName,
       */
     String getDbName(filter) throws {HCatException;

    /**
     return this.dbName;
    }

* Mark partition for event.
     /**
     * @param SetsdbName theThe dbdatabase name.
     * @param tblName The table name.
     * @param dbName database namepartKVs the part k vs
     */
 @param   public void setDbName(String dbName) {eventType the event type
     * this.dbName@throws =HCatException dbName;
the h cat exception
 }

    */**
    public *abstract @returnvoid the table we're going to add the partitions to.
markPartitionForEvent(String dbName, String tblName,
          */
  Map<String, String> partKVs, StringPartitionEventType getTableName(eventType)
 {
       return this.tableName;
   throws }HCatException;

    /**
     * Sets the table name Checks if is partition marked for event.
     *
     * @param tableNamedbName the table we're going to add the partitions to.db name
     */
 @param tblName the publictbl voidname
 setTableName(String tableName) {
  * @param partKVs the this.tableNamepart =k tableName;vs
    }

 *   String getLocation() {
  @param eventType the event type
    return this.location;
* @return true, if }

is partition marked for /**event
     * Sets@throws HCatException the location. h cat exception
     */
    public *abstract @paramboolean location The location of partition in relation to table
isPartitionMarkedForEvent(String dbName, String tblName,
          */
  Map<String, String> public void setLocation(String location) {
partKVs, PartitionEventType eventType)
       this.location = location;
   throws }HCatException;

    /**
     * @returnGets the partitiondelegation specificationtoken.
     */
     LinkedHashMap<String, String> getPartSpec() {
* @param owner the owner
     * @param renewerKerberosPrincipalName the renewer kerberos return this.partSpec;principal name
    }

 * @return the delegation /**token
     * Adds@throws HCatException the partitionh name and value.cat exception
     */
    public * @param colName The column name.abstract String getDelegationToken(String owner, String renewerKerberosPrincipalName) throws
     * @param value The value. HCatException;

     /**/
     * Renew publicdelegation void addPartSpec(String colName, String value) {token.
     *
      this.partSpec.put(colName, value);
    }

* @param tokenStrForm the token str form
    @Override
 * @return the publiclong
 void validateCommandDesc()
   * @throws HCatException the h cat exception
   throws HCatException {*/

    public }

abstract long renewDelegationToken(String tokenStrForm) /* @returnthrows HCatException;

    /** @throws HCatException
     * Cancel @see org.apache.hcatalog.api.HCatCommandDesc#buildQuery()delegation token.
     */
    @Override
    String buildQuery() throws HCatException* {
@param tokenStrForm the token str form
   // TODO Auto-generated* method@throws stub
HCatException the h cat exception
    return null;*/
    public }

abstract void cancelDelegationToken(String tokenStrForm) /* @returnthrows HCatException;

     * @see org.apache.hcatalog.api.HCatCommandDesc#isValidationComplete()/**
     */
 Close the hcatalog @Overrideclient.
    boolean isValidationComplete() {*
     * @throws HCatException //the TODOh Auto-generated method stubcat exception
     */
   return false;
    }

}

HCatDBInfo

Modification to Existing Classes:

In the current code base, HCatTableInfo class acts a wrapper for the Table class, providing only required information to the users. Similar classes
need to be added for database and partition. Also, there needs to appropriate builder methods to construct HCatTableInfo, HCatPartitionInfo and
HCatDatabaseInfo classes.

The current relationship between HCatTableInfo, StorerInfo and PartInfo is as follows: Image Removed

The "PartInfo" class can be modified to act as a wrapper to partition ( similar to HCatTableInfo). The following modifications need to be done:

Remove job properties.
All the properties required by the partition info are available with the Partition. HCatPartitionInfo( present PartInfo) should make use of it.
There is no need to store HCatTableInfo as a member. It is only used to obtain data column and value mapping. There can be a method that can take a HCatTableInfo as input.

Usage

 public abstract void close() throws HCatException;

HCatCreateTableDesc

This class is a sub class of HCatCommandDesc and will be used by the users to create descriptor and validate it for the "create table" command.
Image Added

HCatCreateDBDesc

This class is a sub class of HCatCommandDesc and will be used by the users to create descriptors and validate it for the "create database" command.

!createdb.png|

HCatAddPartitionDesc

This class is a sub class of HCatCommandDesc and will be used by the users to create descriptors and validate it for the "add partition" command.

Image Added

HCatTable

This class encapsulates the table information returned the HCatClient implementation class and provides a uniform view to the user.

Image Added

HCatDatabase

This class encapsulates the database information returned the HCatClient implementation class and provides a uniform view to the user.

Image Added

HCatPartition

This class encapsulates the partition information returned the HCatClient implementation class and provides a uniform view to the user.

Image Added

Usage

Code Block


 Configuration config = new Configuration();
 config.add("hive-site.xml");
HCatClient client = HCatClient.create(config);

Code Block


 Configuration config = new Configuration();
 config.add("hive-site.xml");
 HCatTempletonClient client = new HCatTempletonClient(config);

 HCatCreateTableDesc desc = new HCatCreateTableDesc();
 desc.setTableName("demo_table");
 desc.setDatabaseName("db1");
 desc.setFileFormat("rcfile");
 ArrayList<HCatFieldSchema> cols = new ArrayList<HCatFieldSchema>();
 cols.add(new HCatFieldSchema("col1id", Type.INT, "comment1id columns"));
 cols.add(new HCatFieldSchema("col2value", Type.STRING, "comment2id columns"));
HCatCreateTableDesc tableDesc = descHCatCreateTableDesc.setCols(cols);

 //Validate
 desc.validateCommandDesccreate(db, "testtable", cols).fileFormat("rcfile").build();
 boolean success = client.createTable(desctableDesc);

Discussion Topics

...

Child pages

Versions Compared

Old Version 8

New Version Current

Key

Overview

Design

New Classes

Overview

Design Image Removed

HCatClient

HCatTempletonClient

HCatTempletonDriver

HCatCommandDesc

HCatCreateTableDesc

HCatCreateDBDesc

HCatAddPartitionDesc

HCatDBInfo

Modification to Existing Classes:

Usage

HCatCreateTableDesc

HCatCreateDBDesc

HCatAddPartitionDesc

HCatTable

HCatDatabase

HCatPartition

Usage

Discussion Topics