Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: remove broken link to old doc


Table of Contents

Since version 0.2, HCatalog provides notifications for certain events happening in the system. This way applications such as Oozie can wait for those events and schedule the work that depends on them. The current version of HCatalog supports two kinds of events:


To receive notification that a new partition has been added, you need to follow these three steps.

  1. To start receiving messages, create a connection to a message bus as shown here:

    No Format
    ConnectionFactory connFac = new ActiveMQConnectionFactory(amqurl);
    Connection conn = connFac.createConnection();

  2. Subscribe to a topic you are interested in. When subscribing on a message bus, you need to subscribe to a particular topic to receive the messages that are being delivered on that topic.

    • The topic name corresponding to a particular table is stored in table properties and can be retrieved using the following piece of code:

      No Format
      HiveMetaStoreClient msc = new HiveMetaStoreClient(hiveConf);
      String topicName = msc.getTable("mydb",

    • Use the topic name to subscribe to a topic as follows:

      No Format
      Session session = conn.createSession(true, Session.SESSION_TRANSACTED);
      Destination hcatTopic = session.createTopic(topicName);
      MessageConsumer consumer = session.createConsumer(hcatTopic);

  3. To start receiving messages you need to implement the JMS interface MessageListener, which, in turn, will make you implement the method onMessage(Message msg). This method will be called whenever a new message arrives on the message bus. The message contains a partition object representing the corresponding partition, which can be retrieved as shown here:

    No Format
    public void onMessage(Message msg) {
      // We are interested in only add_partition events on this table.
      // So, check message type first.
           Object obj = (((ObjectMessage)msg).getObject());


  1. Repeat steps one and two from above to establish the connection to the notification system and to subscribe to the topic.
  2. Receive the notification as shown in this example:

    No Format
    HiveMetaStoreClient msc = new HiveMetaStoreClient(conf);
    // Create a map, specifying partition key names and values
    Map<String,String> partMap = new HashMap<String, String>();
    // Mark the partition as "done"
    msc.markPartitionForEvent("mydb", "mytbl", partMap, PartitionEventType.LOAD_DONE);


titleNavigation Links

Previous: Dynamic Partitioning
Next: Storage Based Authorization

General: HCatalog ManualWebHCat ManualHive Wiki HomeHive Project Site Old version of this document (HCatalog 0.5.0): Notification