Skip to end of metadata
Go to start of metadata

QMFv2 Map Message Protocol

Introduction

This document describes the design of a proposed protocol for QMF based on map-messages (offered by the new C++ and Python APIs as well as the existing JMS API).

If adopted, this new protocol will change the formats of the messages used by QMF components to communicate. It will also change some of the message exchange patterns. It will not significantly impact the console and agent APIs and is intended to operate with applications that use the current QMF APIs.

Some highlights of the new design:

  • Current QMF message bodies are in packed binary formats. While quite efficient, this style of formatting makes it difficult to make changes to the format and content for new features. The proposed format is based on encoded maps (a.k.a. dictionaries, field-tables) which are very easily extended and require less context to be useful.
  • QMF currently requires the message broker to participate in the QMF protocol. The proposed protocol removes this requirement and will run properly on any AMQP message broker.
  • QMF Agents currently publish periodic updates of their managed content to a globally accessible topic. This has security implications with regard to access to data. This is also inflexible in that updates to all data are sent at the same intervals. The proposed protocol removes the global publishing of data and introduces a subscription-query whereby a console may request that an agent publish certain data at a certain interval to an indicated target. Such requests can be subject to access control and may be focused on only the data that is needed for a particular application.
  • The proposed protocol allows for more general use of data. For example:
    • Free-form data, that has no object-identifier nor schema, can be transferred. This is useful for complex queries (joins, reports, etc.).
    • Methods can be invoked against an agent in the absence of a managed object.

QMF Protocol

Use of Message Headers

Standard Message Properties

Message Property

Use

correlation-id

Used in request/response/indication sets to correlate responses and indications to their request.

reply-to

Used in requests to indicate the address for the response.

content-type

'amqp/map' or 'amqp/list'

user-id

Supplied in a request if authentication/authorization at the agent is appropriate.

app-id

'qmf2'

Custom Application Headers

Application Header Key

Use

method

'request', 'response', or 'indication'. This field describes the message's role in a particular message-exchange pattern.

partial

Void. If this field is present, it indicates that the message does not contain the complete request or response (i.e. another message will follow using the same opcode and correlation-id)

qmf.opcode

QMF-specific operation code (see list below). The opcode defines what content, if any, is to be found in the message body.

qmf.content

If the opcode is a data indication, this field indicates what kind of data will be found in the message body.

qmf.agent

If this message is a data indication sent by an agent, this field contains the agent's name.

QMF OpCodes

qmf.opcode field

Message Body Data Type

Sent By

Sent To

Description

_exception

QMF_DATA

Agent

reply-to

This general-purpose message can be sent by an agent in response to any request (query, subscription, method) if the request could not be completed for any reason. The QMF_DATA in the message body contains details of why the failure occurred.

_agent_locate_request

QMF_QUERY_PREDICATE

Console

QMF Topic

A console may send an agent-locate-request in order to reach all available agents. The predicate may be used to limit the set of agents that will respond to the request.

_agent_locate_response

QMF_DATA

Agent

reply-to

This is a response to an agent-locate-request. An agent will send an agent-locate-response if it received an agent-locate-request with a predicate that matches its characteristics.

_agent_heartbeat_indication

QMF_DATA

Agent

Topic

Each agent periodically sends a heartbeat message to a topic to indicate that it is alive and connected. The content of the heartbeat message is the list of the agent's characteristics.

_query_request

QMF_QUERY

Console

Agent

A console sends a query to an agent to request that the agent send data to the requester.

_query_response

List of <qmf.content>

Agent

reply-to

The response to a query sent by a console.

_subscribe_request

QMF_SUBSCRIBE

Console

Agent

A console sends a subscribe-request to an agent to receive data matched by a query. A subscription differs from a query request in that it continues to send updated information to the console when the data changes.

_subscribe_response

QMF_SUBSCRIPTION

Agent

reply-to

When an agent receives a subscribe-request, it sends a subscribe-response granting (or refusing) the subscription.  Should the subscription succeed, the response will contain an identifier for the subscription assigned by the Agent. Thereafter, it will send data-indication messages on the same correlation-id with updates when they happen or periodically.  The first data-indication message sent by the agent will contain all matching data, subsequent data-indications will contain only those matching data that has changed since the last update.

_subscribe_cancel_indication

QMF_SUBSCRIPTION_ID

Console

Agent

A console can request that a subscription it created be immediately cancelled. This message must have the same correlation-id as the original request, and contain the subscription identifier as assigned by the Agent.

_subscribe_refresh_indication

QMF_SUBSCRIPTION_ID

Console

Agent

A console can keep a subscription alive by periodically refreshing it by sending a subscribe-refresh-indication. This message must have the same correlation-id as the original request, and contain the subscription identifier as assigned by the Agent.

_data_indication

List of <qmf.content>

Agent

reply-to or topic

A data indication is sent by an Agent when 1) subscription data has changed and needs to be published, 2) an event has occurred and event data is being published, and 3) any other time an agent wants to send unsolicited data.

_method_request

QMF_METHOD_CALL

Console

Agent

A console may invoke a method on an object managed by an agent. It may also invoke a method directly on the agent if appropriate. This message contains the method call including the input arguments.

_method_response

QMF_METHOD_RESULT

Agent

reply-to

A method call always results in a single method result. This message carries either the output arguments from a successful method call or it holds an exception to describe a failure.

QMF Content Types

qmf.content field

Data Type

Description

_schema_id

SCHEMA_ID

Schema class identifier

_schema_class

SCHEMA_CLASS

Schema class definition

_object_id

OBJECT_ID

Managed object identifier

_data

QMF_DATA

Data, managed and/or described or free-form

_event

QMF_EVENT

Event

_query

QMF_QUERY

Query

Message Body Map Formats

SCHEMA_ID

  SCHEMA_ID := { _package_name: STRING,
                 _class_name:   STRING,
                 _type:         '_data' | '_event',
                 _hash:         UUID
               }

Field

Optional

Description

_package_name

no

Package name (namespace) for the described class

_class_name

no

Name of the described class

_type

no

Class type: data or event

_hash

yes

Hash (uuid) to distinguish different versions of a class

SCHEMA_CLASS

  SCHEMA_CLASS := { _schema_id:        SCHEMA_ID,
                    _desc:             STRING,
                    _default_severity: NUMBER,
                    _properties:       [SCHEMA_PROPERTY, ...],
                    _methods:          [SCHEMA_METHOD, ...]
                  }

Field

Optional

Description

_schema_id

no

Identifier for this schema class

_desc

yes

Description of the schema class

_default_severity

yes

Default severity for an event class

_properties

no

List of SCHEMA_PROPERTY elements describing the properties of the class

_methods

no

List of SCHEMA_METHOD elements describing the methods of the class

SCHEMA_PROPERTY

  SCHEMA_PROPERTY := { _name:       STRING,
                       _type:       QMF_TYPE,
                       _index:      BOOL,
                       _access:     'RO' | 'RC' | 'RW',
                       _unit:       STRING,
                       _min:        NUMBER,
                       _max:        NUMBER,
                       _maxlen:     NUMBER,
                       _dir:        'I' | 'O' | 'IO',
                       _desc:       STRING,
                       _references: SCHEMA_ID,
                       _subtype:    QMF_SUBTYPE
                     }

Field

Optional

Description

_name

no

The name of the property

_type

no

The QMF data type of this property

_index

yes

True iff this property is an index of an object. Default is False.

_access

yes

The remote access rules for this property:
RO => Read Only (default if not specified)
RC => Read Create
RW => Read Write

_unit

yes

Annotation. Units of measure for numeric values

_min

yes

Minimum numeric value

_max

yes

Maximum numeric value

_maxlen

yes

Maximum length of a variable length value (in octets)

_dir

yes

Used only for method arguments. Direction of transfer:
I => Input (caller to callee)
O => Output (callee to caller)
IO => Both

_desc

yes

Annotation. Description of the property

_references

yes

If the type is a reference to another managed object, this field may be used to specify the required class for that object

_subtype

yes

May be used to further specify the meaning of the value of this field. For example, a number may actually be a timestamp or a duration. A string may be a reference to another object, or a URL.

QMF_TYPE

  QMF_TYPE := 'TYPE_VOID'   |
              'TYPE_BOOL'   |
              'TYPE_INT'    |
              'TYPE_FLOAT'  |
              'TYPE_STRING' |
              'TYPE_MAP'    |
              'TYPE_LIST'   |
              'TYPE_UUID'

QMF_SUBTYPE

  QMF_SUBTYPE := 'reference' |
                 'url'       |
                 'timestamp' |
                 'duration'

SCHEMA_METHOD

  SCHEMA_METHOD := { _name:      STRING,
                     _desc:      STRING,
                     _arguments: [SCHEMA_PROPERTY, ...]
                   }

Field

Optional

Description

_name

no

The name of the method

_desc

yes

Annotation. Description of this method

_arguments

no

List of SCHEMA_PROPERTY elements that describe the method's arguments

QMF_METHOD_CALL

  QMF_METHOD_CALL := { _object_id:   OBJECT_ID,
                       _method_name: STRING,
                       _arguments:   { EACH_KEY: VALUE },
                       _subtypes:    { EACH_KEY: STRING }
                     }

Field

Optional

Description

_object_id

yes

The identity of the managed object receiving the method call. If not supplied, this method applies generally to the agent.

_method_name

no

The name of the method

_arguments

yes

The input arguments, if any

_subtypes

yes

Subtype information for the input arguments, if any

QMF_METHOD_RESULT

  QMF_METHOD_RESULT := { _arguments: { EACH_KEY: VALUE },
                         _subtypes:  { EACH_KEY: STRING }
                       }

Field

Optional

Description

_arguments

yes

Output arguments from a successful method call, if any

_subtypes

yes

Subtype information for the output arguments, if any

QMF_DATA

  QMF_DATA := { _schema_id: SCHEMA_ID,
                _object_id: OBJECT_ID,
                _values:    { EACH_KEY: VALUE },
                _subtypes:  { EACH_KEY: STRING }
              }

Field

Optional

Description

_schema_id

yes

If this data is "described", this field references the schema class that describes the data.

_object_id

yes

If this data is "managed", this field provides the identifier that can be used to address this managed object.

_values

no

The map of values keyed by their property names

_subtypes

yes

Per-property subtypes that may be used to provide more information about the meaning of a value than its QMF_TYPE

OBJECT_ID

  OBJECT_ID := { _agent_name:  STRING,
                 _agent_epoch: NUMBER,
                 _object_name: STRING
               }

Field

Optional

Description

_agent_name

yes

Name of the agent that is managing the referenced data

_agent_epoch

yes

Numeric epoch of the agent process. This number is managed by the agent and is incremented each time the agent process starts. This field is only present for transient object IDs that must not be the same for a given object across an agent restart. Persistent object IDs must not include this field.

_object_name

no

Name of the data that uniquely identifies the data within the context of the agent.

QMF_QUERY

  QMF_QUERY := { _what:      QMF_QUERY_TARGET,
                 _where:     QMF_QUERY_PREDICATE,
                 _object_id: OBJECT_ID,
                 _schema_id: SCHEMA_ID
               }

Field

Optional

Description

_what

no

Identifies the kind of data being queried

_where

yes

Query predicate to limit the number of results of the query

_object_id

yes

Identifier of a single object being queried

_schema_id

yes

Identifier of a single schema being queried

QMF_QUERY_TARGET

  QMF_QUERY_TARGET := 'SCHEMA_ID' |
                      'SCHEMA'    |
                      'OBJECT_ID' |
                      'OBJECT'

QMF_QUERY_PREDICATE

QMF_SUBSCRIBE

  QMF_SUBSCRIBE := { _query:     QMF_QUERY,
                     _duration:  NUMBER,
                     _interval:  NUMBER
                   }

Field

Optional

Description

_query

no

The query that defines the set of data being subscribed to

_duration

yes

The requested time (in seconds) after which this subscription will be automatically canceled. If a subscribe_refresh_indication is received by the agent running this query, this time interval will start over.

_interval

yes

The request time (in milliseconds) between periodic updates of data in this subscription. The agent may place a minimum on this interval.

QMF_SUBSCRIPTION

  QMF_SUBSCRIPTION := { _subscription_id:     STRING,
                        _duration:  NUMBER,
                        _interval:  NUMBER,
                      }

Field

Optional

Description

_subscription_id

yes

Assigned by the Agent when replying to a successful subscription request. Must be supplied by the Console when sending a subscription refresh or cancel to the Agent for this subscription.

_duration

no

The time (in seconds) after which this subscription will be automatically canceled.

_interval

no

The time (in milliseconds) between periodic updates of data in this subscription.

QMF_SUBSCRIPTION_ID

QMF_SUBSCRIPTION_ID := { _subscription_id:     STRING}

Field

Optional

Description

_subscription_id

no

Supplied by the Console when sending a subscription refresh or cancel to the Agent for this subscription.

Labels
  • No labels