QMFv2 Map Message Protocol
Introduction
This document describes the design of a proposed protocol for QMF based on map-messages (offered by the new C++ and Python APIs as well as the existing JMS API).
If adopted, this new protocol will change the formats of the messages used by QMF components to communicate. It will also change some of the message exchange patterns. It will not significantly impact the console and agent APIs and is intended to operate with applications that use the current QMF APIs.
Some highlights of the new design:
- Current QMF message bodies are in packed binary formats. While quite efficient, this style of formatting makes it difficult to make changes to the format and content for new features. The proposed format is based on encoded maps (a.k.a. dictionaries, field-tables) which are very easily extended and require less context to be useful.
- QMF currently requires the message broker to participate in the QMF protocol. The proposed protocol removes this requirement and will run properly on any AMQP message broker.
- QMF Agents currently publish periodic updates of their managed content to a globally accessible topic. This has security implications with regard to access to data. This is also inflexible in that updates to all data are sent at the same intervals. The proposed protocol removes the global publishing of data and introduces a subscription-query whereby a console may request that an agent publish certain data at a certain interval to an indicated target. Such requests can be subject to access control and may be focused on only the data that is needed for a particular application.
- The proposed protocol allows for more general use of data. For example:
- Free-form data, that has no object-identifier nor schema, can be transferred. This is useful for complex queries (joins, reports, etc.).
- Methods can be invoked against an agent in the absence of a managed object.
QMF Protocol
Use of Message Headers
Standard Headers
Application Headers
Message Type Codes
Type Code |
Name |
Description |
|---|---|---|
agent |
Agent Indication |
Indication sent from an agent periodically or immediately upon receiving an agent-locate with matching criteria. This message is used in agent discovery and as a keep-alive/heartbeat indication. |
agent-locate |
Agent Locate Indication |
Indication sent from a console when the console wishes to discover agents in the network. This message may contain a selector indicating which subset of all agents it wishes to discover. |
response |
Response |
General-purpose response to a request. A response message is always correlated to a specific request and is sent to the reply-to of the request. A response may optionally contain a structured exception or it may optionally contain a list of values (i.e. output arguments from a method call). |
schema-query |
Schema Query Request |
A request sent by a console to an agent to discover information about the schemata of data offered by the agent. A schema query may request a list of packages (i.e. namespaces), a list of schema keys, or full schema details from the agent. |
schema |
Schema Indication |
Indication sent from an agent either to announce newly added schema information or as a side effect of having received a schema query request from a console. |
get-query |
Get Query Request |
A request sent from a console to an agent to get data from the agent. Generally, a get query is used to access managed objects owned by the agent. This message is extensible such that an agent may offer advanced query capabilities (SQL, Reporting, etc.) to consoles that know how to use the capabilities. |
managed-object |
Managed Object Indication |
Indication sent from an agent as a side effect of having received a get-query or a create-subscription. Managed objects have object-identifiers, schema keys, and create/destroy timestamps in addition to their attributes. |
object |
Object Indication |
Indication sent from an agent as a side effect of having received a get-query. Unmanaged objects have schema keys but no object-identifiers or timestamps as they are not managed, can't be addressed, and have no lifecycle. |
data |
Data Indication |
Indication sent from an agent as a side effect of having received a get-query. Data has attributes only and can be used for any general purpose. |
event |
Event Indication |
Indication sent from an agent when the agent application raises an event. |
create-subscription |
Create Subscription Request |
Request sent from a console to an agent to create a subscription query. Such a query behaves like a get-query except that once the matching objects are sent, the query remains open and sends updates when matching objects are changed, added, or deleted. A subscription query has a target (not necessarily the address of the requesting console), a publishing interval to control the frequency of updates, and a time-to-live. If the subscription is not renewed after the time-to-live has expired, the subscription will be cancelled by the agent. |
cancel-subscription |
Cancel Subscription Request |
Request sent from a console to an agent to cancel an existing subscription query. |
renew-subscription |
Renew Subscription Request |
Request sent from a console to an agent to renew an existing subscription query. This causes the time-to-live timer to be reset. |
method |
Method Request |
Request sent from a console to an agent to invoke a method on either the agent itself or on a managed object controlled by the agent. |
Operations
Agent Announce
Console Topic Agent
| | |
| | <---- Agent Indication ---- |
| <------------------------ | |
| | |
Agent Locate
Console Topic Agent
| | |
| ---- Agent Locate Indication -> | |
| | --------------------------> |
| | |
| <-------------------------------------- Agent Indication ---- |
| | |
| | |
| ---- Agent Locate Indication -------------------------------> |
| | |
| <-------------------------------------- Agent Indication ---- |
| | |
Schema Query
Console Agent
| |
| ---- Schema Request ----------------> |
| |
| <------------- Schema Indication ---- |
| <------------- Schema Indication ---- |
| |
| <---------------------- Response ---- |
| |
Schema Announce
Console Topic Agent
| | |
| | <--- Schema Indication ---- |
| <------------------------ | |
| | |
Get Query
Console Agent
| |
| ---- Get Query Request -------------> |
| |
| <--------------- Data Indication ---- |
| <--------------- Data Indication ---- |
| <--------------- Data Indication ---- |
| <--------------- Data Indication ---- |
| <--------------- Data Indication ---- |
| <---------- Last Data Indication ---- |
| |
| <------------------------ Result ---- |
| |
Create Subscription Query
Console Target Agent
| | |
| ---- Create Subscription Request ---------------------> |
| | |
| | <----------- Data Indication ---- |
| | <----------- Data Indication ---- |
| | <----------- Data Indication ---- |
| | <----------- Data Indication ---- |
| | <----------- Data Indication ---- |
| | |
| <------------------------------------------ Result ---- |
| | |
| | <----------- Data Indication ---- |
| | |
| | <----------- Data Indication ---- |
| | |
Cancel Subscription Query
Console Target Agent
| | |
| ---- Cancel Subscription Request ---------------------> |
| | |
| <------------------------------------------ Result ---- |
| | |
| | <----------- Data Indication ---- |
| | <------ Last Data Indication ---- |
| | |
Renew Subscription Query
Console Target Agent
| | |
| ---- Renew Subscription Request ----------------------> |
| | |
| <------------------------------------------ Result ---- |
| | |
Invoke Method
Console Agent
| |
| ---- Method Request ----------------> |
| |
| <------------------------ Result ---- |
| |
Raise Event
Console Topic Agent
| | |
| | <---------- Event Indication ---- |
| <------------------ | |
| | |
Map Body Formats
QMF Data Formats
qmfAgentLocate := {qmfQuery}
qmfAgentIndicate := {qmfAgentInfo}
qmfAgentInfo := agent_info: {name: (sstr),
vendor: (sstr),
product: (sstr),
version: (sstr), -- optional
platform: (sstr), -- optional
arch: (sstr)} -- optional
qmfQuery := query: {what: qmfQueryTarget,
where: qmfQueryPredicate} -- optional
qmfQueryTarget := {schema_package: (void)} ||
{schema_id: (void)} ||
{schema: (void)} ||
{agent: (void)} ||
{object_id: qmfClassSpecifier} ||
{object: qmfClassSpecifier}
qmfClassSpecifier := {class_name: (sstr),
package_name: (sstr), -- optional
hash: (bin128)} -- optional
qmfQueryPredicate := {qmfCompare} ||
{and: listOf(qmfQueryPredicate)} ||
{or: listOf(qmfQueryPredicate)} ||
{not: qmfQueryPredicate}
qmfCompare := list(eq, (sstr), amqpValue) ||
list(ne, (sstr), amqpValue) ||
list(lt, (sstr), amqpValue) ||
list(gt, (sstr), amqpValue) ||
list(le, (sstr), amqpValue) ||
list(ge, (sstr), amqpValue) ||
list(exists, (sstr)) ||
list(re-match, (sstr), (lstr))
qmfData := {values: mapOf(amqpValue),
subtypes: mapOf(sstr) } -- optional, keys must be in values
qmfDescribedData := {(qmfData),
class_id: qmfClassId }
qmfManagedData := {(qmfDescribedData),
object_id: (lstr),
create_time: (datetime),
delete_time: (datetime), -- optional
update_time: (datetime) }
qmfClassId := {package: (sstr),
name: (sstr),
hash: (bin128) }
qmfObjectClass := {(qmfData[qmfProperty|qmfMethod]),
class_id: qmfClassId,
desc: (lstr),
superclass: qmfClassId }
qmfProperty := {amqp_type: (uint8),
access_rule: (sstr),
optional: (void),
unit: (sstr),
desc: (lstr),
subtype: (sstr),
discrete: (void),
min: (amqp_type),
max: (amqp_type),
maxlen: (uint32) }
qmfMethod := {desc: (lstr),
arguments: mapOf(qmfArgument) }
qmdArgument := {amqp_type: (uint8),
unit: (sstr),
desc: (lstr),
subtype: (sstr),
input: (void)
output: (void)