Versions Compared

Old Version 4

changes.mady.by.user Lorenz Quack

Saved on

compared with

New Version 5

changes.mady.by.user Lorenz Quack

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • keep track of "unsettled" delivery-tags & delivery-state
  • source assigns unique "delivery-tag" to delivery attempt
  • can exist without link endpoint ("suspended link")
  • should maintain association with LinkEndpoint if any to do Link recovery/resuming (Rob, thinks this association does not belong in the Terminus. I don't understand why).
  • termini can be assosiated associated with new endpoints ("resuming link")
  • has duribility durability 0 (no state), 1 (config), 2 (config + unsettled durable messages)

...

  • Interface between terminus and session endpoint
  • Lifetime: deleted when either there is a error, the Session ends, or the Link is closed (not just Detached)
  • maintains additional state 
  • endpoint assigns unique delivery-id to message. (alias for delivery-tag?). this loses meaning (becomes invalid) when resuming a link because it is a new link endpoint
  • no state in the endpoint required to resume a link. this comes from the terminus
  • has "handle" to identify the link on the wire (alias). assigned by Attach. local handle = output handle; remote handle = input handle
  • "attached" if has both handles, otherwise "detached" or "half detached"

...

  • We call this Session_1_0
  • mapping between channel
  • must detach link if terminus is not created/found upon Attach (rejecting link)
  • Establish: Terminus does not exist, Link Endpoint does not exist, handle not assigned, no state
  • Resuming: Terminus exists, LinkEndpoint does not exist, handle not assigned, unsettled state
  • Reattaching: Terminus exists, LinkEndpoint exists, handle not assigned, no state
  • Recovering: Terminus exists, endpoint?, handle?, state?

 

Further Considerations

Thread Safety

The Link Registry will be used across multiple connections and thus has to be thread-safe!

Especially Link Stealing seems dangerous. When stealing Link "foo" the Attaching Link (on IO-Thread-A) will have to wait for the Detach of the previously Attached Link being sent and processed by the broker (on IO-Thread-B). If at the same time Link "bar" is stolen from IO-Thread-A by IO-Thread-B there could be a deadlock if we are not careful. (Potentially use ConfigThread as serialising/coordiantor? bottleneck?)

Input Validation

The Link Registry will use user supplied values (link names, container ids). This information can come from both the config and over the wire. We have to make sure we validate both.

...

  • recover link from tuple (sourceContainerId, targetContainerId, linkName)
  • recover unsettled state
  • high frequency & low latency update of unsettled state (JSON fail!)
  • handle different DeliveryStates (AMQP 1.0 section 3.4)

Remembering Remote Terminus

Do we have to persist the remote Terminus? In the normal broker operation this seems unnecessary but when thinking about Broker federation we might want to remember for example whether the remote Terminus is a Queue or a Topic to validate upon resuming the Link.

Personally, I (Lorenz) think this is not necessary but if possible we should keep the design flexible enough to allow for a future extension.

Design

Brainstorming

  • As is currently the case each VirtualHost should have its own registry.
  • Both Sessions and Links are abstract immaterial concepts. The concrete objects are the SessionEndpoints and the LinkEndpoints.
    For Sessions we already only have one object (called Session_1_0 to be aligned to the legacy protocols but this really is the SessionEndpoint).
    We should do the same for Link/-Endpoints and  get rid of one of them. Currently the responsibilities are very unclear and there seems to be no value.
  • The implementation of the registry should be hidden behind an API so for example we do not care whether sending links and receiving links are maintained in the same data structure or in separate data structures.
  • We should ensure we discard the Endpoint upon Detach and recreate it upon Attach to make responsibilities clearLinkEndpoint upon Detach(close=true), Detach(error), or Session#End(). Also it needs to be dissociated from the Terminus.
  • Given the remote container id and the link name a lookup should return a Terminus so that . If the terminus is currently associated with a LinkEndpoint it should be reachable from the Terminus; if not we should be able to recreate a LinkEndpoint (and thus a Link) can be recreated dynamically from the Terminus.
  • Persistence of Termini and persistence of deliveryState/unsettled state separate?unsettled state should be separated. Persistence of unsettled state is out of scope for this.
  • A unsettledStore should support different levels of capability in alignment with the TerminusDurability.
    For example a Memory VH should not advertise any TerminusDurability, a JSON VirtualHost might be able to persist the configuration but not the settled state, while a BDB VH might support all three terminus durability modes.
    For now we do not provide TerminusDurability=2 
  • This is another example of where we should differentiate between ConfigurableObjects and PersistentObjects which seems to be orthogonal concepts. LinkRegistry should be Persistent but not Configurable.
  • Therefore, to not further make bad things worse separate out into distinct store? Same type as ConfigStore?

    • JSON:

      Code Block
      languagetext
      {
  "version" : "1.0",
  "durableLinks" : {
    localContainerId1 : {
      "sending" : {
        remoteContainerId1 : {
          linkName1 : { SourceDef },
          linkName2 : { SourceDef },
          ...
        },
        remoteContainerId2 : {
          linkName1 : { SourceDef },
          ...
        },
        ...
      }
      "receiving" : {
        remoteContainerId1 : {
          linkName1 : { TargetDef },
          ...
        },
        ...
      }
    },
    localContainerId2 : {
    }
  }
}
    • BDB either(both need to be versioned):
      • key: (sourceContainerId, targetContainerId, linkName) 
      • value: TerminusDef
      or two tables for sending and receiving with:
      • key: (localContainerId, remoteContainerId, linkName) 
      • value: SourceDef or TargetDef
    • JDBC (both need to be versioned)JDBC
      • COLUMNS: (sourceContainerId, targetContainerId, linkName, TerminusDef)
      or two tables for sending and receiving with:
      • COLUMNS: (localContainerId, remoteContainerId, linkName, SourceDef or TargetDef)
    The unsettled delivery map:
  • JSON: not supported
  • BDB:
    • key: (sourceContainerId, targetContainerId, linkName, deliveryTag) 
    • value: deliveryState
    • JDBC:COLUMNS: (ref to entry in above table, deliveryTag, deliveryState)

Stpes

  1. Merge (Sending-/Receiving-)LinkEndpoint and (Sending-/Receiving-)Link; Remove LinkAttachment
  2. Create Terminus Interface with methods to associate/dissociate a LinkEndpoint and maintain delivery states
  3. Make Source and Target Derive from Terminus and move the (non-durable) delivery state bookkeeping from the LinkEndpoint to the Terminus
  4. Create class for the LinkRegistry.
    1. encapsulate remoteContainer in the registry
    2. values are Termini
  5. replace current Map<containerId, LinkRegistry> on VH with new LinkRegistry

 

API

  • VH.putLink(String remoteContrainerId, String linkName, Terminus terminus) -> Terminus
  • VH.putIfAbsentLink(String remoteContrainerId, String linkName, Terminus terminus) -> Terminus
  • VH.getLink(String remoteContrainerId, String linkName) -> Terminus
  • VH.removeLink(String remoteContrainerId, String linkName) -> Terminus

...