Camel Ignite component

Available as of Camel 2.17

Apache Ignite In-Memory Data Fabric is a high-performance, integrated and distributed in-memory platform for computing and transacting on large-scale data sets in real-time, orders of magnitude faster than possible with traditional disk-based or flash technologies. It is designed to deliver uncompromised performance for a wide set of in-memory computing use cases from high performance computing, to the industry most advanced data grid, highly available service grid, and streaming. See all features.

This component offers seven endpoints to cover much of Ignite's functionality:

  • Ignite Cache.
  • Ignite Compute.
  • Ignite Messaging.
  • Ignite Events.
  • Ignite Sets.
  • Ignite Queues.
  • Ignite ID Generator.

To use this component, add the following dependency to your pom.xml:

<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-ignite</artifactId>
    <version>${camel.version}</version> <!-- use the same version as your Camel core version -->
</dependency>

Running in OSGi

If running in an OSGi container, please don't miss the OSGi Support section below.

Table of contents

CACHE: Ignite Cache endpoint

This endpoint allows you to interact with an Ignite Cache:

ignite:cache:cacheName?option1=value1&option2=value2...

Offers both a Producer (to invoke cache operations on an Ignite cache) and a Consumer (to consume changes from a continuous query).

The cache value is always the body of the message, whereas the cache key is always stored in the IgniteConstants.IGNITE_CACHE_KEY message header.

Even if you configure a fixed operation in the endpoint URI, you can vary it per-exchange by setting the IgniteConstants.IGNITE_CACHE_OPERATION message header.

Options

OptionTypeDefault valueDescriptionCompulsoryConsumer/producer
operationIgniteCacheOperation enum---

Cache operation to perform.

Possible values: GET, PUT, REMOVE, SIZE, REBALANCE, QUERY, CLEAR.

NProducer
failIfInexistentCachebooleanfalseWhether to fail the initialization if the cache doesn't exist.NProducer
cachePeekModeCachePeekMode (Ignite) enumALLThe cache peek mode used for SIZE operations.NProducer
queryQuery---

The query used for QUERY operations (as a producer) or to launch the continuous query (as a consumer). You can use a reference to a registry bean with a syntax like: ?query=#myQuery.

Y for the consumer / N for the producer

Producer (for QUERY operation) and Consumer
remoteFilterCacheEntryEventSerializableFilter

---

An optional remote filter for the continuous query consumer. You can use a reference to a registry bean with a syntax like: ?remoteFilter=#myRemoteFilter.

NConsumer
oneExchangePerUpdatebooleantrueWhether to send one exchange per cache update, even if multiple changes arrive in a group.NConsumer
fireExistingQueryResultsbooleanfalseWhen starting the continuous query consumer, whether to fire existing cache results.NConsumer
autoUnsubscribeboolean

ContinuousQuery.DFLT_AUTO_UNSUBSCRIBE

Auto-unsubscribe flag on the Continuous Query (see Ignite docs).NConsumer
pageSizeint

ContinuousQuery.DFLT_PAGE_SIZE

Page size on the Continuous Query (see Ignite docs).NConsumer
timeIntervallong

ContinuousQuery.DFLT_TIME_INTERVAL

Time interval in millis on the Continuous Query (see Ignite docs).NConsumer

Headers used

This endpoint uses the following headers:

Header nameConstantExpected typeDescription

CamelIgniteCacheKey

IgniteConstants.IGNITE_CACHE_KEYStringThe cache key for the entry value in the message body.
CamelIgniteCacheQueryIgniteConstants.IGNITE_CACHE_QUERYQueryThe query to run (producer) when invoking the QUERY operation.
CamelIgniteCacheOperationIgniteConstants.IGNITE_CACHE_OPERATIONIgniteCacheOperation enumAllows you to dynamically change the cache operation to execute (producer).
CamelIgniteCachePeekModeIgniteConstants.IGNITE_CACHE_PEEK_MODECachePeekMode enumAllows you to dynamically change the cache peek mode when running the SIZE operation.
CamelIgniteCacheEventTypeIgniteConstants.IGNITE_CACHE_EVENT_TYPEint (EventType constants)This header carries the received event type when using the continuous query consumer.
CamelIgniteCacheNameIgniteConstants.IGNITE_CACHE_NAMEString

This header carries the cache name for which a continuous query event was received (consumer).

It does not allow you to dynamically change the cache against which a producer operation is performed. Use EIPs for that (e.g. recipient list, dynamic router).

CamelIgniteCacheOldValueIgniteConstants.IGNITE_CACHE_OLD_VALUEObjectThis header carries the old cache value when passed in the incoming cache event (consumer).

COMPUTE: Ignite Compute endpoint

This endpoint allows you to run compute operations on the cluster by passing in an IgniteCallable, an IgniteRunnable, an IgniteClosure, or collections of them, along with their parameters if necessary. 

ignite:compute:endpointId?executionType=...&option1=value1&option2=value2...

This endpoint only supports producers.

The host part of the endpoint URI is a symbolic endpoint ID, it is not used for any purposes.

The endpoint tries to run the object passed in the body of the IN message as the compute job. It expects different payload types depending on the execution type.

Expected payload types

Each operation expects the indicated types:

OperationExpected payloads
CALL
Collection of IgniteCallable, or a single IgniteCallable.
BROADCAST
IgniteCallable, IgniteRunnable, IgniteClosure.
APPLY
IgniteClosure.
EXECUTE
ComputeTask, Class<? extends ComputeTask> or an object representing parameters if the taskName option is not null.
RUN
A Collection of IgniteRunnables, or a single IgniteRunnable.
AFFINITY_CALL
IgniteCallable.
AFFINITY_RUN
IgniteRunnable.

Options

OptionTypeDefault valueDescriptionCompulsoryConsumer/producer

executionType

IgniteComputeExecutionType enum

---

The compute operation to perform. Possible values: CALL, BROADCAST, APPLY, EXECUTE, RUN, AFFINITY_CALL, AFFINITY_RUN.

The component expects different payload types depending on the operation. See above.

YProducer

clusterGroupExpression

ClusterGroupExpression---

An expression that returns the Cluster Group for the IgniteCompute instance.

NProducer

taskName

String---The task name, only used when performing the EXECUTE if a ComputeTask is not passed in the payload.NProducer

computeName

String---

The name for the IgniteCompute instances produced by this endpoint.

N

Producer

timeoutMillis

Long

---

The timeout for the compute job.

NProducer

Headers used

This endpoint uses the following headers:

Header nameConstantExpected typeDescription

CamelIgniteComputeExecutionType

IgniteConstants.IGNITE_COMPUTE_EXECUTION_TYPEIgniteComputeExecutionType enumAllows you to dynamically change the compute operation to perform.

CamelIgniteComputeParameters 

IgniteConstants.IGNITE_COMPUTE_PARAMSAny object or Collection of objects.Parameters for APPLY, BROADCAST and EXECUTE operations.

CamelIgniteComputeReducer

IgniteConstants.IGNITE_COMPUTE_REDUCERIgniteReducerReducer for the APPLY and CALL operations.
CamelIgniteComputeAffinityCacheNameIgniteConstants.IGNITE_COMPUTE_AFFINITY_CACHE_NAMEStringAffinity cache name for the AFFINITY_CALL and AFFINITY_RUN operations.

CamelIgniteComputeAffinityKey

IgniteConstants.IGNITE_COMPUTE_AFFINITY_KEYObjectAffinity key for the AFFINITY_CALL and AFFINITY_RUN operations.

MESSAGING: Ignite Messaging endpoint

 The Ignite Messaging endpoint allows you to send and consume messages from an Ignite topic.

ignite:messaging:topicName?option1=value1&option2=value2...

This endpoint supports producers (to send messages) and consumers (to receive messages).

Options

OptionTypeDefault valueDescriptionCompulsoryConsumer/producer
clusterGroupExpression

ClusterGroupExpression

---

An expression that returns the Cluster Group for the IgniteMessaging instance.

NConsumer and Producer
sendMode

IgniteMessagingSendMode enum

UNORDERED

The send mode to use.

Possible values: UNORDERED, ORDERED.

NProducer
timeout

Long

---

The timeout for the send operation when using ordered messages.

NProducer

Headers used

This endpoint uses the following headers:

Header nameConstantExpected typeDescription

CamelIgniteMessagingTopic

IgniteConstants.IGNITE_MESSAGING_TOPICString

Allows you to dynamically change the topic to send messages to (producer).
It also carries the topic on which a message was received (consumer).

CamelIgniteMessagingUUID

IgniteConstants.IGNITE_MESSAGING_UUIDUUIDThis header is filled in with the UUID of the subscription when a message arrives (consumer).

EVENTS: Ignite Events endpoint

The Ignite Events endpoint allows you to receive events from the Ignite cluster by creating a local event listener.

ignite:events:endpointId?option1=value1&option2=value2...

This endpoint only supports consumers.

The Exchanges created by this consumer put the received Event object into the body of the IN message.

Options

OptionTypeDefault valueDescriptionCompulsoryConsumer/producer

events

Set<Integer> or String

EventType.EVTS_ALL

The event IDs to subscribe to. You can set a Set<Integer> directly where the IDs are the different constants in org.apache.ignite.events.EventType, or you can specify a comma-separated list of constant names.
For example: EVT_CACHE_ENTRY_CREATED,EVT_CACHE_OBJECT_REMOVED,EVT_IGFS_DIR_CREATED.

NConsumer

clusterGroupExpression

ClusterGroupExpression

---

An expression that returns the Cluster Group to be used by the IgniteEvents instance.

NConsumer

SETS: Ignite Sets endpoint

The Ignite Sets endpoint allows you to interact with Ignite Set data structures.

ignite:sets:setName?option1=value1&option2=value2...

This endpoint only supports producers.

Options

OptionTypeDefault valueDescriptionCompulsoryConsumer/producer

operation

IgniteSetOperation enum

---

The operation to invoke on the Ignite Set.
Superseded by the IgniteConstants.IGNITE_SETS_OPERATION header in the IN message.
Possible values: CONTAINS, ADD, SIZE, REMOVE, ITERATOR, CLEAR, RETAIN_ALL, ARRAY.

NProducer

configuration

CollectionConfigurationempty CollectionConfiguration

The configuration for this data structure. You can also specify inner properties by using the config. prefix.

For example: ?config.backups=4

NProducer

Headers used

This endpoint uses the following headers:

Header nameConstantExpected typeDescription

CamelIgniteSetsOperation

IgniteConstants.IGNITE_SETS_OPERATIONIgniteSetOperation enum

Allows you to dynamically change the set operation.

QUEUE: Ignite Queue endpoint

The Ignite Queue endpoint allows you to interact with Ignite Queue data structures.

ignite:queue:queueName?option1=value1&option2=value2...

This endpoint only supports producers.

Options

OptionTypeDefault valueDescriptionCompulsoryConsumer/producer

operation

IgniteQueueOperation enum

---

The operation to invoke on the Ignite Queue.

Superseded by the IgniteConstants.IGNITE_QUEUE_OPERATION header in the IN message.

Possible values: CONTAINS, ADD, SIZE, REMOVE, ITERATOR, CLEAR, RETAIN_ALL, ARRAY, DRAIN, ELEMENT, PEEK, OFFER, POLL, TAKE, PUT.

NProducer

configuration

CollectionConfiguration
empty CollectionConfiguration

The configuration for this data structure. You can also specify inner properties by using the config. prefix.

For example: ?config.backups=4

NProducer
timeoutMillisLong---The timeout in milliseconds to use when invoking the OFFER or POLL operations.NProducer
capacityint--- (unbounded)The queue capacity if you'd like to use a bounded queue.NProducer

Headers used

This endpoint uses the following headers:

Header nameConstantExpected typeDescription

CamelIgniteQueueOperation

IgniteConstants.IGNITE_QUEUE_OPERATIONIgniteQueueOperation enum

Allows you to dynamically change the queue operation.

CamelIgniteQueueMaxElementsIgniteConstants.IGNITE_QUEUE_MAX_ELEMENTSInteger or intWhen invoking the DRAIN operation, the amount of items to drain.
CamelIgniteQueueTransferredCountIgniteConstants.IGNITE_QUEUE_TRANSFERRED_COUNTInteger or intThe amount of items transferred as the result of the DRAIN operation.
CamelIgniteQueueTimeoutMillisIgniteConstants.IGNITE_QUEUE_TIMEOUT_MILLISLong or longDynamically sets the timeout in milliseconds to use when invoking the OFFER or POLL operations. 

IDGEN: Ignite ID Generator endpoint

The Ignite ID Generator endpoint allows you to interact with Ignite Atomic Sequences and ID Generators.

ignite:idgen:sequenceName?option1=value1&option2=value2...

This endpoint only supports producers.

Options

OptionTypeDefault valueDescriptionCompulsoryConsumer/producer

operation

IgniteIdGenOperation enum

---

The operation to invoke on the Ignite ID Generator.

Superseded by the IgniteConstants.IGNITE_IDGEN_OPERATION header in the IN message.

Possible values: ADD_AND_GET, GET, GET_AND_ADD, GET_AND_INCREMENT, INCREMENT_AND_GET.

NProducer

configuration

CollectionConfiguration
empty CollectionConfiguration

The configuration for this data structure. You can also specify inner properties by using the config. prefix.

For example: ?config.backups=4

NProducer

Initializing the Ignite component

Each instance of the Ignite component is associated with an underlying org.apache.ignite.Ignite instance. You can interact with two Ignite clusters by initializing two instances of the Ignite component and binding them to different IgniteConfigurations. There are 3 ways to initialize the Ignite component:

  • By passing in an existing org.apache.ignite.Ignite instance. Here's an example using Spring config:
<bean name="ignite" class="org.apache.camel.component.ignite.IgniteComponent">
   <property name="ignite" ref="ignite" />
</bean>
  • By passing in an IgniteConfiguration, either constructed programmatically or through inversion of control (e.g. Spring, Blueprint, etc.). Here's an example using Spring config:
<bean name="ignite" class="org.apache.camel.component.ignite.IgniteComponent">
   <property name="igniteConfiguration">
      <bean class="org.apache.ignite.configuration.IgniteConfiguration">
         [...]
      </bean>
   </property>
</bean>
  • By passing in a URL, InputStream or String URL to a Spring-based configuration file. In all three cases, you inject them in the same property called configurationResource. Here's an example using Spring config:
<bean name="ignite" class="org.apache.camel.component.ignite.IgniteComponent">
   <property name="configurationResource" value="file:[...]/ignite-config.xml" />
</bean>


Additionally, if using Camel programmatically, there are several convenience static methods in IgniteComponent that return a component out of any of these configuration options:

  • IgniteComponent#fromIgnite(Ignite)
  • IgniteComponent#fromConfiguration(IgniteConfiguration)
  • IgniteComponent#fromInputStream(InputStream)
  • IgniteComponent#fromUrl(URL)
  • IgniteComponent#fromLocation(String)

You may use those methods to quickly create an IgniteComponent with your chosen configuration technique.

General options

All endpoints share the following options:

OptionTypeDefault valueDescription
propagateIncomingBodyIfNoReturnValuebooleantrueIf the underlying Ignite operation returns void (no return type), this flag determines whether the producer will copy the IN body into the OUT body.
treatCollectionsAsCacheObjectsbooleanfalseSome Ignite operations can deal with multiple elements at once, if passed a Collection. Enabling this option will treat Collections as a single object, invoking the operation variant for cardinality 1.

OSGi Support

Apache Ignite supports OSGi from version 1.5.0.final onwards.

NOTES when installing on Apache Karaf:

  1. Installing the camel-ignite feature will require the Ignite feature repository to be present.
  2. You must have exported from the JRE (system bundle) some low-level, non-standard packages that Ignite requires.

Please refer to the OSGi section in the Ignite documentation for more information.

  • No labels

0 Comments