Messaging with Fedora4

By default, Fedora4 publishes all events to a topic on a local broker. The topic is called fedora. Each message will contain an empty body and five different header values. Those header values are namespaced so they look like this:

  • org.fcrepo.jms.identifier
  • org.fcrepo.jms.eventType
  • org.fcrepo.jms.properties
  • org.fcrepo.jms.timestamp
  • org.fcrepo.jms.baseURL

Both eventType and properties are comma-delimited lists of events or properties. The eventTypes follow the JCR 2.0 specification and include:

The properties field will list the RDF properties that changed with that event. NODE_REMOVED events contain no properties.

Distributed Deployments

The default configuration is fine for locally-deployed listeners, but it can be problematic in a distributed context. For instance, if the listener is restarted while a message is sent to the topic, that message will be missed. Furthermore, if there is a networking hiccup between fedora's local broker and the remote listener, that too can result in lost messages. Instead, in this case, a queue may be better suited.

Supporting Queues

ActiveMQ supports “virtual destinations”, allowing your broker to automatically forward messages from one location to another. If fedora4 is deployed in Tomcat, the ActiveMQ configuration will be located in WEB-INF/classes/config/activemq.xml. That file can be edited to include the following block:

  <destinationInterceptors>
    <virtualDestinationInterceptor>
      <virtualDestinations>
        <compositeTopic name="fedora">
          <forwardTo>
            <queue physicalName="fedora"/>
          </forwardTo>
        </compositeTopic>
      </virtualDestinations>
    </virtualDestinationInterceptor>
  </destinationInterceptors>

Now a consumer can pull messages from a queue without risk of losing messages.

This configuration, however, will not allow any other applications to read from the original topic. If it is necessary to have /topic/fedora available to consumers, this configuration will be useful:

  <destinationInterceptors>
    <virtualDestinationInterceptor>
      <virtualDestinations>
        <compositeTopic name="fedora" forwardOnly="false">
          <forwardTo>
            <queue physicalName="fedora"/>
          </forwardTo>
        </compositeTopic>
      </virtualDestinations>
    </virtualDestinationInterceptor>
  </destinationInterceptors>

Now, both /topic/fedora and /queue/fedora will be available to consumers.

Distributed Brokers

The above example will allow you to distribute the message consumers across multiple machines without missing messages, but it can also be useful to distribute the message broker across multiple machines. This can be especially useful if you want to further decouple the message producers and consumers. It can also be useful for high-availability and failover support.

ActiveMQ supports a variety of distributed broker topologies. To push messages from both the message queue and topic to a remote broker, this configuration can be used:

  <networkConnectors>
    <networkConnector name="fedora_bridge" dynamicOnly="true" uri="static:(tcp://remote-host:61616)">
      <dynamicallyIncludedDestinations>
        <topic physicalName="fedora"/>
        <queue physicalName="fedora"/>
      </dynamicallyIncludedDestinations>
    </networkConnector>
  </networkConnectors>

Protocol Support

ActiveMQ brokers support a wide variety of protocols. The default Fedora4 configuration includes OpenWire and Stomp. If Fedora's internal broker is bridged to an external broker, please remember to enable the proper protocols on the remote broker. This can be done like so:

  <transportConnectors>
    <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
    <transportConnector name="stomp" uri="stomp://0.0.0.0:61613"/>
  </transportConnectors>

Each transportConnector supports additional options.

Camel

Apache Camel is middleware for defining how messages are processed. It makes filtering, aggregating, splitting and dispatching very easy. Writing Camel routes to work with Fedora is also very easy. For example, a simple route could be:

  <route id="fcrepo4-updates">
    <description>
        Read from the incoming queue and fetch the object from fedora as text/turtle.
    </description>
    <from uri="activemq:queue:fedora"/>
    <to uri="direct:fcrepo-object"/>
    <log message="BODY: ${body}"/>
  </route>
 
  <route id="fedora-object">
    <description>
      Given a path header, fetch the object from fedora.
    </description>
    <from uri="direct:fcrepo-object"/>
    <setHeader headerName="Exchange.HTTP_URI">
      <simple>${header.org.fcrepo.jms.baseURL}</simple>
    </setHeader>
    <setHeader headerName="Exchange.HTTP_PATH">
      <simple>${header.org.fcrepo.jms.identifier}</simple>
    </setHeader>
    <setHeader headerName="Accept">
      <constant>text/turtle</constant>
    </setHeader>
    <to uri="http4://fedora-host/"/>
  </route>

Though it may be easier to just use the fcrepo: component. The fcrepo: component implements much of what the http4 component offers, but it does so in such a way to make it easy to interact directly with fedora.

For example, an equivalent route as the above would be:

  <route id="fcrepo-updates">
    <description>
        Read from the incoming queue and fetch the object from fedora as text/turtle.
    </description>
    <from uri="activemq:queue:fedora"/>
    <to uri="fcrepo:localhost:8080/fcrepo4/rest?type=text/turtle"/>  
  </route>

Camel routes can be deployed as described in the section about Karaf.

fcrepo-messaging.txt · Last modified: 2014/10/28 11:03 by acoburn
 
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International