Message Routing

The inter-application messages in this site are handled by Apache Camel, running in a Karaf (OSGi) container. This makes managing the message routing extremely easy, meaning that I simply copy an XML file into a directory to deploy or update a given set of routes.

Camel routes can be written entirely in XML, defining how a message moves from one endpoint to another, possibly with a series of intermediate transformations. Camel implements something called “Enterprise Integration Patterns” which is a jargon-y way of describing abstract patterns that most asynchronous messaging systems (sooner or later) tend to follow. There is a little learning curve here, but it is well worth the effort.

The routes currently in place handle the integration between fedora and a number of separate services. The code for these is available below, written entirely in Blueprint XML. I have split the routes across several different camel contexts to make things easier for me, but for simple applications, they can all be in a single file. Routes that cross camelContexts are bridged with the vm component, which provides an asynchronous, in-memory queue across applications. Communications between routes within the same camelContext use the seda component.

The main components at present include:

Logging with Camel

There are two ways to handle logging when using Camel. For simple, human-readable notifications, such as “adding PID to Solr”, the <log message=“…”/> construct can be used. For example, this could be used in your route.

  <log message="Added ${header.pid} to Solr"/>

Sometimes, however, it is useful to log the message body and/or headers without turning every component to the DEBUG level. For that, the log component is useful:

  <to uri="log:edu.amherst.acdc.solr?level=DEBUG"/>

When this endpoint is encountered, the entire message body is sent to the log. Most of our production routes have this type of logging present. During typical operation, these messages are not sent to the log (set at INFO), but they can easily be turned on and off without restarting the route. In ${karaf.home}/etc/org.ops4j.pax.logging.cfg, I can turn on logging for particular components by setting this line to the proper logLevel:

  log4j.logger.edu.amherst.acdc.solr=DEBUG

Karaf picks up the change immediately and begins logging DEBUG events generated by that class.

Using Property Placeholders

Many of the camel routes in this system require embedding usernames, passwords and other details that I would prefer not to have explicitly stated in the XML. Using property placeholders, I can specify certain values in an external configuration file while committing the route XML to a version control system (without passwords). This can be done with both Spring and Blueprint style XML files, though Blueprint allows you to use these properties outside of a <camelContext>.

In Blueprint, for example, properties can be accessed both inside and outside the <camelContext> node. When inside the camelContext, however, properties are referred to like so:

  {{fedora.user}}

Outside the camelContext, the same property would be accessed like so:

  ${fedora.user}

A more complete example is below, using Blueprint XML. In this case, the property values come from a file called acdc.fedora.cfg stored in ${karaf.home}/etc.

  <cm:property-placeholder persistent-id="acdc.fedora"/>
 
  <sslContextParameters id="sslContextParameters" xmlns="http://camel.apache.org/schema/blueprint">
    <keyManagers keyPassword="truststore">
      <keyStore
          resource="${truststore.file}"
          password="${truststore.password}"/>
    </keyManagers>
    <trustManagers>
      <keyStore
          resource="${truststore.file}"
          password="${truststore.password}"/>
    </trustManagers>
  </sslContextParameters>
 
  <camelContext id="fedora-routes" xmlns="http://camel.apache.org/schema/blueprint">
    <route id="fedora-enhancer">
      <from uri="seda:aggregated"/>
      <setHeader headerName="Exchange.HTTP_METHOD">
        <constant>GET</constant>
      </setHeader>
      <setHeader headerName="Exchange.HTTP_PATH">
        <simple>/fedora/objects/${header.pid}/objectXML</simple>
      </setHeader>
      <to uri="http4://{{fedora.host}}:{{fedora.port}}/?authUsername={{fedora.user}}&amp;authPassword={{fedora.password}}"/>
      <to uri="seda:insert"/>
    </route>
  </camelContext>
 
  <bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
    <property name="brokerURL">
      <value>failover:(ssl://host1:port,ssl://host2:port)?randomize=true</value>
    </property>
    <property name="userName" value="${activemq.user}"/>
    <property name="password" value="${activemq.password}"/>
  </bean>
message_routing.txt · Last modified: 2013/04/03 14:05 by acoburn@amherst.edu
 
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International