Introduction
This guide gives a brief overview of the CAS Product Server web application from a developer's perspective. It discusses the Java code and resources for the web application such as XML configuration files.
Overview
The goal of the CAS Product Server web application is to serve entities from a file manager, such as products and product types, via HTTP. Originally, the web application was written using Java Enterprise HTTP servlets. The first section below gives details about these servlets. Recently, an attempt is in progress to update the web application to use JAX-RS via Apache CXF.
The source code is located in the webapp/fmprod subdirectory of the OODT project. The application is built using Maven and therefore has a Maven POM build file in the project's root directory. It also tries to follow the standard directory layout for a Maven project.
The HTTP and JAX-RS servlets are all configured in a web descriptor (web.xml) file located in the src/main/webapp/WEB-INF directory. This is a standard web descriptor file that sets parameters for the servlets and maps them to specific paths on the webserver. There's also a context descriptor (context.xml) file located in the src/main/webapp/META-INF directory. This can be used to customize the behaviour of the deployed web application. In particular the context file is used to tell the web application where to find the running file manager and any output configuration files.
The Java source code for the project is located in src/main/java and the unit tests are located in src/test/java. Resource files, such as example configuration files and logging properties files, are located in src/main/resources and src/test/resources.
The original servlets are divided across several packages. Data delivery servlets (that deliver raw files and zip archives) and associated classes are in the org.apache.oodt.cas.product.data package. RDF servlets (that output information on sets of products in RDF format) and associated classes are in the org.apache.oodt.cas.product.rdf package. RSS servlets (that output RSS feeds about products and file transfers) and associated classes are in the org.apache.oodt.cas.product.rss package. The JAX-RS servlet and associated classes are in the org.apache.oodt.cas.product.jaxrs package and subpackages.
Original Servlets
The original servlets are divided into three categories: data delivery servlets that deliver files and zip archives, RDF servlets that deliver RDF feeds, and RSS servlets that deliver RSS feeds.
Data Delivery Servlets
The servlets DataDeliveryServlet and DatasetDeliveryServlet, along with associated classes for data output (i.e. files and zip archives), are located in the org.apache.oodt.cas.product.data package. DataDeliveryServlet is used for outputting product references as files, or complete products (together with metadata) as zip archives. DatasetDeliveryServlet is used to output sets of products (and their metadata) as zip archives.
RDF Servlets
The RDF servlets RDFProductServlet and RDFDatasetServlet, along with associated classes for RDF output, are located in the org.apache.oodt.cas.product.rdf package. RDFProductServlet outputs information about file manager products, whereas RDFDatasetServlet outputs information about file manager product types.
RSS Servlets
The RSS servlets RSSProductServlet and RSSProductTransferServlet, along with associated classes for RSS output, are located in the org.apache.oodt.cas.product.rss package. RSSProductServlet outputs information about file manager products, whereas RSSProductTransferServlet outputs information about the status of currently active file transfers.
JAX-RS Servlet
Maven POM
The Maven POM includes the following dependencies in the dependencies section:
The cxf-rt-frontend-jaxrs dependency is the main Apache CXF dependency for JAX-RS:
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-frontend-jaxrs</artifactId>
<version>2.6.8</version>
</dependency>
The httpclient dependency is used in a class called BackwardsCompatibilityInterceptor, where it is used to parse query parameters from request URLs:
<dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpclient</artifactId> <version>4.2.5</version> </dependency>
The Tika dependency is used to detect the MIME types of files:
<dependency>
<groupId>org.apache.tika</groupId>
<artifactId>tika-core</artifactId>
<version>0.8</version>
</dependency>
The zip4j dependency is used for creating zip archives:
<dependency>
<groupId>net.lingala.zip4j</groupId>
<artifactId>zip4j</artifactId>
<version>1.3.1</version>
</dependency>
The jettison dependency is used to marshal JAXB resources to JSON format:
<dependency>
<groupId>org.codehaus.jettison</groupId>
<artifactId>jettison</artifactId>
<version>1.3.4</version>
</dependency>
The cxf-rt-rs-extenion-providers dependency is used to automatically map file extensions that are included in the URL (such as .xml, .json, .rss) to content types such as application/xml, application/json and application/rss:
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-rs-extension-providers</artifactId>
<version>2.6.8</version>
</dependency>
The cxf-rt-transports-local, easymock, junit and xmlunit are useful for unit testing. For example, cxf-rt-transports-local provides a local transport protocol to send HTTP requests to a JAX-RS service class within unit tests:
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-transports-local</artifactId>
<version>2.6.8</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.easymock</groupId>
<artifactId>easymock</artifactId>
<version>3.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>xmlunit</groupId>
<artifactId>xmlunit</artifactId>
<version>1.4</version>
<scope>test</scope>
</dependency>
</dependencies>
Web Descriptor
The web.xml file, located in the src/main/webapp/WEB-INF directory, is used to configure all of the servlets in the web application. The JAX-RS package only uses one servlet and it is configured as shown below.
<?xml version="1.0" encoding="UTF-8"?>
<web-app ...>
...
<servlet>
<servlet-name>CasProductJaxrsServlet</servlet-name>
<servlet-class>
org.apache.oodt.cas.product.jaxrs.servlets.CasProductJaxrsServlet
</servlet-class>
<init-param>
<param-name>jaxrs.serviceClasses</param-name>
<param-value>
org.apache.oodt.cas.product.jaxrs.services.CasProductJaxrsService
</param-value>
</init-param>
<init-param>
<param-name>jaxrs.providers</param-name>
<param-value>
org.apache.cxf.jaxrs.provider.json.JSONProvider,
org.apache.oodt.cas.product.jaxrs.writers.ReferenceFileWriter,
org.apache.oodt.cas.product.jaxrs.writers.ReferenceZipWriter,
org.apache.oodt.cas.product.jaxrs.writers.ReferenceRssWriter,
org.apache.oodt.cas.product.jaxrs.writers.ProductZipWriter,
org.apache.oodt.cas.product.jaxrs.writers.ProductRssWriter,
org.apache.oodt.cas.product.jaxrs.writers.DatasetZipWriter,
org.apache.oodt.cas.product.jaxrs.writers.DatasetRdfWriter,
org.apache.oodt.cas.product.jaxrs.writers.DatasetRssWriter,
org.apache.oodt.cas.product.jaxrs.writers.TransfersRssWriter
</param-value>
</init-param>
<init-param>
<param-name>jaxrs.inInterceptors</param-name>
<param-value>
org.apache.oodt.cas.product.jaxrs.filters.BackwardsCompatibleInterceptor
</param-value>
</init-param>
<init-param>
<param-name>jaxrs.scope</param-name>
<param-value>prototype</param-value>
</init-param>
<init-param>
<param-name>jaxrs.extensions</param-name>
<param-value>
file=application/octet-stream
json=application/json
rdf=application/rdf+xml
rss=application/rss+xml
xml=application/xml
zip=application/zip
</param-value>
</init-param>
</servlet>
<servlet-mapping>
<servlet-name>CasProductJaxrsServlet</servlet-name>
<url-pattern>/jaxrs/*</url-pattern>
</servlet-mapping>
</web-app>
The JAX-RS servlet class is the CasProductJaxrsServlet class in the org.apache.oodt.cas.product.jaxrs.servlets package. It is mapped to the path /jaxrs/* by the <servlet-mapping> settings shown above. Several parameters are also set for the servlet class. These are:
- jaxrs.serviceClasses (a list of classes that handle RESTful HTTP requests)
- jaxrs.providers (a list of classes that provide outputs in specific formats)
- jaxrs.inInterceptors (a list of classes that can intercept and modify incoming requests)
- jaxrs.scope (the scope of service classes for the servlet, set to 'prototype' to create new instances per request)
- jaxrs.extensions (a mapping of extensions to content types, where extensions are similar to file extensions and can be used in URLs)
Currently, there is a single service class CasProductJaxrsService from the org.apache.oodt.cas.product.jaxrs.services package, a single interceptor class BackwardsCompatibleInterceptor from the org.apache.oodt.cas.product.jaxrs.filters package, multiple providers that each deal with specific resources being output in specific formats, and multiple extension mappings. More details about each class are given in the sections below.
Servlet
The servlet class CasProductJaxrsServlet is located in the org.apache.oodt.cas.product.jaxrs.servlets package. It is the only servlet used by the JAX-RS part of the web application. The service class extends Apache CXF's org.apache.cxf.jaxrs.servlet.CXFNonSpringJaxrsServlet. This provides a simple servlet that can be configured via the web descriptor (web.xml). Aside from basic initialization, CasProductJaxrsServlet also sets up the file manager client and reads in custom configuration files, storing them as attributes in the servlet's context object.
Service
The single service class CasProductJaxrsService is located in the org.apache.oodt.cas.product.jaxrs.servlets package. This class handles all of the HTTP requests for the JAX-RS part of the application. Most of the methods in the service class return resource objects that are marshalled or converted into the appropriate format (see sections below for more details).
Resources
The resource classes are all located in the org.apache.oodt.cas.product.jaxrs.resources package. These are JAXB-annotated representations of file manager entities such as references, products, metadata, file transfers and groups of products (datasets). All of the resource classes have a suffix Resource.
Writers
The writer classes, all located in the org.apache.oodt.cas.product.jaxrs.writers package, are used to provide outputs for specific resources in specific formats. For example, the DatasetRssWriter class provides outputs for instances of the DatasetResource class in application/rss+xml format. All of the writer classes implement the javax.ws.rs.ext.MessageBodyWriter interface and use annotations to specify the content type that they provide. This helps the application to select the appropriate writer to suit the request.
Within these classes, the writeTo method is where we write the desired output to an output stream that is later sent in the response. In this method, we are free to use any available technique to generate the format we need, provided that it can be written to the output stream.
Configurations
The org.apache.oodt.cas.product.jaxrs.configurations package is where classes that represent custom configurations are located. These classes, such as RssConfiguration and RdfConfiguration are used to store configuration properties that are read in from XML configuration files. The XML files are specified in the context descriptor and instances of configuration classes are created and initialized by the CasProductJaxrsServlet servlet class (see above sections for more details).
Filters
The JAX-RS servlet and service classes use a different URL scheme to the original servlets. The intention is to one day replace the original servlets with the JAX-RS servlet and adopt the new URL scheme entirely. In the meantime, during the transition it is useful to have some backwards compatibility with the URLs from original web application.
There are several ways to ensure backwards compatibility. One approach that was experimented with was to introduce another service class just to handle backwards-compatible URLs. But this proved to be a slightly fiddly and intrusive upon the other service class. Instead an alternative approach was taken to use an interceptor class instead. The class BackwardsCompatibleInterceptor in the org.apache.oodt.cas.product.jaxrs.filters extends org.apache.cxf.phase.AbstractPhaseInterceptor. It is used to intercept incoming requests and check the request URL. If a URL matching the original scheme is found, this URL is modified to match the new URL scheme, i.e. it is mapped over from the old scheme to the new scheme.
Exceptions
There are several classes within the org.apache.oodt.cas.product.jaxrs.exceptions package. These all extend javax.ws.rs.WebApplicationException. When a WebApplicationException is thrown it can be used to return an HTTP response. The extension classes, BadRequestException, InternalServerErrorException and NotFoundException are used to return specific messages and response codes.
Development Ideas
The JAX-RS package is very much a work-in-progress. This latest effort was initiated as a student project for Google Summer of Code (2013). But the idea of upgrading the web application to use JAX-RS has been around for some time. Moreover, a lot of great work has already been carried out in other areas of OODT to introduce JAX-RS using Apache CXF.
The output of the Summer of Code project is a basic reworking of the original servlets, minus a few features (but adding a few features in other places). There are many areas that could be improved and expanded. Below are some ideas:
- Redesign the configurations package (the current design is a 'get it working' design and a bit ugly)
- Use ROME for feed writers, i.e. create classes such as ProductFeedWriter to handle all feeds (including RDF, RSS, Atom) to replace individual writers such as ProductRdfWriter and ProductRssWriter. A stumbling block might be the requirement to configure the outputs, and whether ROME's custom modules need to be hard-coded or can be configurable/dynamic.
- Introduce configurable XML and JSON outputs.
- Introduce a ProductTypeResource to represent product types.
- Leverage the full power of OODT. Currently the JAX-RS service only uses basic calls to an XmlRpcFileManagerClient instance. It doesn't yet take advantage of the OODT Product Server, for example to handle large products.
Further Information
Further information can be found in the OODT trunk Javadocs and by contacting the user and developer mailing lists.