Introduction

This tutorial demonstrates how to create a very simple connection between the OODT File Manager and SIS. The connection is made by outputting an RSS feed of data from the File Manager via the REST API and configuring SIS to read in this RSS data.

This module has been removed from Apache SIS. This wiki page is kept as a source of material that we may use when web application development will restart.

Please note, this demonstration is very much a work-in-progress! It represents the first steps in making a connection between OODT and SIS. Work is ongoing and captured by issue OODT-402. As work continues it is hoped that a more formal, elegant and robust connection between OODT and SIS will be defined. This page will be updated accordingly to reflect the latest developments.

In the meantime, if you spot any typos, mistakes or feel that there are much better ways of doing things, please feel free to comment or update the wiki, send messages to the mailing lists or post to the OODT-402 ticket! Your comments will always be much appreciated!

Prerequisites / Assumptions

This tutorial assumes that you already have the OODT File Manager, OODT File Manager REST API and SIS webapp installed and running on your machine.

For the purposes of this demonstration, it is best to start with an empty File Manager ‘test’ repository, i.e. no files yet ingested into your repository, as we will be modifying the meta tags for the GenericFile product type.

(In the future, the aim is to define a separate ‘LocationAware’ product type that will not interfere with the GenericFile definition and can therefore be used with existing repositories.)

File Manager Configuration

The first step is to configure ‘Latitude’ and ‘Longitude’ meta tags for the GenericFile type in File Manager. This is achieved by editing the File Manager policy files (by default these are located in ‘/usr/local/oodt/filemgr/policy/core’ after installation).

Firstly, edit the ‘elements.xml’ file to add the following element definitions between the <cas:elements> … </cas:elements> tags:

<element id="urn:oodt:Latitude" name="Latitude">
  <description>Geographical latitude location.</description>
  <dcElement />
</element>

<element id="urn:oodt:Longitude" name="Longitude">
  <description>Geographical longitude location.</description>
  <dcElement />
</element>

Next, edit the ‘product-type-element-map.xml’ file and map the elements to the GenericFile product type by adding the following tags between the <type> … </type> tags:

<element id="urn:oodt:Latitude"/>
<element id="urn:oodt:Longitude"/>

After updating and saving the files, restart File Manager, for example by entering the following command from the File Manager bin directory:

./filemgr restart

Ingesting Sample Files

Create a few sample text files with associated metadata files. Ensure the metadata files contain metadata for the new ‘Latitude’ and ‘Longitude’ fields.

For example, in the /tmp directory create a ‘geodata.txt’ file containing a few arbitrary words of your own choice. Then create a ‘geodata.txt.met’ metadata file containing the following text:

<cas:metadata xmlns:cas="http://oodt.apache.org/ns/cas">

  <keyval>
    <key>Latitude</key>
    <val>10</val>
  </keyval>

  <keyval>
    <key>Longitude</key>
    <val>30</val>
  </keyval>

</cas:metadata>

Ingest the files into your File Manager repository. Below is an example command for ingesting files using the ‘filemgr-client’ tool from the File Manager bin directory for a File Manager instance running on port 9000.

./filemgr-client --url http://localhost:9000 --operation --ingestProduct \
--productName geodata.txt --productStructure Flat \
--productTypeName GenericFile \
--metadataFile file:///tmp/geodata.txt.met \
--refs file:///tmp/geodata.txt

File Manager REST API Webapp Configuration

Add the following tag definitions to the ‘rssconf.xml’ configuration file for the File Manager REST API webapp. By default, ‘rssconf.xml’ is located in the WEB-INF/classes directory of the webapp. The new tag definitions should be inserted between the <cas:rssconf> … </cas:rssconf> tags.

<tag name="geo:lat" source="[Latitude]">
  <attribute name="xmlns:geo" value="http://www.w3.org/2003/01/geo/wgs84_pos#" />
</tag>

<tag name="geo:long" source="[Longitude]">
  <attribute name="xmlns:geo" value="http://www.w3.org/2003/01/geo/wgs84_pos#" />
</tag>

The ‘xmlns:geo’ namespace definitions are required for ‘geo:lat’ and ‘geo:long’ tags to be interpreted correctly. The above inline solution was chosen as a first step to get the connection working, but there is probably a much more elegant way of doing this.

After updating the ‘rssconf.xml’ file, you can test your updates by doing the following:

1) Restart Tomcat, for example by entering the following commands from the Tomcat bin directory:

./shutdown.sh
./startup.sh

2) Navigate to ‘http://<host>/<webapp>/viewRecent?channel=ALL’. Where <host> is the host location for Tomcat and <webapp> is the directory of your File Manager REST API webapp. For example, ‘’.

After navigating to the above location, you should see an RSS output. Depending on your browser, you may have to ‘view source’ to see the XML. You should be able to see your new tags in the output, for example:

<rss xmlns:cas="http://oodt.apache.org/ns/cas" version="2.0">
  <channel>
    <title>ALL</title>
    <link>
      http://localhost:8080/fmprod/rdf/dataset?type=ALL&typeID=null
    </link>
    <description>ALL</description>
    <language>en-us</language>
    <copyright>Copyright 2010: Apache Software Foundation</copyright>
    <pubDate>Mon, 02 Apr 2012 23:24:07 BST</pubDate>
    <category>ALL</category>
    <generator>CAS File Manager</generator>
    <lastBuildDate>Mon, 02 Apr 2012 23:24:07 BST</lastBuildDate>
    <item>
      <title>geodata.txt</title>
      <description>GenericFile</description>
      <link>
        http://localhost:8080/fmprod/data?productID=532f56ef-7d12-11e1-90fa-8523ce23740f
      </link>
      <pubDate>Tue, 03 Apr 2012 00:22:25 BST</pubDate>
      <geo:lat xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#">10</geo:lat>
      <geo:long xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#">30</geo:long>
      <cas:source>GenericFile</cas:source>
      <source>GenericFile</source>
    </item>
  </channel>
</rss>

SIS Webapp Configuration

Add the URL used in the File Manager REST API to the ‘sis_location_config.xml’ file for the SIS webapp. By default, this is located in the WEB-INF/classes directory of the SIS webapp. Remove all of the other RSS URLs from the file. After editing the file, it should look similar to the following:

<sis:locationConfig xmlns:sis="http://sis.apache.org/ns/sis">
  <capacity>4</capacity>
  <depth>10</depth>
  <url>http://localhost:8080/fmprod/viewRecent?channel=ALL</url>
</sis:locationConfig>

Restart Tomcat to ensure that the changes to the configuration file are detected. [Note: If you have previously run the ‘demo.jsp’ SIS webapp demonstration page, it may be necessary to clear out the stored data from the webapp. This can be done by deleting the folders ‘geodata’ and ‘qtree’ from the main SIS webapp directory inbetween stopping and starting Tomcat. The folders will be recreated by the webapp when the next query is run.]

Running The Demo

To demonstrate the connection we'll use the 'demo.jsp' demo from the SIS web application. Open a web browser and navigate to http://<host>/<webapp>/demo.jsp, where <host> is the Tomcat location and <webapp> is the SIS webapp folder, for example: ‘http://localhost:8080/sis/demo.jsp’.

Enter data in the query form to return an area that should encompass the points defined in your metadata files. For example, the following bounding box query should return an area large enough to include the point defined in the ‘geodata.txt.met’ file:

Lower Left: Latitude: 0, Longitude: 0
Upper Right: Latitude: 50, Longitude: 50

Press the ‘Query’ button and view the map onscreen. All being well, your File Manager files should be visible on the map, as in the following screenshot. Click on the markers on the map to display information about them and confirm that it is indeed your files that are being displayed:

  • No labels

5 Comments

  1. Ross, this is awesome! (smile) Keep it coming!
    Also, I didn't find any other wiki doc/guide to set up or start off with SIS.
    Maybe later you could add the link to SIS in the Introduction.

    1. Thanks Sheryl! I've had a lot of great advice and encouragement from Chris Mattmann and the OODT/SIS communities. I'm hoping to contribute some useful work to OODT/SIS and glad everything's going well so far :)

      You're right about the links - this wiki page definitely needs more links to other guides and information. I'll get on it ASAP.

      For the SIS installation guide, I'm not sure if there's a wiki page yet, but I found the 'README.txt' file in the SIS distribution had some very useful info. I'll see if it's possible if I can copy the info from there over to a new wiki page here.

  2. Hey Ross,

    Awesome work! Instead of using Google Maps you may want to use Leaflet (http://leaflet.cloudmade.com/) or OpenLayers (http://openlayers.org/) due to required API key for the map. This might matter if you are going to host this example on the SIS Wiki or OODT Wiki as the required API would be attached to you.

    Here's how a snippet of Leaflet code to setup the map:

    var map = new L.Map('map');
    // Setup the basemap      
    var osm = new L.TileLayer('http://otile{s}.mqcdn.com/tiles/1.0.0/osm/{z}/{x}/{y}.png', {
        minZoom: 1,
        attribution: 'Basemap <a href="http://creativecommons.org/licenses/by-sa/2.0/" target="_blank">CC-BY-SA</a> by &copy; <a href="http://openstreetmap.org/" target="_blank">OpenStreetMap</a>, Tiles Courtesy of &copy; <a href="http://open.mapquest.com" target="_blank">MapQuest</a>', 
        subdomains: ['1','2','3','4']
    });
    // Add the basemap to the map
    map.addLayer(osm);
    
    // Create a marked for the map and add it
    var marker = new L.Marker(new L.LatLng(10.0, 30.0));
    map.addLayer(marker);
    
    // Attach a given HTML content to the marker. For brevity I truncated the content
    marker.bindPopup("Title: geodata.txt<br/>Link:<a href='http://localhost:8080...>...</a>");
    

    Unlike the GoogleMap API this does not require a key and is an open source product as is the basemap (e.g. Open Street Map served by Mapquest);

    Then this would be a complete open source solution from top to bottom. If you have any questions feel free to hit me up on the SIS or OODT lists.

    1. Hi Paul!

      Great suggestion! Leaflet looks good. If you and the SIS team are happy for me to do so, I'd definitely like to have a go at producing a new demo.jsp file that uses Leaflet. I'll keep you updated on my progress by posting to the mailing lists.

    2. (I accidentally posted a duplicate comment here for my reply to Paul above... I can edit this text but I can't remove the comment itself. If you have 'remove comment' permissions please feel free to remove this comment!)