This page describes a proposal for an intermediate XML format in which FOP's area tree can be serialized and reread for delayed rendering. I've been approached by two parties who would like to have such a feature in FOP.

As is indicated below, the currently implemented intermediate format needs a different approach. The design ideas and discussion are found on the AreaTreeIntermediateXml NewDesign page.

Requirements

• The area tree should be serializable in an XML format so it can be easily manipulated.
• The intermediate XML format should later be reread and normally rendered through FOP's Renderers.
• It should be possible to combine XML files from multiple rendering runs into one document which is then rendered by FOP's Renderers.
• The intermediate format should make it possible to add graphical elements after processing, for example for OMR marks and barcodes for packaging control.
• Allow imposition functions, two-up, n-up, page overlays, scaling and rotation of whole pages.
• One party that approached me wants to do manual modifications on leaf level nodes to modify the appearance of the document, for example, squeezing in one more line of text or something like that. Adventurous, but if they are so brave...
• The implementation must not have a negative impact on performance for people who don't use the intermediate format.
  

Benefits

The intermediate format allows various manipulations of the layouted document. People can do stuff that they can't do during the XSL-FO stage and that they'd have to do by post-processing FOP's output, but after they already lost some information on the way (like the simple-page-master used by a page).

Possible Problems

• The area tree is currently not 100% self-sufficient. There are references to the FO tree, for example. (this has been resolved)
• Different renderers provide different font sources. Rendering a document with a different font source from the one the document was laid out with may produce unexpected results.
• OffDocumentItems are direct signals to the Renderers which not really part of the area tree. Therefore, they are not serialized with the intermediate format. The PS-Extensions can be improved to make them XMLizable, but BookmarkData may be a little more complicated. (this has been resolved)
  

Implementation notes

• The first step will be to make the area tree self-sufficient to make it really reloadable.
• There is a CachedRenderPagesModel which is supposed to save pages in a file as long as they can't be rendered. This class is currently not used in FOP, but it would be good to make sure this class works again which will be a long way towards making sure that the first step is done.
• Serialization to XML can be implemented in two ways (the first has been chosen):
  • The XMLRenderer is used. The problem here is that the renderer would have to be changed in a way that almost certainly would require changes in many layout engine test cases.
  • Serialization is implemented directly on the area classes by generating SAX events (like in ExampleObj2FO). This basically means duplicate work when we already have the XMLRenderer but it places the functionality in the "right" place and leaves the freedom to design this format as needed by the tasks that will be performed with it.
• The Bean-to-XML serialization available since JDK 1.4 cannot be used due to JDK restrictions.
• It's probably best to implement parsing via SAX again to avoid dependencies on other XML libraries like Digester.
• Adding a wrapper around a PageViewport to do things like two-up and overlays should be easy. The Renderers will be taught to handle this additional wrapper object and will simply set up the target coordinate system accordingly. This will have no influence on the relative coordinate handling while rendering the actual area objects. This could also be implemented directly on the PageViewport but with the wrapper the actual area tree can be left untouched.
• Foreign XML content (from i-f-o) will be serialized in their proper namespaces in the intermediate format. That allows the XMLHandlers to still optimize the rendering of foreign XML dialects. Some improvements in the infrastructure around this may be necessary, but should not be a big deal.
• We will need to provide an API to handle the generation of the area tree XML (this is done through the normal XMLRenderer now). Furthermore, we need functions to load the intermediate format and to render it (see AreaTreeParser).
• It should be noted that with this design, there will be no direct support for elements in the area tree namespace to perform painting operations (for shapes, for example). Additional graphics will need to be included as ForeignObjects in SVG or another format wrapped in a positioned Block area.
  

Additional ideas

• Someone could come up with a FOP extension which includes an intermediate format XML file which could automatically be included at a certain point while rendering the main document (Document insertion). If someone implements PDF parsing, the same could be done for PDF.
  

Coding log

• Trying to reactivate the CachedRenderPagesModel shows several problems with the current area tree: Instances of ColorTypeProperty and Font, for example are not serializable. Being a Serializable is not a strict requirement for implementing the intermediate format but it shows problem spots on the way to a fully reloadable area tree.
• Fonts are a special problem: It uses the internal key ("F1", "F2" etc.) to point to the right font. For the area tree, this should be reworked to contain the info on the exact font used during layout. The renderer will have to lookup the font once more to get at the font metrics if it needs them. If a serialized area tree is deserialized later, we need to be able to find out if a font is suddenly not available anymore. We shouldn't do any font substitution anymore in this case. More details here: http://mail-archives.apache.org/mod_mbox/xmlgraphics-fop-dev/200512.mbox/%3c20051222140018.6FF8.DEV@jeremias-maerki.ch%3e
• The first version is now in SVN: Revision 369753. It doesn't support advanced operations such as n-ups, yet.
  

Feedback from users

2007-02-28: The intermediate format has been available for quite some time now. Users seem generally quite happy with the functionality. Two problems have been identified, however:

• The current approach is too slow. The parser is quite complicated and not very fast (ex. many lookups necessary). The renderers contain quite a bit of logic to convert the area tree to the output format.
  • Profiling on a document without custom fonts and images:
    • Rendering takes about 50% of the time
    • AreaTreeParser$Handler.startElement() takes about 32% (a lot of overhead due to Map operations for traits during area tree build-up)
• The XML format is a little complicated and too difficult to generate by hand and to get right.
  

The original approach was tempting because it promised relatively quick results by reusing as much as possible. The main intention of the intermediate format is to have FOP process a document as far as possible, so that later in the final step the ultimate output format can be generated as quickly as possible. The use case behind this: Imagine a high-volume document production system where a lot of documents come in during the day. They are formatted as they come in (so they CPU consumption can be distributed during the day). At some point the printing department decides to print the queued documents. An operator issues a command to generate a big print job containing all queued documents. The documents (saved in intermediate format) are concatenated and enriched using OMR marks or barcodes for automated packaging and other things like job info pages and finally, they are run through a renderer to generate the desired output format (often PostScript, AFP or PCL). This final task has to be very fast so the operator doesn't have to wait too long for the print job to be available for printing. Of course, the same could be done working in the actual output format (i.e. directly in PostScript or AFP), but that means that this functionality would have to be implemented for each output format which with the intermediate format you'd have to do it once.

So, the idea now is to specify a new intermediate format which will cover the original requirements (as seen above) and fix the above two problems. The rough idea is to simplify the intermediate format. That will mean that we will need a different approach for the Renderers. In the short term the original renderers will remain untouched. Maybe at some point they can be replaced by the new counterparts which are expected to be smaller and easier to develop since they don't have to cover the same amount of functionality as the current ones. The current XMLRenderer will likely remain in place for the time being if only to help with unit testing the layout engine.