Configurable look-and-feel

This change is being introduced in 7.1.1 to resolve the following feature request:

https://issues.apache.org/jira/projects/JUNEAU/issues/JUNEAU-82

 

The change introduces an external folder in the JVM working directory containing images and stylesheets used by the microservice:

The REST configuration section can be used to tailor the header and footer on the pages:

The BasicRestConfig interface (which defines the default settings for BasicRestServlet) is now defined as:

Note that the theme files are externally accessible and can be modified to produce any look-and-feel you desire.

The microservice still works without the files directory.  An embedded devops.css is included in the jar as a default spreadsheet.

If you're testing out changes in the theme stylesheets, you may want to set the following system property that prevents caching of those files so that you don't need to restart the microservice each time a change is made:

 

Quick update on the Swagger UI enhancements.

The design allows you to pull in Swagger from any of the following sources (or any combination):

  • Localized Swagger JSON files on the classpath.
  • Swagger defined on class-level and method-level annotations.
  • Swagger fragments defined in resource bundles.
  • Auto-detection if not found above (paths, methods, schemas, etc...).

The general idea is you can let Juneau auto-generate the Swagger documentation, or provide your own Swagger JSON, or do a combination of auto and manual.

 

Here's the Petstore app OPTIONS page based on the Swagger.io Petstore example...

 

Operations are expandable and gives you information about the schema.  

In this case, the schema information was auto-detected and generated using the JsonSchemaSerializer. 

 

Examples for all the supported types are provided via a select when an @Example-annotated method exists on the POJO class (or otherwise defined in the Swagger JSON).

 

For example, RDF....

 

 

 

Swagger UI enhancements

A quick update on the work I've been doing around Swagger enhancements.  There's still much work to do, but I wanted to provide a sneak-peek.

As previously requested, it would be nice to have more user-friendly OPTIONS pages based on Swagger UI. 

Here's an example of what it looks like so far...

It's implemented as a per-media-type PojoSwap<Swagger,Div> that replaces a Swagger object with HTML5 beans when rendered as HTML.

 

The examples can be embedded in the Swagger JSON or annotations. They can also come from the beans themselves via a new @Example annotation that can be applied to static methods on a bean:

These examples beans are then serialized to all the supported languages and inserted into the generated Swagger.

 

Here's what it looks like in the light and dark themes...

 

 

There are also general improvements coming to the Swagger API itself such as better support for definitions and references, and other fixes for pain-points I'm discovering along the way.

 

Documentation Updates

I think I'm finally finished cleaning up the documentation.  I've been working at it since December and it's been quite time consuming, but I'm pretty happy with the results:

http://juneau.apache.org/index.html#documentation.html

There's a LOT of documentation to go through, but anyone willing to get into the depths of Juneau and do some proofreading?  You might even find some interesting features you never knew existed. (smile)

Configuration API changes

We've made some significant changes to the Configuration API in 7.1.0.

Feature include:

  • Support for pluggable storage.
  • File-system watcher integration support.
  • New builder-based design.

The documentation has been rewritten for the juneau-config module and can be found here:

http://juneau.apache.org/site/apidocs/overview-summary.html#juneau-config

Homepage header image vote

Anyone have any preferences on changing the page header on our web page?

All but the first one are pulled from pexels.com.

1.

2.

3.

4.

5.

6.

7.

8.

9.

10

 

One last one that I kind of like....

 

FYI....the topics in the Javadoc overview document are now collapsable with zoom-in/out hover pointers:

(Technically they were always collapsable, but it wasn't obvious that you could do so without the mouse pointer)

Javadoc Link Tester

FYI....I just added a utility for finding broken links in the generated Javadocs.  I found hundreds of them.

It's just a simple class called JavadocLinkTester.  Just run it without args after running mvn javadoc:aggregate and it will find any broken links..

 

I recommend using it if you make any extensive updates to the Javadocs.

I looked for existing tools online but came up empty.  I also tried using the W3C link checker utility, but it was REALLY slow since it actually crawled all the links.

It would be nice to include this as part of the Maven build but I'm not familiar with creating Maven components.  

Source can be found here:

https://github.com/apache/juneau/blob/master/juneau-core/juneau-core-test/src/test/java/org/apache/juneau/utils/JavadocLinkTester.java

Support for POJO builders.

The Juneau parsers now support parsing into beans created through builders:

A typical code pattern is shown below:

This feature will be added in 7.1.0.

Full details can be found here:

http://juneau.apache.org/site/apidocs/overview-summary.html#juneau-marshall.PojoBuilders

 

 

Swagger DTO updates

I just delivered some significant updates to the Swagger DTO API.  These will be delivered as a part of 7.1.0.

 

New generalized getter and setter methods allow you to now set arbitrary fields:

The get/set methods can be used on existing properties as well.

The helper setter methods can now take in JSON as well as beans:

 

Other changes:

  • The toString() method serializes directly to JSON.
  • All the Javadocs have been cleaned up.
  • Added 250+ JUnit testcases to cover the Swagger DTO.

It may not be well known that we maintain extensive package-level Javadocs in our code.  Whereas the overview document is meant to provide a general overview of the features in Juneau, the package-level docs are intended to provide in-depth information.

To help make these docs more visible, I've added "Additional Information" sections to the overview document that mirror the TOCs of the package-level docs.

Here's an example:

Caveat - Some of these package-level docs need some cleanup.  Also, some of the detailed information currently in the overview document can probably be moved into the package-level docs now that the subtopics are more visible.

 

I've made some improvements to the APIs involved with serializing and parsing HTTP parts (e.g. headers, form data, query parameters, path variables).  This was based on pain-points experienced by the Apache Streams team using the client and proxy APIs.  My hope is that this simplifies their code.

I've added a new package in the juneau-marshall project:

I removed the part handling code in the UON and URL-encoding marshallers and put them in these classes.  

The SimpleUonPartSerializer implementation treats everything as plain-text except for beans and maps (which continue to be serialized as UON objects).  This is now the DEFAULT part marshaller in both the client and server APIs.  So if you were creating your own part serializers (or using StringReaders) to get around how strings were being serialized by UrlEncodingSerializer, this change lets you get rid of them.

This change sacrifices "purity" for "practicality".  It should make adoption considerably easier.

Note that this replaces the existing PartSerializer/PartParser/PartType classes, so there will be code adoption.  But it should be straightforward.

Refer to the 7.0.2 release notes for more information:
http://juneau.apache.org/site/apidocs/overview-summary.html#7.0.2

I've just delivered changes that improve the internals of the Serializer/Parser APIs.  I've been working on this for about a month.  I plan on introducing these changes in 7.0.2.

The general idea is to standardize on the following design:

  • Context - A thread-safe read-only object (e.g. BeanContext, JsonSerializer).
  • ContextBuilder - A thread-safe builder for contexts (e.g. BeanContextBuilder, JsonSerializerBuilder).
  • Session - A non-thread-safe single-use object with configuration combined from context and runtime args (e.g. BeanSession, JsonSerializerSession).
  • PropertyStore - A thread-safe read-only set of configuration properties.  Each Context contains one PropertyStore.
  • PropertyStoreBuilder - A thread-safe builder for PropertyStore objects.  Each ContextBuilder contains one PropertyStoreBuilder.

 

Here are the highlights:

  • Caching improvements on serializers and parsers have reduced execution time of the core JUnits by approximately 1/3.
    The 17000+ JUnit tests now execute in less than 10 seconds (down from around 13s) and have a cache-reuse hit rate of 98% (164104 serializers/parsers/bean-contexts retrieved, but only 1801 created from scratch).

  • All the various separate *Context classes (e.g. JsonSerializerContext) have been folded into their respective serializer or parser classes (e.g. JsonSerializer).
    Additionally, these classes are their own bean contexts.
    For example, the class hierarchy of JsonSerializer is now:


 

  • Session objects also now have a consistent class hierarchy.
    For example, the class hierarchy of JsonSerializerSession is now:

 

  • Builder objects also now have a consistent class hierarchy.
    For example, the class hierarchy of JsonSerializerBuilder is now:


 

  • The PropertyStore class has been completely rewritten.
    It is now a read-only configuration store build using the PropertyStoreBuilder class.
    The previous PropertyStore class was overly-complicated with many read/write locks to ensure thread-safety.
    The new design shifts to a builder-based model with read-only PropertyStore 'snapshot-in-time' objects.
    PropertyStores can be used as hash keys to allow caching and reuse of Serializers and Parsers.

  • ContextBuilders are created by Context.create() and Context.builder() methods and Contexts are created by the ContextBuilder.build().
    Examples:
    • JsonSerializer s = JsonSerializer.create().simple().sq().build();  // Create a serializer from scratch.
    • JsonSerializer s = JsonSerializer.DEFAULT.builder().simple().sq().build();  // Clone and modify an existing serializer.

       

 

Web page updates

A few web page updates....

I've simplified the ABOUT page and moved the rest of it into a separate COMPONENTS page with subpages....

I also added a Table of Contents section to the DOCUMENTATION page since it seemed kinda empty...

 

That is all...

 

In an attempt to help determine the cause of the intermittent REST call failures in the juneau-microservice-test project, I've made some debugging enhancements to the microservice.

The test project will now generate 4 log files...

 

The jetty-thread-dump.log file is generated by Jetty on the event of a failed request and shows a snapshot of the current state of the threads in Jetty.  The retry handler in TestMicroservice is shown below and will stop the tests as soon as the first error occurs...

 

The jetty-requests.log is generated by adding this to the jetty.xml file...

This just shows if Jetty actually got the request from the last call.

 

Note that the jetty.xml file now supports embedded Juneau variables (e.g. $C{Logging/logDir}).

I also created a separate Jetty section in the config file...

Note that we now support dynamically specifying the port again through the use of the $S{availablePort} variable.

 

The test.0.log file now contains the entire debug logging generated by Jetty (yea...it's big...7.8MB).  I had to create a new org.apache.juneau.microservice.JettyLogger class to serve as a simple bridge between Jetty logging and java-util-logging, but it now allows all the logging to be directed to a single log file.  The configuration for doing this is shown below:

If an error is occurring somewhere in Jetty, it should show up here.

 

The third-party-proxy-resource.txt file contains log entries from RestHooks added to the ThirdPartyProxyResource class (which seems to be a common place for failure).  This makes sure that the failure isn't occurring within our logic.  If we see START/PRE/POST/END messages for our calls, then we know we're not causing a problem.

 

If you do encounter testcase failure due to HTTP request errors, I recommend using WireShark listening on port 10001:

This should help show whether the requests appear to be failing on the server or client side...