Versions Compared

Old Version 4

changes.mady.by.user Andrew Phillips

Saved on

compared with

New Version Current

changes.mady.by.user Ignasi Barrera

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

* Until we switch to JUnit (where we'd use test rules), use a base test class. This should setup a field for the server and tear it down, and hold assets that are shared across tests. This makes the subtypes easier to read and also reduces errors. [Example in GCE](https://github.com/jclouds/jclouds-labs-google/blob/master/google-compute-engine/src/test/java/org/jclouds/googlecomputeengine/internal/BaseGoogleComputeEngineApiMockTest.java).
* Only use MockWebServer's port; Do not use its host. Test hosts in the cloud often botch hostnames. Just point anything that uses the mock server's url. Ex. `"http://localhost:" + server.getPort() + path`.

...

* Test anything "interesting" carefully. Particulary with pagination, test the second page, not just the first. It is "ok" to pin this to a separate class while working on other things, but at least have one class that tests pagination using multiple http responses. For example, in [GCE](https://github.com/jclouds/jclouds-labs-google/blob/master/google-compute-engine/src/test/java/org/jclouds/googlecomputeengine/internal/ToIteratorOfListPageExpectTest.java).
* Don't do multiple "api tests" in the same method. Ex. doing a list followed by a get in the same test method makes the test harder to understand than making different tests. The only reason to load multiple responses is if there's an implicit extra request due to authentication, or there's a continuation like pagination, or you are testing a complex type such as `BlobStore` which needs multiple http requests for the same command.

...

For each method that returns paginated results, you will need two methods in the api: one without parameters that returns the entire list, and the other one, with the pagination parameters, that is able to return a single page. In jclouds:

  • The first one, returning a single page, must return an IterableWithMarker<T>, which is just an Iterable with a marker that points to the next page.
  • The second one, the entire list, must return a PagedIterable<T>, which is an Iterable that has a single page (an IterableWithMarker<T>) and knows how to get the next page given the next page marker.

Having this in mind, you need to figure out two things:

  • How to build the IterableWithMarker<T> so it contains a marker with all the info required to fetch the next page.
  • How to build the PagedIterable<T> to automatically fetch pages given a marker.

Building the first one should be pretty straightforward. You just need to create a class that extends theIterableWithMarker<T> abstract class and make sure the returned marker has all the info to fetch the next page. In your example, the "meta" fields could serve this purpose. Once you have your implementation, you just need to create a ResponseParser that builds your class given an HttpResponse. That's what thethe ParseImages class in Glance does.

Next thing you have to do, is build a PagedIterable<T> given an IterableWithMarker<T>. Since the APIs always return a single page, in the end, the HttpResponses for the "listAll" and "listSinglePage" methods will be the same. That's why most "listAll" methods use the same response parser than above, and then use a transformation function to transform the resulting IterableWithMarker<T> into a PagedIterable<T>(see the ImageApi annotations).

So, that ToPagedIterable is just a function that transforms an IterableWithMarker<T> into aPagedIterable<T>. This can be done in many ways, depending on the API. This functions usually build the iterable using the helper methods in the PagedIterables class, such as the advance one. If you take a look at that method you'll see that the PagedIterable is build given an IterableWithMarker<T> and an advancing function. That advancing function transforms an Object to an IterablewithMarker<T> where the Object is the current marker to the next page, and the result should be the next page. That advancing function will only be invoked if the marker to the next page is present.

So, in the end, you just need to create a function to transform an IterableWithMarker<T> into aPagediterable<T>. How you build it is up to you. Here are a few examples you can take a look at, to get the whole idea:

  • In Abiquo, for example, each page returns a link to the next one, so there is a generic function that simply performs a GET call to fetch the next page, given the marker (the link).
  • In Glance, there is the ParseImages#ToPagedIterable that extends an existing helper class to build a new API call. The class it extends populates the caller arguments to the function, but this might not be the best example in your case and could bring confusion.
  • There is a helper class that you could also use (and perhaps this is what I'd recommend if there is no generic way to do pagination for all API calls). You could create a function that extends the existingexisting ArgsToPagedIterable. That class already has the common logic and you only have to implement its abstract method. That method must return a function that returns the next page given anIterableWithMarker<T>, but also receives as a parameter the list of arguments (the arguments of the invoked java method) used in the previous API call, which can be useful to build the call to the next page. You can take the ParseImages#ToPagedIterable as an example to get the idea, but apply that to the ArgsToPagedIterable class.