Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 4.0

This page describes the workload portion of Olio i.e. the load that is generated by the Olio driver against the Olio application.

Definitions

  • Concurrent Users: The number of users emulated during a test run of the workload. The workload is scaled by increasing the number of Concurrent Users. We also use the term Active Users to represent the set of users actually performing work against the application.
  • Users : The term Users is usually a short-hand for Concurrent Users or Active Users.
  • Registered Users: This is the number of users known to the application i.e. the number of users in the database. The number of registered users is 100 times the number of Concurrent Users and reflects the fact that social networking applications have a large number of users but typically only a fraction of them are actively using the application.
  • Operations: An operation is a request for a particular page made by the emulated user. Each operation may involve multiple HTTP request/response cycles depending on the embedded objects on a page. On average, there are 20 HTTP requests made per operation.
  • Response Time: This is the time taken between the first byte of the first http request for an operation to the time the last byte of the last http request has been received. As such, this is a measure of the server response time and does not include any client-side processing.
  • Cycle Time: This is the inter-arrival time between subsequent operations made by a concurrent user. To more realistically represent real workloads, each emulated user will wait for some time before submitting the next operation. In Olio, the mean cycle time is 5 seconds. This allows us to drive a larger emulated user load although to be truly realistic, the cycle time should be upto 30 seconds or larger. Using larger cycle times will require emulating 100's of thousands (or even millions) of users, so to provide a workload that is easier to run, we use 5 seconds.
  • Operations Mix: This is the ratio of the various operations submitted by concurrent users. Olio uses a state-transition matrix mix to define the probability of transitioning from one operation to the next. See the Faban documentation for more details.

Olio Operations

There are 7 different Operations defined by Olio. A brief description of each is given below.

...

The AddPerson and AddEvent operations are relatively expensive in terms of cpu time consumed, since they involve image thumbnail generation.

Requirements and Metrics

Throughput

The primary performance metric is a throughput metric of operations/second representing the total number of operations performed during the steady-state interval divided by the number of seconds in the steady-state interval. The metric is valid only if all of the requirements for a run have passed as indicated by the summary report.

Response Times

There are requirements for 90% response time (the time in which 90% of all operations in staeady state should have completed) for all of the operations as specified below :

Operation

90% Response Time

HomePage

1 second

Login

1 second

TagSearch

2 seconds

EventDetail

2 seconds

PersonDetail

2 seconds

AddPerson

3 seconds

AddEvent

4 seconds

Operation Mix

A valid run needs to meet the following operation mix requirements. Note that the driver is designed to meet these requirements as long as there are a reasonable number of operations that were executed during the run.Note that since the AddPerson is a very small percentage of the mix, you will need to ensure that a sufficient number of operations are executed to meet this requirement. In practice, we have had no problem in achieving passing runs even with low concurrent users (50 or 100 users) as long as the steady state period is at 600-900 seconds.

Operation

%Total Mix

HomePage

26.15%

Login

10.22%

TagSearch

33.45%

EventDetail

24.68%

PersonDetail

2.61%

AddPerson

0.84%

AddEvent

2.84%

Cycle Times

Olio uses a negative exponential distribution of cycle times (see Cycle timings section in Faban documentation) to simulate a realistic web workload. The distribution uses a mean of 5 seconds, starting at 0 seconds and is truncated at 25 seconds for all of the operations except the AddPerson and AddEvent operations.

...