This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Page tree
Skip to end of metadata
Go to start of metadata

JMeter High-Level Requirements

"must" and "should" are used in the following sense:

  • Failure to comply to a "must" requirement should be considered a Normal, Major, Critical, or Blocker bug.
  • Failure to comply to a "should" requirement should be considered as a Minor or Normal bug.

Functionality requirements:

  • Must be able to exercise the functionality of any HTTP/HTTPS site which respects common conventions (e.g. that GET requests don't have a body, that query-string parameters are URL-encoded and separated by & and =,...).[This includes using different test data for each thread or for each test loop, reusing data extracted from previous responses, etc...]

  • Should be able to exercise the functionality of any HTTP/HTTPS site.
  • When generating HTTP/HTTPS load, should simulate the behaviour of real browsers at the HTTP protocol level.
  • Shouldn't break on illegal but common stuff (e.g. "Location:" headers without a protocol://server in the URL, fake SSL certificates,...).
  • Should show test results in both numerical and graphical form. The graphs and reports should be nice-looking enough to show them to a customer. [Yes, I know it's pretty weak as a requirement, but that's the meaning of "high-level" after all, isn't it? ]

Portability:

  • Must be 100% Java.
  • Should run on all JDKs from 1.4 to current.
  • Must run on win32 and linux.
  • Should run on all platforms (assuming enough resources available).

Usability:

  • Ease of installation: should be able to set up a single load-generation workstation on any site in a few hours – even in the presence of ugly firewall settings. [This probably requires a command-line interface.]

  • Ease of use: any user with enough knowledge to install JMeter and configure his own web browser should be able to create and run simple test scripts to test HTTP sites in minutes – assuming no ugly firewall settings – just by following the manuals.
  • Should be reasonably easy to set up and use a cluster of load-generation workstations, even in the presence of ugly firewall settings. [This is probably one of the very weak aspects of JMeter right now.]

Performance:

  • Should be able to drive an application server at 100% with one single similarly-powered load-generation machine – assuming an average application is running on the server which doesn't require specially complex processing of responses. [This one is pretty weak too... ideas for making it better?]

Maintainability:

  • [JavaDoc] must be correct. [Yes, this means a plain lie in a JavaDoc comment can't be considered a Minor bug. Also means we should always be careful to keep JavaDoc up-to-date when we touch at the code.]

  • JavaDoc should be complete.
  • No labels