This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Child pages
  • AB Testing of Search Engines

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 4.0

Intro...

TODO: intro......

Some Examples of Search Engine AB Testing

Outline for one possible AB testing tool:

  • Searches will be predefined
  • Run the same search across 2 or more engines
  • Merge the results, eliminate duplicates
  • Ask user to rank each result on a 1 to 4 scale

"Delta Testing": A variation of AB testing tool:

  • Searches will be predefined
  • Run the same search across 2 engines
  • Merge the results, remove duplicates
  • The only results presented to the user are those that only 1 engine brought back.
  • Ask user to rank each result on a 1 to 4 scale

The rationale here is that if both engines agree on a document, there's less benefit in having a human tester look at it.  Of course both engines could made a mistake and brought back an incorrect document, but spot checks should be used to detect that.

Web Site "Secret" AB testing...

  • rotation...