Intro...
TODO: intro......
Some Examples of Search Engine AB Testing
Outline for one possible AB testing tool:
- Searches will be predefined
- Run the same search across 2 or more engines
- Merge the results, eliminate duplicates
- Ask user to rank each result on a 1 to 4 scale
"Delta Testing": A variation of AB testing tool:
- Searches will be predefined
- Run the same search across 2 engines
- Merge the results, remove duplicates
- The only results presented to the user are those that only 1 engine brought back.
- Ask user to rank each result on a 1 to 4 scale
The rationale here is that if both engines agree on a document, there's less benefit in having a human tester look at it. Of course both engines could made a mistake and brought back an incorrect document, but spot checks should be used to detect that.
Web Site "Secret" AB testing...
- rotation...