...
- each user gets their own sandbox as discussed on RulesProjSandboxes
- checked-in rules in the sandboxes are mass-checked in the nightly mass-checks
- to migrate a rule from "sandbox" (dev) to "core" (production) ruleset uses C-T-R; ie. votes are not required in advance
- also C-T-R to migrate from "sandbox" to "extra" ruleset
Rules that get promoted from a "sandbox" to "core" should pass the following criteria:
- pass "--lint"!
- S/O ratio of 0.95 or greater (or 0.05 or less for nice rules)
- > 0.25% of target type hit (e.g. spam for non-nice rules)
- < 1.00% of non-target type hit (e.g. ham for non-nice rules)
These numbers are really just ball-park figures and should be fine-tuned as we go. (DuncanFindlay) We can automate those criteria pretty easily. We can also vote for rules that don't pass those criteria, but we think should be put into core for some reason.
...
- not too slow
TODO: need an automated way to measure that, probably derived from the timing plugin in bug 4517. - TODO: criteria for overlap with existing rules?
- TODO: Scores. How are we going to score stuff that's pushed out with sa-update? (I can't see any documentation on it at the moment) Is there going to be logs kept and perceptron runs or simply votes for interim scores based on the mails they target.
"Promotion" is a manual process, where the rule developer copies the lines for that rule from one file to a file in the 'core' directory. The promotable rules will be highlighted in some way in the rule-QA app.
...
Rules will be autorenamed, if there's a collision between a new rule name and one that's already been output by the compiler.