Introduction

Normalizers are used to normalize attribute's values before we store them into the backend, when they are parts of an indexed attributes, or when they are used in a DN.

The reason we normalize those values is that we need to compare the request value with the stored value when searching, adding, comparing or deleting them. If we do that using the user provided value, we may have duplicated values in the backend or we migth nof found some entries.

Normalizing a value will be done with respect to it's specific matching rule, as described in RFC 4517, paragraph 4.2 and following.

List of normalizers

relation between MatchingRules and Normalizers

The following table presents the list of all used matching rules in ADS :

Normalizers

Here is the table which presents the list of Normalizers we have to implement :

String 'preparation'

When we have to normalize a String in ADS, we have to prepare it, using the six following steps, has described in RFC 4518 :

1. Transcode
2. Map
3. Normalize
4. Prohibit
5. Check Bidi (Bidirectional)
6. Insignificant Character Handling
   This preparation is necessary to be able to compare assertion values with attribute values stored in the backend. Sadly, RFC 4518 has been written with Unicode in mind, and we are supposed to support the full set of Unicode chars. This is obviously very difficult, and will imply a lot of change in the server, as we are supporting 16 bit chars only. Normalizing for instance will be supported by Java 6, but is not supported in Java 5.

Transcoding

We won't have to transcode the values in ADS, as every string is already transcoded to UTF-8. The RFC specify that we should transcode all the PrintableStrings to Unicode strings, and as UTF-8 is a way to encode an Unicode String , this is ok.

Mapping

The mapping is the action of transforming some Unicode chars to another one. The following table show all the needed trasnformations :

We also have to lowercase all the uppercase chars. The following table gives all the transformation we must do, including those which produces more than one character.

The Prepare Map, Lowercasing page contains all the mapping to apply.

This mapping is implemented in the class PrepareString

 Normalizing

The normalization steps is the process to transform complex characters to many simples ones, like 'Schön' will be transfromed to 'Scho\u0308n', in order to ease comparizons. Sadly, Java 4 does not have the tools to do this normalization (Java 6 has it . The question is : why do we have to normalize ?

 Prohibiting

We have to avoid some chars. The prohibit method will check for each invalid character, and will throw an exception if one of them is met.

The prohibited characters are listed in tables A.1, C.3, C.4, C.5 and C.8 in RFC 3454. The character U-FFFD is also prohibited.

 Checking Bidi

Bidirectional characters are ignored, and thus eliminated from the String. Those bidirectionnal characters are listed in table D.1 and D.2 in RFC 3454 

 Insignifiant character handling

We have three different kind of Strings to handle :

• Numeric Strings
• Telephone Numbers
• Printable Strings with case ignore and exact string matching

Numeric Strings :

All the spaces will be removed. For instance, "12 34  56    7" is transformed to "1234567"

Telephone Number Strings :

 All spaces and hyphen are deleted : "(33)1   12  340-56-78" is transformed to "(33)112345678"

Printable Strings :

 This is much more complex. We must remove spaces at the beginning and at the end of the string, but we also have to substitute all the consecutive spaces to a single space into the String, except a space which is followed by a combining mark character.

 The following finite state machine is implemented to cover all those special cases :

Actions :

The state engine is implemented in this form :

1. remove all spaces at the beginning of the string, except if there is a combining mark after a space
2. then remove the spaces at the end of the string
3. if there are some non space characters, process them as if they were a list of chars and spaces : (char* space*)*, where spaces are replaced by a single space