...
The idea is simple yet extremely powerful. Enable an LDAP server to version itself by tracking changes in a change log. Allow tagging of specific revisions to provide a mechanism for taking snapshots of the server state to potentially be reverted. Thanks to the design of ApacheDS this is something that is not incredibly difficult to implement.
Use Cases and Potential Value
There are several ways in which this feature can be leveraged by our users and us while developing ApacheDS:
Wiki Markup - Could be used for a transaction journal
- Record a series of changes and play them back.: great for testing complex operation sequences
- Revert test servers after using them without a costly reinstall and restart
- To audit changes to answer simple questions:
- who made the change
- what was changed (entries, attributes, values)
- when was the change made
- how was the change made (add, delete, modify, moddn, modrdn)
- To inquire about the change history at several levels
- entire server
- by a user
- on a region of the tree
- by dates
- by change types
- per entry
- per attribute
- Quickly assess change patters in the DIT
- Tagging revisions for specific states
- Rolling back/forward server state
- Requesting attributes and entries by revision
- Searching the server for entries by revision
These usage scenarios can occur when managing several things however nothing best characterizes the need than managing environment configurations within an infrastructure. There are situations where the configuration of several LDAP enabled applications may need to be rolled back to an earlier state or inquiries need to be made about who changed what, when for the sake of providing thorough reports to auditors.
| Note | ||
|---|---|---|
| ||
Some people may say why ask: "Why do I need this versioning and snapshoting feature if I can get LDAP transactions . can be used?" Well transactions are great but they don't stop mistakes, they just make sure they occur transactionally. Users can still delete or change things in the server incorrectly after committing a transaction. Upon realizing the error the transaction cannot be rolled back if it has already been committed. So no transactions don't save your arse you in this case. Replication also does not save you. If you make a mistake when making a change that change will replicate throughout the cluster so you still need versioning a way to rollback to older revisions. |
History
A couple years ago I came across the idea of versioning changes inside ApacheDS. At first the idea seemed amazing yet insurmountable to implement with pre-1.0 versions of the server. The best we could do at the time was to test and demo the idea with a simple change log interceptor example in a presentation: Embedding ApacheDS. Ersin Er was interested in getting involved with writing interceptors at the time so he and I discussed the feature. He took a crack at implementing the log as a text based LDIF log file and did a great job. We can still use much of this code today for a simple logger.
...
Protocol Considerations
As far as LDAP, the protocol, is concerned we have some advantages and can have good synergy with it.
Operational Attributes: revisions
...
Taking snapshots are easy and can be done via extended operations as well but what about reverting the server back to it's original state. Tools might be able to do this if they can access the changelog (where would it be?) especially with reverse LDIFs to revert changes. However issues arise like managing operational attributes such as modifiersName etc. When applying a change log the tool cannot easily and securely apply it with the credentials of each user who made the modification to guarantee that the servers operatioinal attribute house keeping does it's thing. You could get around this if the server supported the LDAP Proxied Authorization Control. Presuming you had the tools, access to the logs, and the proxied authorization control available the tool still could fail during the course of applying the changes and the server could go down to leave the server's state inconsistent with respect to your application data. Oh ok so So if transactions actually make it into the specifications we could solve this problem too with tooling.
...
This can easily be implemented in ApacheDS as a custom partition exposed by the ChangeLog service to view the log information. Of course implementation and backing stores will dictate the quality of the search experience but this is up to implementations to remedy.
Design Proposal
Every change in an LDAP server can be logged and tracked using an LDIF. A special log of LDIF entries can track each new revision of the server. This can be used to audit changes on the DIT, on subtrees of the DIT, their entries and even on attributes. Each entry can track revisions in the change log using
More interesting however is the ability
We don't have any way to define snapshots in the server. This is a very interesting functionality we may want to have. XWhat will it be good for ?
- being able to log all the requests for debug purpose or for post-mortem analysis
- create a journal we can replay from a specific point if the server has crashed and we want to restore the backend to a valid state
- being able to rollback the backend up to a previous state (like SVN/CVS does)
- use the journal to update other instances remotly and asynchronously
- many more usages I don't have on top of my head right now
To be able to implement all those functionalities, we have to implement some basic bricks :
- a new interceptor to store all the requests
- some extended operations to set a Tag, to rollback or to discard a tag
- define the exact semantic of each new functionalities
Simple Logger
The first very basic functionality is to implement a basic logger. It has already been added, but some more elements need to be defined, like the way to start the logger, either via an LDAP extended request, or programmatically.
There is an existing page started by Ersin where you have a description of the existing ChangeLog interceptor :
Logging Subsystem
Semantic
We need to be able to start the logger, to stop it, to configure it, to select the requests, to select the users, etc. Here are a list of parameters we may want to set :
...
Parameter
...
Description
...
Mandatory
...
Logger name
...
The logger's name
...
...
Active
...
Activate or desactivate the log. Default to TRUE, and when set to FALSE, no other parameters should be given
...
Max Size
...
The maximum size of the log file
...
...
Saved Requests
...
The list of requests we want to save (AddRequest, DelRequest, for instance)
Default to ALL_REQUESTS
...
Saved Attribute
...
A list of attributes we want to store in the file. It may be associated with the
previous selection. '*' and '+' will be used for 'all users attribute' and 'all
operationnal attributes'
Default to '*'
...
Subentry
...
Define the subentry we want the logger to apply when logging. The user may want
to allow some users to log or not, or may define a set of entries which may be logged, etc.
Default to null : everything is loggable by default
...
Rotate
...
Define the way the file will be rotated : Daily, when exceeding a given size, after a certain period of time
You might want to allow updates to this partition to only create snapshot tags. This way you don't need an extended operation to take a snapshot.
Design
Since this takes up a lot of space we're do this on a child page here: ChangeLog Design
...
Active
...
Activate or desactivate the log. Default to TRUE
...
When we have defined those parameters, we will have to describe them using ASN.1, and to implement the ExtendedOperation.