Some ideas on how we might make versioned properties inheritable.
What Is Property Inheritance?
Property inheritance provides a mechanism to find the versioned properties set on the path-wise parents of a given path. Conversely it can be viewed as a mechanism by which a versioned property set on a given path also applies to that path's children.
What's Driving This?
Desire for some form of inherited properties has existed almost from the dawn of Subversion. This latest effort is in response to a recent proposal regarding Server Dictated Configuration (see http://svn.haxx.se/dev/archive-2012-01/0032.shtml). That proposal made use of a new mechanism by which server dictated configurations (notably auto-props and global-ignores) would be communicated from the server to the client. In the feedback on the proposal several people pointed out that versioned properties provide a possible alternative solution (and that the TortoiseSVN project was already using pseudo-inheritable properties to address some of the same problems – see http://tortoisesvn.net/docs/nightly/TortoiseSVN_en/tsvn-dug-propertypage.html). Despite its origins in the server defined configuration work, this wiki describes only a generic inheritable properties design (though my the ultimate goal is to use these inheritable properties to implement the server dictated configuration feature – pburba).
What This Design Is and What It Isn't
This design provides the bare minimum to support the basics of inherited properties:
- A way to get a path's inherited properties from the repository.
- A way to cache a working copy's inherited properties locally so that disconnected WC operations remain disconnected.
That's it, it's really just a small extension of our existing versioned property capabilities. Since any versioned property can be inherited, there is no concept of setting "inheritable" properties. Our existing APIs are sufficient. If your own personal vision of inherited properties includes something outside of these two bullets (e.g. merging inherited properties), then that is outside of this design's scope (at least initially).
How Is This Design Different?
It's not really. A lot of the ideas here come from the dev list, #svn-dev, and conversations with other devs. The difference, if there is any, is that this design aims to be as simple as possible and steer clear of complications which have thwarted inheritable properties in the past. If there is more than one reasonable behavior to choose from, it usually goes with the one that is simpler to explain/understand/implement/maintain. This means that not everyone will be happy, it may not cover every use-case, but I hope it covers most of them – pburba.
Inheritable Properties Today
Subversion has supported merge-tracking using the svn:mergeinfo property since the 1.5 release. The svn:mergeinfo property is inheritable in some merge-sensitive contexts, notably 'svn merge', 'svn log -g', and 'svn mergeinfo'. For example, say we have this simple branch structure at r700 in our repository:
And suppose the mergeinfo 'projX/trunk:509-612' is set on 'projX/branches'. If we perform a merge targeting 'projX/branches/src/foo.c', the merge logic considers the svn:mergeinfo property on this target path to effectively be 'projX/trunk/src/foo.c:509-612'. However, unlike a true inheritable property however, svn:mergeinfo is not inheritable outside of merge-tracking aware" contexts. For example, the propget and proplist subcommands recognize the svn:mergeinfo when it is explicitly set on a path, but does not "see" it on those paths' descendants:
Differentiating 'Inheritable' Vs. 'Normal' Properties
This is easy, there is no difference, there is no such thing as an "inheritable" property. Versioned properties, both Subversion's reserved properties and custom user properties, can be interpreted as inheritable, but otherwise function as they always have in terms of repository storage, setting properties, valid property names and values, etc.. What is proposed here is a new way of looking at the familiar versioned property. The only differentiation that is important as far as the design is this: Is a property value inherited or explicit? A path may have property 'PropName' explicitly set on it or property values may be inherited by a child path from some parent path(s) which also have 'PropName' explicitly set on them.
General Inheritance Rules
A path (we'll refer to this as the 'child' path from here on) inherits a given property from any of the child's path-wise ancestors (the 'parent' paths) on which the given property is explicitly set. This means that a given path can inherit multiple values for a given property from different parents. Further, the child path may also have the given property explicitly set on itself. This obviously differs from svn:mergeinfo, where inheritance only occurs when a child lacks explicit mergeinfo and only the nearest parent's mergeinfo is inherited 1 . The new APIs will provide a list of parent paths/URLs and the properties inherited from each. It will be up to the caller, user, script, etc., to decide what it wants to do with this information. The goal here is maximum flexibility. It will be possible to implement a mergeinfo-like inheritance scheme or to merge multiple values together, again, whatever the consumer wants to do.
If a child path which inherits a property is copied, the inherited properties are not copied with it. The new destination path inherits properties from its new parents. This means that depending on the copy destination the new path may or may not inherit the same property, and even if it does, it may inherit a different value. This puts the onus on users to set their inheritable properties as close to the root of the repository tree as makes sense for the property in question (while considering any authorization restrictions that might thwart inheritance). Have a property you want to apply to the whole repository? Set it on the root. Want it to apply only to a single project? Set it on the root of the project. Note that if a path has an inheritable property explicitly set on it, the property is copied just like any other versioned property.
 Generic inherited properties also differ from mergeinfo in that the property value a child inherits from a parent is not based on the path-difference between the parent and child. The exact property value on the parent is the value the child will inherit.
Repository Inheritance Rules
Inheritance of properties within the repository is pretty straightforward, for a given property 'PropName':
- A repository child_path@REV may inherit one or more values of the 'PropName' property from any of the child's parents (i.e. its path-wise ancestors @REV) which have the 'PropName' property explicitly set on them.
Working Copy Inheritance Rules
- Inheritance within the working copy is a bit more complicated. For a given property 'PropName' there are three basic cases of WC inheritance:
- If the peg and operative revisions
- If the peg and operative revisions are unspecified, then the working copy child path inherits properties from its parents in the actualInherited Properties Cache section). "Working copy roots" here are defined as:
- A directory whose parent directory is not a working copy.
- A directory or file which is switched relative to its immediate parent.
- The root of a directory external.
- A file external.
These default WC inheritance rules have some important implications:
- A path, in an unmodified working copy at a uniform revision N, inherits the same properties that the base of that path would inherit in the repository at revision N.
- A path in a modified working copy effectively inherits properties from the proto-revision that any modified WC represents (a set of changes that could potentially be committed as a new revision).
- Mixed-revision working copies are similar to #2 except that there is no proto-revision. So unlike svn:mergeinfo and like tsvn:auto-props, inheritance across mixed-revision boundaries in the working copy is allowed by default.
Leaving Julian's questions for now...
Inherited Properties Cache
A child path that inherits a property from its parent may not have ready access to that parent in the working copy (e.g. the root of the working copy is a subtree of the parent path). To ensure that traditionally disconnected operations (i.e. those that require no access to the repository, like 'svn add') remain disconnected, we will maintain a cache of properties inherited by the root of the working copy. Whenever a new working copy is checked out, any properties inherited by the root of the working copy will be cached in the working copy. If a subtree within a working copy is switched, a separate cache will be created for the root of that subtree. File externals will have their own cache as well. Whenever the WC is updated the cache(s) will be refreshed.
The cache will be stored in a new BLOB column in the NODES table called 'inherited_props'. If the node is the base node of a WC root then this column is populated with a serialized skel of the node's inherited properties. In all other cases it is NULL.
Directory externals are treated as self-contained working copies with their own inherited properties cache. File externals, despite not being their own WC, are also treated as WC roots and get their own cache. The end result: Property inheritance never crosses external boundaries in the WC.
The svn:mergeinfo property can be inherited by any child path the user has read access to, even if the user has no access to the parent path the svn:mergeinfo property is explicitly set on. Generic property inheritance is more restrictive:
- Properties can only be inherited from paths which the user has read access to.
- Inheritance from a grandparent can occur across an intermediate parent the user has no read access to, as long as the user can read the grandparent.
For example, given a repository structure like this:
And this authz:
If jrandom asks via the appropriate API what properties
^/Projects/ProjX/Docs/Manual inherits, he would see the property:value pairs from
^/Projects, but not
Yes, this means that an administrator could set up a restrictive authz configuration that could thwart what might be desirable inheritance, e.g. very few users have read access to the repository root, but properties are set on the root that are intended to have repository-wide application. The assumption is that if an administrator sets inheritable properties on the repository root intending that they apply to the entire repos for all users, then he will ensure that all users have read access to the root.
Merging Inherited Properties
This design purposefully avoids any concept of how inherited properties (possibly from multiple parents) might be merged together with explicit properties. In some cases if a path has an inheritable property set on it, then it might make sense to consider that property's value as the complete and full value for that path, end of story (e.g. svn:mergeinfo). On the other hand, it's easy to imagine use-cases where it might be useful to merge a path's explicit and inherited properties. However, both cases depend on the semantics of the particular inherited property. There is no way to develop a one-size-fits-all approach to merging inheritable properties. So while the API changes below support the ability to get a path's explicit and inherited properties, how to merge these values (if at all) will be handled on a case-by-case basis.
Since "inherited" properties are really just a new way of looking at versioned properties, most subcommands will only notice the paths on which explicit properties are set. For example, if we have an unmodified working copy and then make a local change to a parent path's explicit property:
- 'svn status' won't show any property mods on the parent's children paths.
- 'svn diff' will only show the property difference on the parent path.
The are a few exceptions to this business-as-usual behavior. A few subcommands require some non-user-visible changes:
- checkout (co): Populates the inherited properties cache for the WC root.
- switch (sw): Creates/clears the inherited properties cache for the root of the switched/unswitched subtree.
- update (up): Updates the inherited properties cache(s) for any WC roots within the operational depth of the update.
- upgrade: Creates an empty (but non-null) inherited properties caches for all WC roots, but doesn't populate any of them since that would require contacting the repository which upgrade shouldn't need to do. Subsequent updates will popluate the caches.
Two subcommands, propget (pget, pg) and proplist (plist, pl), will support a new option enabling users to view inherited properties on a path: '--show-inherited-props'. When used, any properties inherited by the target of the subcommand will be displayed. Properties inherited by subtrees of the target are not shown. For example: Given this repository structure with the explicit properties noted:
Without the --show-inherited-props option only explicit properties are shown (as has always been the case):
With the --show-inherited-props option, explicit properties are shown as normal, but any properties inherited by the target are also shown. For example, a recursive propget on ^/branch shows the explicit properties on the target and its children, but also shows that the target inherits prop:foo from the root of the repository:
A target with no explicit properties might still inherit a property from a parent:
If a path inherits a property from multiple parents, all the parents are shown:
If the target is a WC path and some of the inherited properties come from the repository, those from the repository are listed by URL rather than WC path:
The above examples apply only to the general case. Subversion reserved properties that we deem inheritable may exhibit differing behavior, depending on how we define inheritance merging for the property in question. For example, might show either the explicit mergeinfo on or the mergeinfo existing on nearest path-wise parent with explicit mergeinfo (or possible nothing if there was no explicit mergeinfo on the target or its parents).
We may want to support different values for --show-inherited-props, possibly:
--show-inherited-props=['all' (default) | 'nearest' ]
'all' - Shows the target's explicit properties and all path-wise parents with inheritable properties.
'nearest' - Shows the target's explicit properties and the nearest path-wise parent for any properties which are *not* explicitly set on the target. This would be useful for seeing inheritable properties which follow a strict override model like svn:mergeinfo.
Rev svn_proplist_receiver_t to svn_proplist_receiver2_t, adding the argument apr_array_header_t *inherited_props. Also allow the possibility that the existing apr_hash_t *prop_hash argument may be null:
[JAF] What do the keys of 'inherited_prop_hash' represent if 'path' is a working copy path but some of the inherited properties are set on repository paths that are above the WC root?
[PTB] JAF - It's a bit awkward, but I was thinking WC relative paths for props inherited from WC parents and URLs for props inherited from the repository (via the inherited prop cache). We obviously can't use WC keys in all cases, but using URLs in all cases won't work either because a WC parent might not exist in the repository due to local changes.
[PTB] Instead of using a hash mapping parent "paths" to a hash of properties, it probably makes more sense to use an array of structures which contain the path/URL and the property hash as members. The array would be sorted by the depth of the path/URL and would allow the caller to easily determine the nearest parent of PATH if that is all that it needs (think of a property with a svn:mergeinfo-like straight override model of inheritance where all that ever matters is the nearest parent). Done - change noted above.
Rev svn_client_proplist3 to svn_client_proplist4, adding the argument svn_boolean_t get_target_inherited_props as well as a scratch pool and use the new svn_proplist_receiver2_t callback:
Rev svn_client_propget4 to svn_client_propget5, adding the argument apr_array_header_t **inherited_props:
Rev svn_ra_get_dir2 to svn_ra_get_dir3, adding the argument apr_array_header_t **inherited_props.* * Also allow the possibility that the existing apr_hash_t **props argument may be null:
Rev svn_ra_get_file to svn_ra_get_file2, adding the argument apr_array_header_t **inherited_props. Also allow the possibility that the existing apr_hash_t **props argument may be null:
Rev svn_fs_node_proplist to svn_fs_node_proplist2, adding the argument apr_hash_t **inherited_table_p. Also allow the possibility that the existing apr_hash_t **table_p argument may be null:
New APIs and Public Structs
The suggested design above has some known drawbacks and trade-offs:
- [JAF] I note that there is no ability to specify that an inheritable property should be deleted rather than overridden with a new value. Thus we can't use absent/present semantics similar to 'svn:needs-lock'. If we set an inheritable equivalent of 'svn:needs-lock = *' on a parent dir, there is no way to designate a particular child node as explicitly removing that inherited property. That's probably fine – we can just avoid using the absent/present semantics for that kind of purpose – but it deserves to be mentioned.
- Generic inherited properties also differ from mergeinfo in that the property value a child inherits from a parent is not based on the path-difference between the parent and child. The exact property value on the parent is the value the child will inherit. ↩
- See args to svn_client_proplist4 and svn_client_propget5 in the new APIs section. These are the primary APIs for getting inherited props from the WC. ↩
- svn_opt_revision_unspecified or svn_opt_revision_working. ↩
- The working tree plus property and text changes. ↩