Subversion uses three kinds of deltas: A tree delta describes the difference between two arbitrary directory trees, the way a traditional patch describes the difference between two files. For example, the delta between directories A and B could be applied to A, to produce B. Tree deltas can also carry ancestry information, indicating how the files in one tree are related to files in the other tree. And deltas can describe changes to file meta-information, like permission bits, creation dates, and so on. The repository and working copy use deltas to communicate changes. A text delta describes changes to a string of bytes, such as the contents of a file. It is analogous to traditional patch format, except that it works equally well on binary and text files, and is not invertible (because context and deleted data are not recorded). A property delta describes changes to a list of named properties (see ). The term delta without qualification generally means a tree delta, unless some other meaning is clear from context. In the examples below, deltas will be described in XML, which happens to be Subversion's (now mostly defunct) import/export patch format. However, note that deltas are an abstract data structure, of which the XML format is merely one representation. Later, we will describe other representations: for example, there is a serialized representation (useful for streaming protocols, among other things), and a db-style representation, used for repository storage. The various representations of a given delta are (in theory, anyway) perfectly isomorphic to one another, since they describe the same underlying structure. A text delta describes the difference between two strings of bytes, the source string and the target string. Given a source string and a target string, we can compute a text delta; given a source string and a delta, we can reconstruct the target string. However, note that deltas are not invertible: you cannot always reconstruct the source string given the target string and delta. The standard Unix “diff” format is one possible representation for text deltas; however, diffs are not ideal for internal use by a revision control system, for several reasons: Diffs are line-oriented, which makes them human-readable, but sometimes makes them perform poorly on binary files. Diffs represent a series of replacements, exchanging selected ranges ofthe old text with new text; again, this is easy for humans to read, butit is more expensive to compute and less compact than some alternatives. Instead, Subversion uses the VDelta binary-diffing algorithm, as described in Hunt, J. J., Vo, K.-P., and Tichy, W. F. An empirical study of delta algorithms. Lecture Notes in Computer Science 1167 (July 1996), 49-66. Currently, the output of this algorithm is stored in a custom data format called svndiff, invented by Greg Hudson <>, a Subversion developer. The concrete form of a text delta is a well-formed XML element, having the following form: <text-delta>data</text-delta> Here, data is the raw svndiff data, encoded in the MIME Base64 format. A property delta describes changes to a property list, of the sort associated with files, directories, and directory entries, and revision numbers (see ). A property delta can record creating, deleting, and changing the text of any number of properties. A property delta is an unordered set of name/change pairs. No two pairs within a given property delta have the same name. A pair's name indicates the property affected, and the change indicates what happens to its value. There are two kinds of changes: set value Change the value of the named property to the byte string value. If there is no property with the given name, one is added to the property list. delete Remove the named property from the property list. At the moment, the set command can either create or change a property value. However, this simplification means that the server cannot distinguish between a client which believes it is creating a value afresh, and a client which believes it is changing the value of an existing property. It may simplify conflict detection to divide set into two separate add and change operations. In the future, we may add a text-delta change, which specifies a change to an existing property's value as a text delta. This would give us a compact way to describe small changes to large property values. The concrete form of a property delta is a well-formed XML element, having the following form: <property-delta>change…</property-delta> Each change in a property delta has one of the following forms: <set name='name'>value</set> <delete name='name'/> The name attribute of a set or delete element gives the name of the property to change. The value of a set element gives the new value of the property. If either the property name or the property value contains the characters ‘&’, ‘<’, or ‘'’, they should be replaced with the sequences ‘&#38’, ‘&#60’, or ‘&#39’, respectively. A tree delta describes changes between two directory trees, the source tree and the target tree. Tree deltas can describe copies, renames, and deletions of files and directories, changes to file contents, and changes to property lists. A tree delta can also carry information about how the files in the target tree are derived from the files in the source tree, if this information is available. The format for tree deltas described here is easy to compute from a Subversion working directory, and easy to apply to a Subversion repository. Furthermore, the size of a tree delta in this format is independent of the commands used to produce the target tree — it depends only on the degree of difference between the source and target trees. A tree delta is interpreted in the context of three parameters: source-root, the name of the directory to which this completetree delta applies, revision, indicating a particular revision of … source-dir, which is a directory in the source tree that we arecurrently modifying to yield … … target-dir — the directory we're constructing. When we start interpreting a tree delta, source-root, source-dir, and target-dir are all equal. As we walk the tree delta, target-dir walks the tree we are constructing, and source-dir walks the corresponding portion of the source tree, which we use as the original. Source-root remains constant as we walk the delta; we may use it to choose new source trees. A tree delta is a list of changes of the form <tree-delta>change…</tree-delta> which describe how to edit the contents of source-dir to yield target-dir. There are three kinds of changes: <delete name='name'/> Source-dir has an entry named name, which is not present in target-dir. <add name='name'>content</add> target-dir has an entry named name, which is not present in source-dir; content describes the file or directory to which the new directory entry refers. <open name='name'>content</open> Both source-dir and target-dir have an entry named name, which has changed; content describes the new file or directory. Any entries in source-dir whose names aren't mentioned are assumed to appear unchanged in target-dir. Thus, an empty tree-delta element indicates that target-dir is identical to source-dir. In the change descriptions above, each content takes one of the following forms: <file ancestor>prop-delta text-delta</file> The given target-dir entry refers to a file, f. Ancestor indicates which file in the source tree f is derived from, if any. Prop-delta is a property delta describing how f's properties differ from that ancestor; it may be omitted, indicating that the properties are unchanged. Text-delta is a text delta describing how to construct f from that ancestor; it may also be omitted, indicating that f's text is identical to its ancestor's. <file ancestor/> An abbreviation for <file ancestor></file> — a fileelement with no property or text delta, thus describing a file identicalto its ancestor. <directory ancestor>prop-delta tree-delta</directory> The given target-dir entry refers to a subdirectory, sub. Ancestor indicates which directory in the source tree sub is derived from, if any. Prop-delta is a property delta describing how sub'sproperties differ from that ancestor; it may be omitted, indicating thatthe properties are unchanged. Tree-delta describes how to construct sub from that ancestor; it may be omitted, indicating that the directory is identical to its ancestor. Tree-delta should be interpreted with a new target-dir of target-dir/name. Since tree-delta is itself a complete tree delta structure, tree deltas are themselves trees, whose structure is a subgraph of the target tree. <directory ancestor/> An abbreviation for <directory ancestor></directory> — a directory element with no property or tree delta, thus describing a directory identical to its ancestor. The content of a add or open tag may also contain a property delta, describing changes to the properties of that directory entry. In the file and directory elements described above, each ancestor has one of the following forms: ancestor='path' The ancestor of the new or changed file or directory is source-root/path, in revision. When this appears as an attribute of a file element, the element's text delta should be applied to source-root/path. When this appears as an attribute of a directory element, source-root/path should be the new source-dir for interpreting that element's tree delta. new='true' This indicates that the file or directory has no ancestor in the source tree. When followed by a text-delta, that delta should be applied to the empty file to yield the new text; when followed by a tree-delta, that delta should be evaluated as if source-dir were an imaginary empty directory. nothing If neither an ancestor nor a new attribute is given, this is an abbreviation for ancestor='source-dir/name', with the same revision number. This makes the common case — files or directories modified in place — more compact. If the ancestor spec is not new='true', it may also contain the text revision='rev', indicating a new value for revision, in which we should find the ancestor. If a filename or path appearing as a name or path in the description above contains the characters ‘&’, ‘<’, or ‘'’, they should be replaced with the sequences ‘&#38;’, ‘&#60;’, or ‘&#39;’, respectively. Suppose we have the following source tree: /dir1/file1 file2 dir2/file3 file4 dir3/file5 file6 If we edit the contents of /dir1/file1, we can describe the effect on the tree with the following tree delta, to be applied to the root: <tree-delta> <open name='dir1'> <directory> <tree-delta> <open name='file1'> <file>text-delta</file> </open> </tree-delta> </directory> </open> </tree-delta> The outer tree-delta element describes the changes made to the root directory. Within the root directory, there are changes in dir1, described by the nested tree-delta. Within /dir1, there are changes in file1, described by the text-delta. If we had edited both /dir1/file1 and /dir1/file2, then there would simply be two open elements in the inner tree-delta. As another example, starting from the same source tree, suppose we rename /dir1/file1 to /dir1/file8: <tree-delta> <open name='dir1'> <directory> <tree-delta> <delete name='file1'/> <add name='file8'> <file ancestor='/dir1/file1'/> </add> </tree-delta> </directory> </open> </tree-delta> As above, the inner tdelta describes how /dir1 has changed: the entry for /dir1/file1 has disappeared, but there is a new entry, /dir1/file8, which is derived from and textually identical to /dir1/file1 in the source directory. This is just an indirect way of describing the rename. Why is it necessary to be so indirect? Consider the delta representing the result of: renaming /dir1/file1 to /dir1/tmp, renaming /dir1/file2 to /dir1/file1, and renaming /dir1/tmp to /dir1/file2 (in other words, exchanging file1 and file2): <tree-delta> <open name='dir1'> <directory> <tree-delta> <open name='file1'> <file ancestor='/dir1/file2'/> </open> <open name='file2'> <file ancestor='/dir1/file1'/> </open> </tree-delta> </directory> </open> </tree-delta> The indirectness allows the tree delta to capture an arbitrary rearrangement without resorting to temporary filenames. Another example, starting from the same source tree: rename /dir1/dir2 to /dir1/dir4, rename /dir1/dir3 to /dir1/dir2, and move file3 from /dir1/dir4 to /dir1/dir2. Note that file3's path has remained the same, even though the directories around it have changed. Here is the tree delta: <tree-delta> <open name='dir1'> <directory> <tree-delta> <open name='dir2'> <directory ancestor='/dir1/dir3'> <tree-delta> <add name='file3'> <file ancestor='/dir1/dir2/file3'/> </add> </tree-delta> </directory> </open> <delete name='dir3'/> <add name='dir4'> <directory ancestor='/dir1/dir2'> <tree-delta> <delete name='file3'/> </tree-delta> </directory> </add> </tree-delta> </directory> </open> </tree-delta> In other words: /dir1 has changed; the new directory /dir1/dir2 is derived from the old /dir1/dir3, and contains a new entry file3, derived from the old /dir1/dir2/file3; there is no longer any /dir1/dir3; and the new directory /dir1/dir4 is derived from the old /dir1/dir2, except that its entry for file3 is now gone. Some more possible maneuvers, left as exercises for the reader: Delete dir2, and then create a file named dir2. Rename /dir1/dir2 to /dir1/dir4; move file2 into /dir1/dir4; and move file3 into /dir1/dir3. Move dir2 into dir3, and move dir3 into /. It is sometimes useful to represent a set of changes to a tree without providing text deltas in the middle of the stream. Text deltas are often large and expensive to compute, and tree deltas can be useful without them. For example, one can detect whether two changes might conflict — whether they change the same file, for example — without knowing exactly how the conflicting files changed. For this reason, our XML representation of a tree delta allows the text deltas to come after the </tree-delta> closure. This allows the client to receive early notice of conflicts: during a svn commit command, the client sends a tree-delta to the server, which can check for skeletal conflicts and reject the commit, before the client takes the time to transmit the (possibly large) textual changes. This potentially saves quite a bit of network traffic. In terms of XML, postfix text deltas are split into two parts. The first part appears "in-line" and contains a reference ID. The second part appears after the tree delta is complete. Here's an example: <tree-delta> <open name="foo.c"> <file> <text-delta-ref id="123"> </file> </open> <add name="bar.c"> <file> <text-delta-ref id="456"> </file> </add> </tree-delta> <text-delta id="123">data</text-delta> <text-delta id="456">data</text-delta> The static XML forms above are useful as an import/export format, and as a visualization aid, but we also need a way to express a delta as a series of operations, to implement directory tree diffing and patching. Subversion defines a standard set of such operations in the vtable svn_delta_edit_fns_t, a set of function prototypes which anyone may implement (see svn_delta.h). Each function in an instance of svn_delta_editor_t (colloquially known as an editor) implements some distinct subtask of editing a directory tree. In fact, if you compare the editor function prototypes to the XML elements described previously, you'll notice a fairly strict correspondence: there's one function for replacing a directory, another function for replacing a file, one for adding a directory, another for adding a file, a function for deleting, and so on. Although the editor interface was designed around the general idea of making changes to a directory tree, a specific implementation's behavior depends on its role. For example, the versioning filesystem library offers an editor that creates new revisions, while the working copy library offers an editor that updates working copies. And the network layer offers an editor that turns editing calls into wire protocol, which is then converted back into editing calls on the other side! All of these different tasks can share a single interface, because they are all fundamentally about the same thing: expressing and applying differences between directory trees. Like the XML forms, a series of editor calls must follow certain nesting conventions; these conventions are implicit in the interface, in that some of the functions take arguments that can only be obtained from previous calls to other editor functions. Editors can best be understood by watching one work on a real directory tree. For example: Suppose that the user has made a number of local changes to her working copy and wants to commit them to the repository. Let's represent her changes with the same tree-delta from a previous example. Notice that she has also made textual modifications to file3; hence the in-line <text-delta>: <tree-delta> <open name='dir1'> <directory> <tree-delta> <open name='dir2'> <directory ancestor='/dir1/dir3'> <tree-delta> <add name='file3'> <file ancestor='/dir1/dir2/file3'> <text-delta>data</text-delta> </file> </add> </tree-delta> </directory> </open> <delete name='dir3'/> <add name='dir4'> <directory ancestor='/dir1/dir2'> <tree-delta> <delete name='file3'/> </tree-delta> </directory> </add> </tree-delta> </directory> </open> </tree-delta> So how does the client send this information to the server? In a nutshell: the tree-delta is streamed over the network, as a series of individual commands given in depth-first order. Let's be more specific. The server presents the client with an object of type struct svn_delta_edit_fns_t, colloquially known as an editor. An editor is really just table of functions; each function makes a change to a filesystem. Agent A (who has a private filesystem) presents an editor to agent B. Agent B then calls the editor's functions to change A's filesystem. B is said to be driving the editor. As Karl Fogel likes to describe the process, if one thinks of the tree-delta as a lion, the editor is a "hoop" that the lion jumps through – each portion of the lion being decomposed through time. B cannot call the functions in any willy-nilly order; there are some logical restrictions. In particular, as B drives the editor, it receives opaque data structures which represent directories and files. It must use and pass these structures, known as batons, to make further function calls. As an example, let's watch how the client would transmit the above tree-delta to the repository. (The description below is slightly simplified. For exact interface details, see subversion/include/svn_delta.h.) [Note: in the examples below, and throughout Subversion's code base, you'll see references to 'baton' objects. This is simply a project convention, a name given to structures that define contexts for functions. Many APIs call these structures 'userdata'. In Subversion, we like the term 'baton', because it reminds us of one function “handing off” context to another function.] The repository hands an "editor" to the client. The client begins by calling root_baton = editor->open_root(); The client now has an opaque object, root_baton, which represents the root of the repository's filesystem. dir1_baton = editor->open_dir("dir1", root_baton); Notice that root_baton gives the client free license to make any changes it wants in the repository's root directory – until, of course, it calls editor->close_dir(root_baton). The first change made was a replacement of dir1. In return, the client now has a new opaque data structure that can be used to change dir1. dir2_baton = editor->open_dir("dir2", "/dir1/dir3", dir1_baton); The dir1_baton is now used to open dir2 with adirectory whose ancestor is /dir1/dir3. file_baton = editor->add_file("file3", "/dir1/dir2/file3", dir2_baton); Edits are now made to dir2 (using dir2_baton). In particular, a new file is added to this directory whose ancestor is /dir1/dir2/file3. Now the text-delta associated with file_baton needs to be transmitted: window_handler = editor->apply_textdelta(file_baton); Text-deltas themselves, for network efficiency, are streamed in "chunks". So instead of receiving a baton object, we now have a routine that is able to receive any number of small "windows" of text-delta data.We won't go into the details of the svn_txdelta_* functions right here; but suffice it to say that these routines are used for sending svndiff data to the window_handler routine. editor->close_file(file_baton); The client is done sending the file's text-delta, so it releases the file baton. editor->close_dir(dir2_baton)); The client is done making changes to dir2, so it releases its baton as well. The client isn't yet finished with dir1, however; it makes two more edits: editor->delete_item("dir3", dir1_baton); dir4_baton = editor->add_dir("dir4", "/dir1/dir2", dir1_baton);(The function's name is delete_item rather than delete to avoid gratuitous incompatibility with C++, where delete is a reserved keyword.) Within the directory dir4 (whose ancestry is /dir1/dir2), the client removes a file: editor->delete_item("file3", dir4_baton); The client is now finished with both dir4, as well as its parent dir1: editor->close_dir(dir4_baton); editor->close_dir(dir1_baton); The entire tree-delta is complete. The repository knows this when the root directory is closed: editor->close_dir(root_baton); Of course, at any point above, the repository may reject an edit. If this is the case, the client aborts the transmission and the repository hasn't changed a bit. (Thank goodness for transactions!) Note, however, that this "editor interface" works in the other direction as well. When the repository wishes to update a client's working copy, it is the client's reponsibility to give a custom editor-object to the server, and the server is the editor-driver. Here are the main advantages of this interface: Consistency. Tree-deltas move across the network, in both directions, using the same interface. Flexibility. Custom editor-implementations can be written to do anything one might want; the editor-driver has no idea what is happening on the other side of the interface. For example, an editor might Output XML that matches the tree-delta DTD above; Output human-readable descriptions of the edits taking place; Modify a filesystem Whatever the case, it's easy to "swap" editors around, and make client and server do new and interesting things.