Committer's Guide
Prev |  | Next

# 4 Subversion Primer

## 4.1 Introduction

The FreeBSD source repository switched from CVS to Subversion on May 31st, 2008. The first real SVN commit is r179447.

There are mechanisms in place to automatically merge changes back from the Subversion repository to the CVS one, so regular users should not notice a difference, however developers most certainly will.

Subversion is not that different from CVS when it comes to daily use, but there are differences. Subversion has a number of features that should make developers' lives easier. The most important advantage to Subversion (and the reason why FreeBSD switched) is that it handles branches and merging much better than CVS does. Some of the principal differences are:

- Commits are atomic.

- Revision numbers apply across the repository--all files that were modified in the same commit have the same revision number.

- Branching and tagging are namespace operations.

- Directories are versioned.

- Files and directories can have arbitrary, versioned metadata attached to them.

- Files and directories can be copied, with full history tracking.

- No more contortions due to CVS weakness such as applying patch(1) files at compile time in order to avoid touching vendor branch code.

- No more repo-copies.

Subversion can be installed from the FreeBSD Ports Collection, by issuing the following commands:

    # cd /usr/ports/devel/subversion
    # make clean install

## 4.2 Getting Started

There are three ways to obtain a working copy of the tree from Subversion. This section will explain them.

### 4.2.1 Direct Checkout

The first is to check out directly from the main repository:

    % svn checkout svn+ssh://svn.freebsd.org/base/head /usr/src

The above command will check out a `CURRENT` source tree as ``/usr/src/``, which can be any target directory on the local filesystem. Omitting the final argument of that command causes the working copy, in this case, to be named head, but that can be renamed safely.

`svn+ssh` means the SVN protocol tunnelled over SSH. The name of the server is `svn.freebsd.org`, `base` is the path to the repository, and `head` is the subdirectory within the repository.

If your FreeBSD login name is different from your login name on your local machine, you must either include it in the URL (for example `svn+ssh://jarjar@svn.freebsd.org/base/head`), or add an entry to your `~/.ssh/config` in the form:

    Host svn.freebsd.org
    	User jarjar

This is the simplest method, but it's hard to tell just yet how much load it will place on the repository. Subversion is much faster than CVS, however.

> Note: The `svn diff` does not require access to the server as SVN stores a reference copy of every file in the working copy. This, however, means that Subversion working copies are very large in size.

### 4.2.2 Checkout from a Mirror

You can check out a working copy from a mirror by simply substituting the mirror's URL for `svn+ssh://svn.freebsd.org/base`. This can be an official mirror or a mirror you maintain yourself using `svnsync` or similar.

There is a serious disadvantage to this method: every time something is to be committed, a `svn switch --relocate` to the master repository has to be done, remembering to `svn switch` back to the mirror after the commit. Also, since `svn switch` only works between repositories that have the same UUID, some hacking of the local repository's UUID has to occur before it is possible to start using it.

Unlike with CVS and csup, the hassle of a local `svnsync` mirror probably is not worth it unless the network connectivity situation or other factors demand it. If it is needed, see the end of this chapter for information on how to set one up.

### 4.2.3 Checkout from a Local Mirror Using SVK

The third alternative is to use SVK to maintain a local mirror. It is a version control system build on top of Subversion's storage engine. It is identical to Subversion in most respects, except that it allows for setting up parts of repositories as mirrors of other repositories, and keeping local branches for merging back into the upstream repositories. There are extensions that allow SVK to mirror CVS and Perforce repositories in addition to Subversion ones.

Like everything, SVK has its disadvantages, one being that local revision numbers will not match upstream revision numbers. This makes it difficult to `svk log`, `svk diff`, or `svk update` to an arbitrary upstream revision.

To set up a mirror of the FreeBSD repository, do:

    % svk mirror svn+ssh://svn.freebsd.org/base //freebsd/base

The local SVK repository will be stored in `~/.svk/local/`, but can be moved to an alternate location. If it is moved, `~/.svk/config` should be amended manually to reflect the move.

Any path can be used, not just the one in the example above. A common pattern is to place mirrors under `//mirror`, e.g. `//mirror/freebsd/base/`, and local branches under `//local`.

To pull down the contents of the repository to the mirror:

    % svk sync //freebsd/base

> Note: `svk sync` will take a very long time, possibly several days over a slow network connection. Peter Wemm `<peter@FreeBSD.org>` has a tarball that can be used to jumpstart the mirror, but only if one does not exist already.

To use the tarball referenced above:

    % cd ~
    % scp freefall:/home/peter/dot_svk_r179646.tbz2 .
    % tar xf dot_svk_r179646.tbz2

Then edit `~/.svk/config` and replace `/scratch/tmp/peter/.svk/local/` with the equivalent of `/home/`jarjar`/.svk/local/`.

You can check out files directly from your mirror, once it has been created:

    % svk checkout //freebsd/base/head /usr/src

Unlike SVN, SVK does not store metadata or reference copies in the working copy. All metadata is recorded in `~/.svk/config`; reference copies are not used at all because SVK always operates on a local repository.

When committing from a working copy like the one above, SVN will commit directly to the upstream repository, then syncronise the mirror.

However, the killer app for SVK is the ability to work without a network connection. To do that, a local branch must be set up:

    % svk mkdir //local/freebsd
    % svk copy //freebsd/base/head //local/freebsd/head

Once again, any path can be used, it does not have to specifically be the one in the example.

Before use, the local branch has to be synchronised, like so:

    % svk pull //local/freebsd/head

Then check out from the newly created local branch:

    % svk checkout //local/freebsd/head /usr/src

The point of this exercise is showing that it is possible to commit work-in-progress to a local branch, and only push it to the upstream repository when work is complete. The easy way to push is with `svk push`, but there is a serious disadvantage to it: it will push every single commit made to the local branch incrementally instead of lumping them all into a single commit. Therefore, using `svk smerge` is preferable.

### 4.2.4 `RELENG_*` Branches and General Layout

In `svn+ssh://svn.freebsd.org/base`, base refers to the source tree. Similarly, ports refers to the ports tree, and so on. These are separate repositories with their own change number sequences, access controls and commit mail.

For the base repository, HEAD refers to the -CURRENT tree. For example, `head/bin/ls` is what would go into `/usr/src/bin/ls` in a release. Some other key locations are:

- /stable/`n` which corresponds to `RELENG_`n``.

- /releng/`n.n` which corresponds to `RELENG_`n_n``.

- /release/`n.n.n` which corresponds to `RELENG_`n_n_n`_RELEASE`.

- /vendor* is the vendor branch import work area. This directory itself does not contain branches, however its subdirectories do. This contrasts with the stable, releng and release directories.

- /projects and /user feature a branch work area, like in Perforce. As above, the /user directory does not contain branches itself.

## 4.3 Daily Use

This section will explain how to perform common day-to-day operations with Subversion. There should be no difference between SVN and SVK in daily use, except for the revision renumbering mentioned earlier.

> Note: SVN and SVK commands that have direct CVS equivalents usually have the same name and abbreviations. For example: checkout and co, update and up, and commit and ci.

### 4.3.1 Help

Both SVN and SVK have built in help documentation. It can be accessed by typing the following command:

    % svn help

### 4.3.2 Checkout

As seen earlier, to check out the FreeBSD head branch:

    % svn checkout svn+ssh://svn.freebsd.org/base/head /usr/src

At some point, more than just `HEAD` will probably be useful, for instance when merging changes to stable/7. Therefore, it may be useful to have a partial checkout of the complete tree (a full checkout would be very painful).

To do this, first check out the root of the repository:

    % svn checkout --depth=immediates svn+ssh://svn.freebsd.org/base

This will give `base` with all the files it contains (at the time of writing, just `ROADMAP.txt`) and empty subdirectories for `head`, `stable`, `vendor` and so on.

Expanding the working copy is possible. Just change the depth of the various subdirectories:

    % svn up --set-depth=infinity base/head
    % svn up --set-depth=immediates base/release base/releng base/stable

The above command will pull down a full copy of `head`, plus empty copies of every `release` tag, every `releng` branch, and every `stable` branch.

If at a later date merging to `7-STABLE` is required, expand the working copy:

    % svn up --set-depth=infinity base/stable/7

Subtrees do not have to be expanded completely. For instance, expanding only `stable/7/sys` and then later expand the rest of `stable/7`:

    % svn up --set-depth=infinity base/stable/7/sys
    % svn up --set-depth=infinity base/stable/7

Updating the tree with `svn update` will only update what was previously asked for (in this case, `head` and `stable/7`; it will not pull down the whole tree.

> Note: Decreasing the depth of a working copy is not possible.

### 4.3.3 Anonymous Checkout

It is possible to anonymously check out the FreeBSD repository with Subversion. This will give access to a read-only tree that can be updated, but not committed to. To do this, use one of the following commands:

    % svn co svn://svn.freebsd.org/base/head /usr/src
    % svn co http://svn.freebsd.org/base/head /usr/src

### 4.3.4 Updating the Tree

To update a working copy to either the latest revision, or a specific revision:

    % svn update
    % svn update -r12345

### 4.3.5 Status

To view the local changes that have been made to the working copy:

    % svn status

CVS has no direct equivalent of this command. The nearest would be `cvs up -N` which shows local changes and files that are out-of-date. Doing this in SVN is possible too, however:

    % svn status --show-updates

### 4.3.6 Editing and Committing

Like CVS but unlike Perforce, SVN and SVK do not need to be told in advance about file editing.

`svn commit` works like the equivalent CVS command. To commit all changes in the current directory and all subdirectories:

    % svn commit

To commit all changes in, for example, ``lib/libfetch/`` and ``usr/bin/fetch/`` in a single operation:

    % svn commit lib/libfetch usr/bin/fetch

### 4.3.7 Adding and Removing Files

> Note: Before adding files, get a copy of auto-props.txt and add it to `~/.subversion/config` according to the instructions in the file. If you added something before you've read this, you may use `svn rm --keep-local` for just added files, fix your config file and re-add them again. The initial config file is created when you first run a svn command, even something as simple as `svn help`.

As with CVS, files are added to a SVN repository with `svn add`. To add a file named foo, edit it, then:

    % svn add foo

Files can be removed with `svn remove`:

    % svn remove foo

Subversion does not require `rm`ing the file before `svn rm`ing it, and indeed complains if that happens.

It is possible to add directories with `svn add`:

    % mkdir bar
    % svn add bar

Although `svn mkdir` makes this easier by combining the creation of the directory and the adding of it:

    % svn mkdir bar

In CVS, the directory is immediately created in the repository when you `cvs add` it; this is not the case in Subversion. Furthermore, unlike CVS, Subversion allows directories to be removed using `svn rm`, however there is no `svn rmdir`:

    % svn rm bar

### 4.3.8 Copying and Moving Files

The following (obviously) creates a copy of `foo.c`, named `bar.c`:

    % svn copy foo.c bar.c

To move and rename a file:

    % svn move foo.c bar.c

The above command is the exact equivalent of:

    % svn copy foo.c bar.c
    % svn remove foo.c

Neither of these operations have equivalents in CVS.

### 4.3.9 Log and Annotate

`svn log` will show all the revisions that affect a directory and files within that directory in reverse chronological order, if run on a directory. This contrasts with `cvs log` in that CVS shows the complete log for each file in the directory, including duplicate entries for revisions that affect multiple files.

`svn annotate`, or equally `svn praise` or `svn blame`, is equivalent to `cvs annotate` in everything but output format.

### 4.3.10 Diffs

`svn diff` displays changes to the working copy of the repository. Diffs generated by SVN are unified by default, unlike CVS, and include new files by default in the diff output.

As with CVS, `svn diff` can show the changes between two revisions of the same file:

    % svn diff -r179453:179454 ROADMAP.txt

It can also show all changes for a specific changeset. The following will show what changes were made to the current directory and all subdirectories in changeset 179454:

    % svn diff -c179454 .

### 4.3.11 Reverting

Local changes (including additions and deletions) can be reverted using `svn revert`. Unlike `cvs up -C`, it does not update out-of-date files--it just replaces them with pristine copies of the original version.

### 4.3.12 Conflicts

If a `svn update` resulted in a merge conflict, Subversion will remember which files have conflicts and refuse to commit any changes to those files until explicitly told that the conflicts have been resolved. The simple, not yet deprecated procedure is the following:

    % svn resolved foo

However, the preferred procedure is:

    % svn resolve --accept=working foo

The two examples are equivalent. Possible values for `--accept` are:

- `working`: use the version in your working directory (which one presumes has been edited to resolve the conflicts).

- `base`: use a pristine copy of the version you had before `svn update`, discarding your own changes, the conflicting changes, and possibly other intervening changes as well.

- `mine-full`: use what you had before `svn update`, including your own changes, but discarding the conflicting changes, and possibly other intervening changes as well.

- `theirs-full`: use the version that was retrieved when you did `svn update`, discarding your own changes.

## 4.4 Advanced Use

### 4.4.1 Sparse Checkouts

The equivalent to `cvs checkout -l`, which checks out a directory without its subdirectories, is `svn checkout -N`. Unlike CVS, SVN remembers the `-N` so that a `svn update` does not end up pulling down the subdirectories. In Subversion 1.5 and newer, `-N` has been deprecated in favour of the `--depth` option which allows for precise control. Therefore:

    % svn checkout -N svn+ssh://svn.freebsd.org/base ~/freebsd

is equivalent to:

    % svn checkout --depth=empty svn+ssh://svn.freebsd.org/base ~/freebsd

Valid arguments to `--depth` are:

- `empty`: the directory itself without any of its contents.

- `files`: the directory and any files it contains.

- `immediates`: the directory and any files and directories it contains, but none of the subdirectories' contents.

- `infinity`: anything.

The `--depth` option applies to many other commands, including `svn commit`, `svn revert`, and `svn diff`.

Since `--depth` is sticky, there is a `--set-depth` option for `svn update` that will change the selected depth. Thus, given the working copy produced by the previous example:

    % cd ~/freebsd
    % svn update --set-depth=immediates .

The above command will populate the working copy in `~/freebsd` with `ROADMAP.txt` and empty subdirectories, and nothing will happen when `svn update` is executed on the subdirectories. However, the following command will set the depth for head (in this case) to infinity, and fully populate it:

    % svn update --set-depth=infinity head

### 4.4.2 Direct Operation

Certain operations can be performed directly on the repository, without touching the working copy. Specifically, this applies to any operation that does not require editing a file, including:

- `log`, `diff`.

- `mkdir`.

- `remove`, `copy`, `rename`.

- `propset`, `propedit`, `propdel`.

- `merge`.

Branching is very fast. The following command would be used to branch `RELENG_8`:

    % svn copy svn+ssh://svn.freebsd.org/base/head svn+ssh://svn.freebsd.org/base/stable/8

This is equivalent to the following set of commands which take minutes and hours as opposed to seconds, depending on your network connection:

    % svn checkout --depth=immediates svn+ssh://svn.freebsd.org/base
    % cd base
    % svn update --depth=infinity head
    % svn copy head stable/8
    % svn commit stable/8

### 4.4.3 Merging with SVN

This section deals with merging code from one branch to another (typically, from head to a stable branch).

> Note: In all examples below, `$FSVN` refers to the location of the FreeBSD Subversion repository, `svn+ssh://svn.freebsd.org/base/`.

#### 4.4.3.1 About Merge Tracking

From the user's perspective, merge tracking information (or mergeinfo) is stored in a property called `svn:mergeinfo`, which is a comma-separated list of revisions and ranges of revisions that have been merged. When set on a file, it applies only to that file. When set on a directory, it applies to that directory and its descendants (files and directories) except for those that have their own `svn:mergeinfo`.

It is not inherited. For instance, `stable/6/contrib/openpam/` does not implicitly inherit mergeinfo from `stable/6/`, or `stable/6/contrib/`. Doing so would make partial checkouts very hard to manage. Instead, mergeinfo is explicitly propagated down the tree. For merging something into `branch/foo/bar/`, the following rules apply:

1. If `branch/foo/bar/` doesn't already have a mergeinfo record, but a direct ancestor (for instance, `branch/foo/`) does, then that record will be propagated down to `branch/foo/bar/` before information about the current merge is recorded.

2. Information about the current merge will not be propagated back up that ancestor.

3. If a direct descendant of `branch/foo/bar/` (for instance, `branch/foo/bar/baz/`) already has a mergeinfo record, information about the current merge will be propagated down to it.

If you consider the case where a revision changes several separate parts of the tree (for example, `branch/foo/bar/` and `branch/foo/quux/`), but you only want to merge some of it (for example, `branch/foo/bar/`), you will see that these rules make sense. If mergeinfo was propagated up, it would seem like that revision had also been merged to `branch/foo/quux/`, when in fact it had not been.

#### 4.4.3.2 Selecting the Source and Target

Because of mergeinfo propagation, it is important to choose the source and target for the merge carefully to minimise property changes on unrelated directories.

The rules for selecting the merge target (the directory that you will merge the changes to) can be summarised as follows:

1. Never merge directly to a file.

2. Never, ever merge directly to a file.

3. Never, ever, ever merge directly to a file.

4. Changes to kernel code should be merged to `sys/`. For instance, a change to the ichwd(4) driver should be merged to `sys/`, not `sys/dev/ichwd/`. Likewise, a change to the TCP/IP stack should be merged to `sys/`, not `sys/netinet/`.

5. Changes to code under `etc/` should be merged at `etc/`, not below it.

6. Changes to vendor code (code in `contrib/`, `crypto/` and so on) should be merged to the directory where vendor imports happen. For instance, a change to `crypto/openssl/util/` should be merged to `crypto/openssl/`. This is rarely an issue, however, since changes to vendor code are usually merged wholesale.

7. Changes to userland programs should as a general rule be merged to the directory that contains the Makefile for that program. For instance, a change to `usr.bin/xlint/arch/i386/` should be merged to `usr.bin/xlint/`.

8. Changes to userland libraries should as a general rule be merged to the directory that contains the Makefile for that library. For instance, a change to `lib/libc/gen/` should be merged to `lib/libc/`.

9. There may be cases where it makes sense to deviate from the rules for userland programs and libraries. For instance, everything under `lib/libpam/` is merged to `lib/libpam/`, even though the library itself and all of the modules each have their own Makefile.

10. Changes to manual pages should be merged to `share/man/man`N`/`, for the appropriate value of `N`.

11. Other changes to `share/` should be merged to the appropriate subdirectory and not to `share/` directly.

12. Changes to a top-level file in the source tree such as `UPDATING` or `Makefile.inc1` should be merged directly to that file rather than to the root of the whole tree. Yes, this is an exception to the first three rules.

13. When in doubt, ask.

If you need to merge changes to several places at once (for instance, changing a kernel interface and every userland program that uses it), merge each target separately, then commit them together. For instance, if you merge a revision that changed a kernel API and updated all the userland bits that used that API, you would merge the kernel change to sys, and the userland bits to the appropriate userland directories, then commit all of these in one go.

The source will almost invariably be the same as the target. For instance, you will always merge `stable/7/lib/libc/` from `head/lib/libc/`. The only exception would be when merging changes to code that has moved in the source branch but not in the parent branch. For instance, a change to pkill(1) would be merged from `bin/pkill/` in head to `usr.bin/pkill/` in stable/7.

#### 4.4.3.3 Preparing the Merge Target

Because of the mergeinfo propagation issues described earlier, it is very important that you never merge changes into a sparse working copy. You must always have a full checkout of the branch you will merge into. For instance, when merging from HEAD to 7, you must have a full checkout of stable/7:

    % cd stable/7
    % svn up --set-depth=infinity

The target directory must also be up-to-date and must not contain any uncommitted changes or stray files.

#### 4.4.3.4 Identifying Revisions

Identifying revisions to be merged is a must. If the target already has complete mergeinfo, ask SVN for a list:

    % cd stable/6/contrib/openpam
    % svn mergeinfo --show-revs=eligible $FSVN/head/contrib/openpam

If the target does not have complete mergeinfo, check the log for the merge source.

#### 4.4.3.5 Merging

Now, let's start merging!

##### 4.4.3.5.1 The Principles

Say you would like to merge:

- revision `$R`.

- in directory $target in stable branch $B.

- from directory $source in head.

- $FSVN is `svn+ssh://svn.freebsd.org/base`.

Assuming that revisions $P and $Q have already been merged, and that the current directory is an up-to-date working copy of stable/$B, the existing mergeinfo looks like this:

    % svn propget svn:mergeinfo -R $target
    $target - /head/$source:$P,$Q

Merging is done like so:

    % svn merge -c$R $FSVN/head/$source $target

Checking the results of this is possible with `svn diff`.

The svn:mergeinfo now looks like:

    % svn propget svn:mergeinfo -R $target
    $target - head/$source:$P,$Q,$R

If the results are not exactly as shown, assistance may be required before committing as mistakes may have been made, or there may be something wrong with the existing mergeinfo, or there may be a bug in Subversion.

##### 4.4.3.5.2 Merging into the Kernel (`sys/`)

As stated above, merging into the kernel is different from merging in the rest of the tree. In many ways merging to the kernel is simpler because there is always the same merge target (`sys/`).

Once `svn merge` has been executed, `svn diff` has to be run on the directory to check the changes. This may show some unrelated property changes, but these can be ignored. Next, build and test the kernel, and, once the tests are complete, commit the code as normal, making sure that the commit message starts with Merge `r226222` from head, or similar.

#### 4.4.3.6 Precautions Before Committing

As always, build world (or appropriate parts of it).

Check the changes with `svn diff` and `svn stat`. Make sure all the files that should have been added or deleted were in fact added or deleted.

Take a closer look at any property change (marked by a `M` in the second column of `svn stat`). Normally, no svn:mergeinfo properties should be anywhere except the target directory (or directories).

If something looks fishy, ask for help.

#### 4.4.3.7 Committing

Make sure to commit a top level directory to have the mergeinfo included as well. Do not specify individual files on the command line. For more information about committing files in general, see the relevant section of this primer.

### 4.4.4 Vendor imports with SVN

> Important: Please read this entire section before starting a vendor import.

> Note: Patches to vendor code fall into two categories:

> - Vendor patches: these are patches that have been issued by the vendor, or that have been extracted from the vendor's version control system, which address issues which in your opinion can't wait until the next vendor release.

> - FreeBSD patches: these are patches that modify the vendor code to address FreeBSD-specific issues.

> The nature of a patch dictates where it should be committed:

> - Vendor patches should be committed to the vendor branch, and merged from there to head. If the patch addresses an issue in a new release that is currently being imported, it must not be committed along with the new release: the release must be imported and tagged first, then the patch can be applied and committed. There is no need to re-tag the vendor sources after committing the patch.

> - FreeBSD patches should be committed directly to head.

#### 4.4.4.1 Preparing the tree

If importing for the first time after the switch to Subversion, flattening and cleaning up the vendor tree is necessary, as well as bootstrapping the merge history in the main tree.

##### 4.4.4.1.1 Flattening

During the conversion from CVS to Subversion, vendor branches were imported with the same layout as the main tree. This means that the `pf` vendor sources ended up in `vendor/pf/dist/contrib/pf`. The vendor source is best directly in `vendor/pf/dist`.

To flatten the `pf` tree:

    % cd vendor/pf/dist/contrib/pf
    % svn mv $(svn list) ../..
    % cd ../..
    % svn rm contrib
    % svn propdel -R svn:mergeinfo .
    % svn commit

The `propdel` bit is necessary because starting with 1.5, Subversion will automatically add `svn:mergeinfo` to any directory that is copied or moved. In this case, as nothing is being merged from the deleted tree, they just get in the way.

Tags may be flattened as well (3, 4, 3.5 etc.); the procedure is exactly the same, only changing `dist` to `3.5` or similar, and putting the `svn commit` off until the end of the process.

##### 4.4.4.1.2 Cleaning up

The `dist` tree can be cleaned up as necessary. Disabling keyword expansion is recommended, as it makes no sense on unmodified vendor code and in some cases it can even be harmful. OpenSSH, for example, includes two files that originated with FreeBSD and still contain the original version tags. To do this:

    % svn propdel svn:keywords -R .
    # svn commit

##### 4.4.4.1.3 Bootstrapping merge history

If importing for the first time after the switch to Subversion, bootstrapping `svn:mergeinfo` on the target directory in the main tree to the to the revision that corresponds to the last related change to the vendor tree, prior to importing new sources:

    % cd head/contrib/pf
    % svn merge --record-only svn+ssh://svn.freebsd.org/base/vendor/pf/dist@180876 .
    % svn commit

#### 4.4.4.2 Importing new sources

With two commits--one for the import itself and one for the tag--this step can optionally be repeated for every upstream release between the last import and the current import.

##### 4.4.4.2.1 Preparing the vendor sources

Unlike in CVS where only the needed parts were imported into the vendor tree to avoid bloating the main tree, Subversion is able to store a full distribution in the vendor tree. So, import everything, but merge only what is required.

A `svn add` is required to add any files that were added since the last vendor import, and `svn rm` is required to remove any that were removed since. Preparing sorted lists of the contents of the vendor tree and of the sources that are about to be imported is recommended, to facilitate the process.

    % cd vendor/pf/dist
    % svn list -R | grep -v '/$' | sort >../old
    % cd ../pf-4.3
    % find . -type f | cut -c 3- | sort >../new

With these two files, `comm -23 ../old ../new` will list removed files (files only in `old`), while `comm -13 ../old ../new` will list added files only in `new`.

##### 4.4.4.2.2 Importing into the vendor tree

Now, the sources must be copied into ``dist`` and the `svn add` and `svn rm` commands should be used as needed:

    % cd vendor/pf/pf-4.3
    % tar cf - . | tar xf - -C ../dist
    % cd ../dist
    % comm -23 ../old ../new | xargs svn rm
    % comm -13 ../old ../new | xargs svn --parents add

If any directories were removed, they will have to be `svn rm`ed manually. Nothing will break if they are not, but they will remain in the tree.

Check properties on any new files. All text files should have `svn:eol-style` set to `native`. All binary files should have `svn:mime-type` set to `application/octet-stream` unless there is a more appropriate media type. Executable files should have `svn:executable` set to `*`. No other properties should exist on any file in the tree.

Committing is now possible, however it is good practice to make sure that everything is OK by using the `svn stat` and `svn diff` commands.

##### 4.4.4.2.3 Tagging

Once committed, vendor releases should be tagged for future reference. The best and quickest way to do this is directly in the repository:

    % svn cp svn+ssh://svn.freebsd.org/base/vendor/pf/dist svn+ssh://svn.freebsd.org/base/vendor/pf/4.3

Once that is complete, `svn up` the working copy of ``vendor/pf`` to get the new tag, although this is rarely needed.

If creating the tag in the working copy of the tree, `svn:mergeinfo` results must be removed:

    % cd	vendor/pf
    % svn cp dist 4.3
    % svn propdel svn:mergeinfo -R 4.3

#### 4.4.4.3 Merging to head

    % cd head/contrib/pf
    % svn up
    % svn merge --accept=postpone svn+ssh://svn.freebsd.org/base/vendor/pf/dist .

The `--accept=postpone` tells Subversion that it shouldn't complain because merge conflicts will be taken care of manually.

It is necessary to resolve any merge conflicts. This process is the same in SVN as in CVS.

Make sure that any files that were added or removed in the vendor tree have been properly added or removed in the main tree. To check diffs against the vendor branch:

    % svn diff --no-diff-deleted --old=svn+ssh://svn.freebsd.org/base/vendor/pf/dist --new=.

The `--no-diff-deleted` tells Subversion not to complain about files that are in the vendor tree but not in the main tree, i.e. things that would have previously been removed before the vendor import, like for example the like the vendor's makefiles and configure scripts.

Using CVS, once a file was off the vendor branch, it was not able to be put back. With Subversion, there is no concept of on or off the vendor branch. If a file that previously had local modifications, to make it not show up in diffs in the vendor tree, all that has to be done is remove any left-over cruft like FreeBSD version tags, which is much easier.

If any changes are required for the world to build with the new sources, make them now, and keep testing until everything builds and runs perfectly.

#### 4.4.4.4 Committing the vendor import

Committing is now possible! Everything must be committed in one go. If done properly, the tree will move from a consistent state with old code, to a consistent state with new code.

### 4.4.5 Reverting a Commit

Reverting a commit to a previous version is fairly easy:

    % svn merge -r179454:179453 ROADMAP.txt
    % svn commit

Change number syntax, with negative meaning a reverse change, can also be used:

    % svn merge -c -179454 ROADMAP.txt
    % svn commit

This can also be done directly in the repository:

    % svn merge -r179454:179453 svn+ssh://svn.freebsd.org/base/ROADMAP.txt

Reverting the deletion of a file is slightly different. Copying the version of the file that predates the deletion is required. For example, to restore a file that was deleted in revision N, restore version N-1:

    % svn copy svn+ssh://svn.freebsd.org/base/ROADMAP.txt@179454
    % svn commit

or, equally:

    % svn copy svn+ssh://svn.freebsd.org/base/ROADMAP.txt@179454 svn+ssh://svn.freebsd.org/base

Do not simply recreate the file manually and `svn add` it--this will cause history to be lost.

### 4.4.6 Fixing Mistakes

While we can do surgery in an emergency, do not plan on having mistakes fixed behind the scenes. Plan on mistakes remaining in the logs forever. Be sure to check the output of `svn status` and `svn diff` before committing.

Mistakes will happen, but, unlike with CVS, they can generally be fixed without disruption.

Take a case of adding a file in the wrong location. The right thing to do is to `svn move` the file to the correct location and commit. This causes just a couple of lines of metadata in the repository journal, and the logs are all linked up correctly.

The wrong thing to do is to delete the file and then `svn add` an independent copy in the correct location. Instead of a couple of lines of text, the repository journal grows an entire new copy of the file. This is a waste.

### 4.4.7 Setting up a svnsync Mirror

You probably do not want to do this unless there is a good reason for it. Such reasons might be to support many multiple local read-only client machines, or if your network bandwidth is limited. Starting a fresh mirror from empty would take a very long time. Expect a minimum of 10 hours for high speed connectivity. If you have international links, expect this to take 4 to 10 times longer.

A far better option is to grab a seed file. It is large (~1GB) but will consume less network traffic and take less time to fetch than a svnsync will. This is possible in one of the following three ways:

    % rsync -va --partial --progress freefall:/home/peter/svnmirror-base-r179637.tbz2 .

    % rsync -va --partial --progress rsync://repoman.freebsd.org:50873/svnseed/svnmirror-base-r215629.tar.xz .

    % fetch ftp://ftp.freebsd.org/pub/FreeBSD/development/subversion/svnmirror-base-r221445.tar.xz

Once you have the file, extract it to somewhere like `home/svnmirror/base/`. Then, update it, so that it fetches changes since the last revision in the archive:

    % svnsync sync file:///home/svnmirror/base

You can then set that up to run from cron(8), do checkouts locally, set up a svnserve server for your local machines to talk to, etc.

The seed mirror is set to fetch from `svn://svn.freebsd.org/base`. The configuration for the mirror is stored in `revprop 0` on the local mirror. To see the configuration, try:

    % svn proplist -v --revprop -r 0 file:///home/svnmirror/base

Use `propset` to change things.

### 4.4.8 Committing High-ASCII Data

Files that have high-ASCII bits are considered binary files in SVN, so the pre-commit checks fail and indicate that the `mime-type` property should be set to `application/octet-stream`. However, the use of this is discouraged, so please do not set it. The best way is always avoiding high-ASCII data, so that it can be read everywhere with any text editor but if it is not avoidable, instead of changing the mime-type, set the `fbsd:notbinary` property with `propset`:

    % svn propset fbsd:notbinary yes foo.data

### 4.4.9 Maintaining a Project Branch

A project branch is one that's synced to head (or another branch) is used to develop a project then commit it back to head. In SVN, dolphin branching is used for this. A dolphin branch is one that diverges for a while and is finally committed back to the original branch. During development code migration in one direction (from head to the branch only). No code is committed back to head until the end. Once you commit back at the end, the branch is dead (although you can have a new branch with the same name after you delete the branch if you want).

As per http://people.freebsd.org/~peter/svn_notes.txt, work that is intended to be merged back into HEAD should be in `base/projects/`. If you are doing work that is beneficial to the FreeBSD community in some way but not intended to be merged directly back into HEAD then the proper location is `base/user/`your-name`/`. This page contains further details.

To create a project branch:

    % svn copy svn+ssh://svn.freebsd.org/base/head svn+ssh://svn.freebsd.org/base/projects/spif

To merge changes from HEAD back into the project branch:

    % cd copy_of_spif
    % svn merge svn+ssh://svn.freebsd.org/base/head
    % svn commit

It is important to resolve any merge conflicts before committing.

To collapse everything back at the end:

    % svn write me

## 4.5 Some Tips

In commit logs etc., rev 179872 should be spelled r179872 as per convention.

Speeding up checkouts and minimising network traffic is possible with the following recipe:

    % svn co --depth=empty svn+ssh://svn.freebsd.org/base fbsvn
    % cd fbsvn
    % svn up --depth=empty stable
    % svn up head
    % cd stable
    % cp -r ../head/ 7
    % cd 7
    % svn switch svn+ssh://svn.freebsd.org/base/stable/7
    % cd ..
    % cp -r 7/ 6
    % cd 6
    % svn switch svn+ssh://svn.freebsd.org/base/stable/6

What this bit of evil does is check out head, stable/7 and stable/6. We create the empty checkout directories under SVN's control. In SVN, subtrees are self identifying, like in CVS. We check out head and clone it as stable/7. Except we don't want the head version so we switch it to the 7.x tree location. SVN downloads diffs to convert the head files to stable/7 instead of doing a fresh checkout. The same goes for stable/6. This does, however, definitely count as abuse of the working copy client code!

Checking out a working copy with a stock Subversion client withouth FreeBSD-specific patches (`WITH_FREEBSD_TEMPLATE`) will mean that `$FreeBSD$` tags will not be expanded. Once the correct version has been installed, trick Subversion into expanding them like so:

    % svn propdel -R svn:keywords .
    % svn revert -R .

This is not a good idea if uncommitted patches exist, however.

Prev | Home | Next
Version Control System Operations |  | Conventions and Traditions