Version Control - CVS Procedures, Database Upgrades, Etc.

by Andrew Piskorski Last Revised: 07 May 2005

Articles and Tutorials:

Books and Reference Manuals:

Other Useful CVS Links:

Obsolete Links:
These used to be useful, but their links are permanently broken:

Companion Documents

This is intended to be a policy document and cheatsheet for doing version control and release management on a website, principally via use of CVS. While it contains extensive "howto" information, it is not intended to be either a reference manual or general tutorial on CVS - read the above articles first.

We will also try to make this document generally applicable, by placing any information highly specific to our own project in a separate companion document, listed above.

Possible alternatives to CVS:

In short, CVS is neither the perfect tool for software development nor for versioning UNIX system administration files, but while several potentially superior alternatives exists, currently (early 2003) none of them are a clear and compelling alternative to CVS. CVS is the de-facto standard. So unless you already know why you should use something other than CVS, you can safely ignore this section and go on to the rest of this document.

The Better SCM Initiative has some good info comparing the different not-CVS systems that might actually be or become viable alternatives to CVS. Rick Moen's list of version control systems on Linux, has useful descriptions, and Martin Pool (developer of rsync and distcc, etc.) has extensive thoughtful commentary on version control systems. Many others systems are listed at Google.

Technically, Bitkeeper is probably an excellent (perhaps the best) alternative, but it is closed source. Features and user reports sound very positive. And among other things, Linus Torvalds chose it for the Linux kernel, so it has clearly been proven on at least one very large distributed software project. I have not used it, but it sounds better than at least any of the other closed source version control systems I've read about.

Eventually, Subversion, OpenCM, Arch, Darcs, or Monotone might be able to replace CVS as the de-facto open source version control tool of choice, but I'm not really familiar with any of them. Caveat Hacktor. (For more related version control links, see also these posts, as well as some others in that thread.)

TODO: I really should get around to experimenting with at least the more interesting (distributed, change set oriented, Open Source, relatively stable and ready to use) systems, Real Soon Now.

Overview:

For web development, project version and configuration control falls roughtly into three main bins: CVS, Database Upgrades, and Other. This document originally dealt only with the CVS piece of the puzzle (hence the filename), but will now briefly touch on the other two as well.

CVS changes:

If you've read Ron's article, above, you know there are three basic ways to use CVS for Web Development version control: One Branch, Two Branches, and Many Branches.

So the first question is which method of using CVS do we use? My past experience says to use either One Branch or Many Branches. For simple projects, One Branch will often be simplest, fastest, and best. For some large, complicated, or long-term projects - particularly after they already have a Production server launched and in regular use - the Many Branches method may be useful. (And in fact, if you have released one or more Production versions of your code to outside users, you are pretty much forced to use the Many Branches method in some form.)

As for the the Two Branches method, I don't recomend it. While I have not run a project using it, it strikes me as just as much work as Many Branches, without all the benefits.

Also, if you do use the Many Branches you'll probably want to avoid mixing in the Two Branches method as well. I have mixed the two, and it can work fine, but the extra complexity usually isn't worth dealing with. Keep things simple if you can.

Note that nothing about CVS locks you into any of the above methods. We can and will use the "other methods" should circumstances warrant it.

Non-CVS changes:

Any and all changes not handled via CVS go in the upgrade.sql file. This is usually database changes but may be anything at all. In other words, if making the change happen involves anything other than just doing a cvs update, it should probably be in the upgrade.sql file.

Admitedly, shoving things like "Go to /acs-admin/apm/ and use the web UI, install ACS package X, mount it on URL Y, and also do Z, because I was to lazy to figure out how to have a script set that up automatically." into upgrade.sql is a kludge. But at least that way all needed non-cvs changes are noted in one place.

You will probably see a natural change in your use of the upgrade.sql file as your site moves from developing something brand new, to launching and then maintaining a production site. When you start a brand new project and do not have any Production site launched, whenever you make changes to your data model you will want to simply drop and re-create your package, so you won't have much genuine data model upgrade stuff in there. Later on, necessity may force you to make all data model changes, even on Dev, via this upgrade script, as that's what you'll need to do on Production. And in fact, I first came up with this "put all upgrades into upgrade.sql" scheme in exactly that situation (the Muniversal project, which I worked on for roughly 10 months, roughly 8 months of which were after we'd launched a Production site).

Whatever you do, don't make changes to your data model one way on Dev, and then expect to come up with a whole different way of implementing those changes on your existing Production site! That's really asking for trouble.

In general, when making all data model changes as alterations to the live database via upgrade.sql on Muniversal, I found that it was not unusual to run into some problems when upgrading Staging, but most of the time the upgrade to Production went quite smoothly. In other words, one pass of testing was usually enough to work the bugs out.

(Note: If there exists a tool that can version and diff live relational data models as well as CVS can version and diff text files, all without losing any data in the tables, I'd like to hear about it!)

More on the One Branch method, and cvs-stable.tcl:

For the one branch method, typically, we use one or two UNIX machines, three AOLservers, and three database users. All development takes place on the Development server. Developers check their changes into CVS continually. The development machine also hosts the staging server, with its own AOLserver and Oracle user. The staging server has two uses: (1) as a testing area for new code, and (2) as a testbed for updating the production server.

The very simplest way to do this is with no tags or any other form of explicit control at all. Just commit things on Dev, and when you're ready Flow of code is controlled solely by when you do "cvs update" commands in the Staging or Production working copy. This almost isn't worth mentioning here as it is the simplest and most obvious way to use CVS. However, you run the risk of accidentally pushing out code to Staging or Production when it isn't ready, and then of not even knowing what version was actually in use on Production before your mistake. So from now on we will assume from here on that you want at least a bit more insurance than that.

The cvs-stable.tcl script helps us do this with three features. Of these features, two, the -r and -u switches are simply convenience features which we could do with any simple script, or even manually. The third, the -n switch, fills a glaring hole in the CVS API. Here's a quick look at this script's current command-line options: 

$ ~/bin/cvs-stable.tcl -h

Usage:
  cvs-stable.tcl [-h] {-n|-t|-u} [-p] [-r TAG] [ file ... ]
Description:

  Does one of three things, all related to the 'one branch' method of
  using CVS version control - see also:
    "http://www.piskorski.com/cvs-conventions.html"
  You MUST specify either the -n, -t, or -u switch, as the FIRST
  option after the command name.  Here's what each does:

  -n : Displays all files which do NOT have the tag TAG on the current
  working revision of the file.  Use to answer the question, which
  files on Dev are committed but NOT yet marked as ready to go to
  Staging?

  -t : Tags files as 'stable-staging' or 'stable-production', ready to go
  to Staging or Production, respectively.  Also keeps a short rolling
  history of where the cvs tag was before, to help you in case you
  mess up.

  -u : Runs a cvs update command which updates the files in a working
  updates files to the version currently tagged with 'stable-staging'
  (or 'stable-production').  It also sets that as a sticky tag, as is
  normal for the 'cvs update -r TAG' command.

Options:
  -h     : Display this help message.
  -r TAG : Use cvs tag TAG.  Defaults to 'stable-staging'.
  -p     : Use the Production default tag 'stable-production' rather than
           'stable-staging'.

  Note:  The command-line option parsing currently isn't very
  sophisticated so keep the options in the exact same order as shown
  in the pattern above, and do NOT try to do unix-style combined flags
  like '-htp'.

When a developer has a stable version of a file, ready to be pushed to Staging, he can do:

$ cvs-stable.tcl -t myfile
and the file is tagged as ready for the staging server. (Note that this simply tags the last committed version, it does not commit your current changes.)

However, it is usually simpler and better to simply tag everything, like so:

$ cvs-stable.tcl -t
Why? Well, the script keeps a short rolling history in CVS of where the "stable-staging" tag was before. Do a "cvs stat -v" and you will see tags like "stable-staging-1" and "stable-staging-2". If you run "cvs-stable.tcl -t" by accident, or if anything breaks after you push code out to Staging, those tags will help you figure out how to (manually) put things back the way they were before. This is simpler and more robust if the "stable-staging" tag is always moved for all files at once, as a single atomic "this stuff is ready to go to Staging" operation.

In other words, I recommend only running "cvs-stable.tcl -t" right before you push out code to Staging, once per code push-out to Staging. (Disaster is not likely to befall you if you ignore this guideline, but things will probably go more smoothly if you follow it.)

Now, go to the Staging checkout, and run this:

$ cvs-stable.tcl -u

That sets the stick tag "stable-staging" on all files in the Staging checkout, and updates them to that tagged revision. (Note, since the tag is sticky, you only have to use the -u switch once in the Staging checkout. From then on, you can just do a normal "cvs update -d" to update files to the new location of the stable-staging tag. It won't hurt anything to use "cvs-stable.tcl -u" again instead, but it is doing more work, and is overkill.)

Now you're basically set, when Dev is ready to go to Staging you just tag it with "cvs-stable.tcl -t", then go update Staging, that's it. The problem though, is how do you check what files have been committed but not yet tagged as "stable-staging", so you can decide whether they're ready to be tagged as stable-staging?

That's where "cvs-stable.tcl -n" comes in - in fact it is the primary purpose of that script. Simply run it in your Dev working copy, and it will tell what files which committed files present in Dev are not the same version as that tagged stable-staging. E.g.: 

$ ~/bin/cvs-stable.tcl -n
Tag:  stable-staging

File: log.c            	Status: Up-to-date
   Working revision:	1.3	Sun Mar 13 08:32:14 2005
   Repository revision:	1.3	/home/cvsroot/aol4-src/aolserver/nsd/log.c,v
	stable-staging           	(revision: 1.1)
   Working revision  '>'  tagged revision.

File: pathname.c       	Status: Up-to-date
   Working revision:	1.2	Mon Mar 14 11:38:48 2005
   Repository revision:	1.2	/home/cvsroot/aol4-src/aolserver/nsd/pathname.c,v
	stable-staging           	(revision: 1.1)
   Working revision  '>'  tagged revision.
That's it as far as the cvs-stable.tcl script goes. Note that it displays all CVS commands and output, so you can see what's going on.

With this One Branch process, tagged code is moved continually from Dev to Staging (and never the reverse). Any non-CVS changes that need to go to Staging along with the code will be done and noted in the upgrade.sql file by the person moving the code to Staging.

Watchdog is installed on the staging server, so any errors are sent via email to the development team.

When it is time to do a Production release, we tag all the files on Staging with a Production release tag (see the docs on the -p switch above), and then update Production.

Finally, if you want to be more conservative, instead of repeatedly moving the "stable-staging" tag forward, you could instead place a new unique tag (e.g., "stable-staging-2005-05-07") each time you roll code out to Staging or Production, either manually (not hard) or by enhancing the cvs-stable.tcl script to support it. (And you can of course delete really old tags later, to reduce clutter.) This more rigorous method is generally not desirable for Staging, but depending on your project, you may want to consider it for Production.

More on the Many Branches method:

TODO: [Basically, there's not that much to it, as you use only CVS commands excluseively - I never found the need for any anciallary shell scripts as I did witht the One Branch method. See the articles and books listed above.]

Individual vs. Shared Checkouts:

TODO:

[Interpreted and Database-backed (argues for shared Dev) vs. compiled and non-db-backed (argues for individual Dev checkouts), etc.]

Where is the repository?

In the companion document, note both where the CVS repository located, and what each of the projects inside it are.

Record all Vendor Branches:

All code from external sources shall be imported onto a vendor branch.

Note that vendor branches are per CVS project, just like everything else in CVS.

Originally, my thinking was that you should be sure to always use the same vendor branch numbers and names across all our CVS projects, and if for some reason you don't or can't, change the list of vendor branches project specific document to be per-cvs-project.)

Currently (2002/12/19), I suspect that doing so is pointless and possibly counter productive. Better to always import on the default 1.1.1 vendor branch, and only specify a differen branch number when you are truly have code from multiple vendors.

For example, take the example of OpenACS. ArsDigita released ACS 4.2. OpenACS took over all mainenance and evelopment of the codebase, and released OpenACS 4.5. Should OpenACS 4.5 be imported onto the different OpenACS branch? No, it is the next release of the same code, just import it onto the same 1.1.1 branch, which happened to previously be the ArsDigita vendor branch for this code - but not more.

That said, document which way you're doing it!

Tag Naming Conventions:

When naming tags please you will want to follow specific conventions, and record any additions, modifications, or exceptions. Here's a recomendation:

Example Tags:

One Branch method:
tag name, e.g. explanation
stable-staging Tag indicating the current stable version of a file, ready for release to Staging. The exact tag name is defined in the cvs-tag-staging.sh and cvs-up-staging.sh scripts, not here.
release-2001-06-10 Release from Staging to Production. Every Production release has it's own tag, and the same tag will be on every file.
 
Many Branches method:
tag name, e.g. explanation
root-of_release-2000-08-30 Root of tag, placed on the trunk, immediately before doing the branch.
release-2000-08-30_branch Branch tag for release-2000-08-30.
release-2000-08-30_fix-1 First batch of fixes merged back from the Staging branch to the Dev trunk.
release-2000-08-30_fix-2 Second batch of fixes merged back from the Staging branch to the Dev trunk.
release-2000-09-07 There's no _branch suffix, so this means we're merging new stuff from Dev across onto a pre-existing Staging branch.

Some Tagging Rules:

Tagging:

Keep in mind that CVS operations are not necessarily atomic, and tagging a whole source tree can be slow (e.g., 10+ minutes).

There are roughly three (3) different ways to specify where to place a tag:

By default, the tag commands do NOT move a tag if it already exists. So if you're doing something funky, you can just tag all the special cases first on a file-by-file basis, then go back and tag all the rest of the files at once by just issuing a normal recursive tag command.

For example, I had a list of 8 or so files that had more recent revisions committed on Dev, but which I wanted to keep frozen at their older, release-2000-08-30_fix-2 revision on Staging. (Don't ask...) So when preparing to merge some new Dev work across to the existing Staging branch, I did:

$ cd /web/myproject-dev
$ cvs tag -r release-2000-08-30_fix-2 release-2000-09-07 tcl/myproj-defs.tcl tcl/myproj-scheduled-procs.tcl tcl/myproj-widgets.tcl www/admin/trading/trades.tcl www/pvt/trades-by-trader.tcl www/pvt/workspace.tcl
$ cvs tag -R release-2000-09-07

Renaming Tags

There's no special command to rename a tag. Instead, you use the old tag to tell CVS where to attach the new tag, and then once you're sure you did that right, you delete the old tag. E.g., for non-branch tags:
$ cvs tag -r release-2000-08-08-FIX01 release-2000-08-08_fix-1
$ cvs tag -d release-2000-08-08-FIX01
The above is fine for normal tags. However, I think think it may not properly maintain the "branchness" of branch tags - but I'm not sure - more experiment is required. So I recommend that you do not attempt to rename branch tags in this manner, unless you want to experiment.

Shell Tips:

Environment Variables

They may already be set appropriately system-wide for your use, but you'll want to check that the environment variables CVSROOT and CVS_RSH are set to your satisfaction. In particular, you'll want:

CVS_RSH=ssh
You also probably want one of the following in your ~/.profile or ~/.bash_profile:
export EDITOR="emacs -nw"
export EDITOR=emacsclient
export EDITOR=gnuclient

Output Redirection

This has nothing to do with CVS per-se, but it is useful: When doing CVS commands that produce volumnuous output which you may actually want to inspect - e.g., merges - you may want to redirect both standard output and standard error to a file. With a Bourne-style shell (e.g., Bash), simply tack something like:

>> ~/m.txt 2>> ~/m.txt
onto the end of your command line. E.g., when merging fixes from Staging back to Dev, you might do:
cvs up -R -kk -j release-2000-09-24_fix-2 -j release-2000-09-24_fix-3 >> ~/m.txt 2>> ~/m.txt
where m.txt is the file in your home directory that you want all output redirected to.

Importing Code onto a Vendor Branch:

Generally, any code that you receive from an outside source rather than writing yourself should be imported onto a vendor branch, rather than simply added.

Ask yourself these questions: Might I get a ever get code updates or fixes from that same source? Do I want to have a convenient, automatic way of diffing the code we're using vs. the code as we received it? If the answer to any of those questions is "yes", then you definitely want to use a vendor branch.

Initial install of ACS:

When you grab ACS out of the tarball, you might as well check it all into CVS first thing, before you run the install scripts or anything. Don't wait till later. It goes like this: 

$ mkdir /web/mysite-dev
Go and untar acs-4-2-beta.tar.gz into /web/mysite-dev
$ cd /web/mysite-dev/
$ cvs -d ls.arsdigita.com:/cvsweb import -m "Importing ACS 4.2beta" mysite ArsDigita acs-4-2-beta-R20010307 
$ cd /web/
$ cvs -d ls.arsdigita.com:/cvsweb checkout -d mysite-dev mysite

Note that we specified a repository to use with -d ls.arsdigita.com:/cvsweb, but we did not specify a vendor branch number, so CVS is going to use the default 1.1.1 branch. In this case that's fine - we'll be using 1.1.1 as our ArsDigita - but you should probably make a habit of always specifying the vendor branch. See below.

Importing a new ACS Package

Later on, as you download and install separate packages, you'll want to be sure to check each of these into CVS too. It's straightforward:

First, untar everything into an appropriate directory. You could do this anywhere, but I'll assume you used the ACS APM to do this, so all the package files are aleady installed in their normal ACS place, you just haven't check the code into CVS yet.

So, for example for the Monitoring package:

$ cd /web/mysite-dev/packages/monitoring/
$ cvs import -m "Importing Monitoring package v. 4.0.0a" mysite/packages/monitoring ArsDigita monitoring-4-0-0a-R20010201
$ cd ..
$ rm -r monitoring
$ cvs up -d monitoring

If you know the CVS (or other version control system) tag that the vendor used internally for his release, then that's probably what you want to use for the e.g. "monitoring-4-0-0a-R20010201" release tag above. If you don't know, just make up something reasonable.

Note that we told CVS to import the files into the "mysite/packages/monitoring" subdirectory. Make sure you get that part correct, or you will successfully import your files into the wrong place in your CVS repository. (At which point you'll want to go and carefully delete them by hand...)

More Easy Import Examples:

Note that by default CVS ignores some files like *.so completely. So if you want to check those in, you'll want to reset the cvs ignore list with the -I! switch:

$ cvs import -I! -m "Importing stock AOLserver 3.2+ad12 Solaris binary distribution from ArsDigita." aol3 ArsDigita aolserver-3_2_ad12

Sometimes you'll do a normal import, but will also want to do a bit of remedial fix-up work if the software vendor screwed something up:

$ cvs import -b 1.1.17 -m "Importing TclODBC v. 2.2.1, from http://sourceforge.net/projects/tclodbc/" tcl-src/tclodbc SourceForge tclodbc-2-2-1
$ cd ..
$ cvs checkout tcl-src
$ cd tcl-src/tclodbc/config
$ cvs import -b 1.1.17 -m "Importing missing config/ subdirectory, which is now in the SourceForge CVS, but is not tagged with tclodbc-2-2-1 like it should be." tcl-src/tclodbc/config SourceForge tclodbc-2-2-1-missing-config
$ cvs tag tclodbc-2-2-1

Importing AOLserver+ad12 source distribution:

$ cvs import aol3-src ArsDigita aolserver-3_2_ad12
Importing Tom Poindexter's Odbcisql, from http://sourceforge.net/projects/tclodbc/:
$ cvs import -b 1.1.19 tcl-src/odbcisql Tom-Poindexter odbcisql-1-0
Importing nsopenssl 1.1b source distribution, from http://scottg.net/:
$ cvs import -b 1.1.7 aol3-src/nsopenssl scottg nsopenssl-1-1-b
Importing Monitoring package v. 4.0.0a:
$ cvs import mysite/packages/monitoring ArsDigita monitoring-4-0-0a-R20010201
Importing Developer Support package v. 4.0.1a:
$ cvs import mysite/packages/acs-developer-support ArsDigita acs-devloper-support-release-4-0-1a

Upgrading Code - importing code that needs to be merged with existing code:

When importing a new version of code you already have, the process is exactly the same as importing brand new code, except you add a bunch of extra steps. Note that both Fogel and Cederqvist have some good instructions on this.

Remember, when you import code onto a "vendor branch" you really are putting code onto a branch, not the main trunk. When they code doesn't exist at all yet on the main trunk, CVS also makes the stuff you just imported onto the vendor branch rev. 1.1.1 on the trunk. But if you already have a revision 1.1.23 of the foo.sql file, say, then importing a revised foo.sql onto the vendor branch doesn't do anything to the foo.sql on the trunk - you have to merge and commit first.

So to import code that needs to be merged in with existing code, the steps are:

  1. Import new stuff onto vendor branch.
  2. Checkout a new working copy of the trunk with -kk.
  3. Merge in changes from vendor branch, in that working copy.
  4. Resolve any conflicts.
  5. Commit.
  6. Go to any existing checkout that you normally use for your Development work, and update to get the new stuff you just committed.

Note that:

Here are some examples:

Upgrading from ACS 4.2beta to 4.2:

$ cvs import -b 1.1.1 -m "Importing ACS 4.2 (not beta)." mysite ArsDigita acs-4-2-R20010417
$ cd ~ ; cvs checkout -kk -d mysite-tmp mysite
$ cd mysite-tmp
$ cvs up -kk -j acs-4-2-R20010417
$ cvs commit
$ cd /web/mysite-dev/
$ cvs -q up

Importing Extop's (Joe Bank's <jbank@arsdigita.com>) enhanced ACS Developer Support package:

$ cvs import -b 1.1.3 mysite/packages/acs-developer-support Extop Extop-production-update-20010518
$ cd ~
$ cvs checkout -kk -d mysite-tmp mysite
$ cd mysite-tmp/packages/acs-developer-support
$ cvs up -kk -d -j Extop-production-update-20010518
$ cvs commit -m "Merged in changes from Extop-production-update-20010518"
$ cd /web/mysite-dev/packages/acs-developer-support
$ cvs up -d

Importing ACS-Notification tweaks from Proteome:

$ cvs import -b 1.1.5 mysite/packages/acs-notification Proteome proteome-final
$ cvs checkout -kk -d mysite-tmp mysite
$ cd mysite-tmp/packages/acs-notification
$ cvs up -kk -d -j proteome-final
$ cvs commit
$ cd /web/mysite-dev/packages/acs-notification
$ cvs up -d

Importing AOLserver nsodbc driver v. 1:

$ cvs import -b 1.1.11 aol3-src/aolserver/nsodbc AOLserver nsodbc_v1
$ cd ~/tmp
$ cvs checkout -kk aol3-src
$ cd aol3-src/aolserver/nsodbc/
$ cvs up -kk -d -j nsodbc_v1
$ cvs commit -m "Merged in nsodbc_v1 changes from www.aolserver.com."
$ cd /web/aol3-src/aolserver/nsodbc/
$ cvs up -d

Occasionally you may get just one or two extensively changed files from someone. Here's such an example. Below, I start with onle the single new version of the table-display-procs.tcl file in the current directory. Then:

$ cvs import -b 1.1.5 mysite/packages/acs-tcl/tcl Proteome proteome-final
$ cd ~ ; cvs checkout -kk -d mysite-tmp mysite
$ cd mysite-tmp/packages/acs-tcl/tcl
$ cvs up -kk -d -j proteome-final table-display-procs.tcl
$ cvs commit table-display-procs.tcl
$ cd /web/mysite-dev/packages/acs-tcl/ ; cvs up table-display-procs.tcl

Branching:

In Ron's article, he describes three methods for managing releases for client projects - One Branch, Two Branches, and Multiple Branches. Multiple Branches is the most complex of the three, so this discussion is going to assume a project using primarily the Multiple Branches method, with a bit of the Two Branches method thrown in for flavor.

Remember that in CVS, branching is really just a special kind of tagging. The (only?) difference, is that a "branch tag" is placed against a special revision number of the file - see the Cederqvist manual about this

Steps to make a new branch to Staging/Production:

  1. Make sure all fixes on the current Staging branch are committed, and merged back into the Dev trunk. See the separate section below on this.
  2. Place the root-of tag on the trunk.
  3. Place the branch tag.
  4. Checkout the new branch onto Staging.
  5. Do whatever non-CVS things you need to do - data model upgrades, restarting AOLserver, etc.
  6. Repeat the previous two steps on Production.

Steps 2 - 4 are the meat of the braching work. We give examples of them here:

Let's place the two tags. Since we're always taging the trunk with a root-of tag, we should always use this root-of tag to tell CVS where to put the branch tag. E.g.:

$ cd /web/myproject-dev
$ cvs tag root-of_release-2000-08-30
$ cvs tag -r root-of_release-2000-08-30 -b release-2000-08-30_branch

If you want to use rtag instead of tag, it's almost exactly the same, except that you also have to specify the module or repository root, in this case 'cvs-module':

$ cd /web/myproject-dev
$ cvs rtag root-of_release-2000-08-30 cvs-module
$ cvs rtag -r root-of_release-2000-08-30 -b release-2000-08-30_branch cvs-module

Now, checkout the new branch onto Staging:

$ cd /web/myproject-staging
$ cvs update -kk -d -r release-2000-08-30_branch

Finally, when you're ready to upgrade Production to the new Staging branch, you use the exact same update command:

$ cd /web/myproject
$ cvs update -kk -d -r release-2000-08-30_branch

Merging fixes from the Staging branch back to the Development trunk:

Steps to merge changes from Staging back to Dev:

  1. Make sure everything on Staging is committed.
  2. Try to make sure ever on Dev is committed too - it is less confusing this way (but see below).
  3. Tag Staging with an appropriate fix-n tag. This identifies your batch of fixes.
  4. Optionally, check out a separate copy of the Dev trunk into your own temporary directory (see below).
  5. Use cvs update or cvs checkout to merge the files from Staging to Dev.
  6. Resolve any conflicts, and commit the modified files on Dev.
  7. Do whatever non-CVS things you need to do - data model upgrades, restarting AOLserver, etc.

The basics of merging

You use the -j (join) switch with a tag name to actually make the merge happen. When you merge the first set of fixes on a Staging branch back to Dev, you only need one -j tag, to tell CVS what to join into the current working directory. E.g.:
$ cd /web/myproject-dev
$ cvs up -R -kk -j release-2000-08-08_fix-1
When you are merging your second set of fixes, you need to use two -j tags, to tell CVS "merge changes from between these two tags". If you don't, CVS will try to merge the fix-1 changes in again, causing lots of bogus conflicts. So for the second and subsequent batches of fixes, do e.g.:
$ cd /web/myproject-dev
$ cvs up -R -kk -j release-2000-08-08_fix-1 -j release-2000-08-08_fix-2

Checking out a separate working copy of Dev instead

Now, when you want to merge fixes back from the Staging branch to the Dev trunk, there will often be un-committed files on the trunk. And since the -n switch in a command like:
$ cd /web/myproject-dev
$ cvs -nq up -R -kk -j release-2000-08-08_fix-1 -j release-2000-08-08_fix-2
does not show you everything that the command would do for real without the -n switch to stop it, you can't get a list of the files that will have conflicts prior to actually doing it.

It might be nice to have a script which uses 'cvs status -v' to generate a list of all files which DON'T have the same revision number on two tags.

Now, if you don't mind not knowing what files are going to be affected by the merge, you can simply go ahead and use the above command without the -nq, resolve all conflicts, and commit.

But, if you're not sure, you probably don't want to merge directly from Staging back to Dev like the above. Instead, checkout a new working copy of the Dev trunk and do the merge there. (This means that you will be merging the fixes from Staging back into the version in the repository on Dev, not the - possibly modified - files in the Dev working copy.)

You can do the checkout of a new Dev working copy and the merge from Staging all in one step. E.g.:

$ cd ~/
$ cvs -d ls.arsdigita.com:/cvsweb co -kk -j root-of_release-2000-08-30 -j release-2000-08-30_fix-1 cvs-module > cvs-co.log

(Note that the first -j tag is probably unnecesary.)

or:
$ cd ~/
$ cvs -d ls.arsdigita.com:/cvsweb co -kk -j release-2000-08-08_fix-1 -j release-2000-08-08_fix-2 cvs-module
Then, fix any conflicts, commit as usual when you're done, and delete the extra working copy.

[It might be nice to have a script to conveniently parse out all the conflict messages from the output of the above merge command, to give a concise list of the conflicts that need to be addressed. But in practice, I just do 'cvs -nq up' to find the files with conflicts, write down the list, and then resolve them one at a time.]

Finally, remember, your fixes are now all in the CVS repository for the Dev trunk now, but you still need to check them out into the primary Dev working area. E.g., do something like:

$ cd /web/myproject-dev
$ cvs up
And if necessary, then resolve any conflicts and commit. (You should never have conflicts at this stage unless you had un-checked in changes in your Dev tree.)

Merging from Dev to Staging - without creating a new branch:

Note that it can be confusing to be doing merges back and forth from Dev to a Staging branch. I recomend attempting to avoid having to merge from Dev to Staging. Instead, try to only create a new branch from Dev to Staging, and then merge fixes back from Staging to Dev. This will make your life simpler.

Steps:

For example:
$ cd /web/myproject-dev
$ cvs tag -R release-2000-09-07
$ cd /web/myproject-staging
$ cvs -q up -d -R -kk -j release-2000-09-07
TODO: Note that there may be something wrong or non-optimal with the above, because when I did it, I got lots of totally bogus but rather complicated conflicts. E.g., doing:
$ cd /web/myproject-staging
$ find . \( -name "*.tcl" -o -name "*.sql" \) -exec grep '<<<' {} /dev/null \;
found several conflicts in each of the following files:
./www/trading/order-enter-n-4.tcl
./www/doc/sql/myproj-upgrade-db.sql
./www/doc/sql/myproj-orders_pb.sql
./www/doc/sql/myproj-orders_ph.sql
Beware of sticky tags! [TODO: Consider writing more about the dangers and uses of sticky tags.]

The -A will remove all sticky tags, but although this may fix your immediate problem, this can cause you big trouble on branch. I don't know yet how to add sticky tags back to a file, or if doing so is even plausible.

Here's another real-life example of merging from Dev to an already existing branch on Staging. In this case, I only wanted to merge over certain specific files. I don't necessarily recomend this - think about the potential problems I might have had if I later had to do a second round of merges, this time with a larger group of files that partially overlappted this set. But, it worked well enough at the time:

$ cd /web/myproject-dev
$ cvs tag release-2000-09-08 tcl/myproj-scheduled-procs.tcl www/doc/sql/myproj-orders_pb.sql

$ cd /web/myproject-staging
$ cvs -nq up -j release-2000-09-07 -j release-2000-09-08 tcl/myproj-scheduled-procs.tcl www/doc/sql/myproj-orders_pb.sql

$ cvs commit tcl/myproj-scheduled-procs.tcl www/doc/sql/myproj-orders_pb.sql

One more example. The only tricky part is to get the first tag which indicates the current state correct during the update command.

$ cd /web/myproject-dev
$ cvs -nq tag ulla-order-enter-2000-09-11 tcl/myproj-order-entry-widgets.tcl www/trading/order-enter-n.tcl

$ cd /web/myproject-staging
$ cvs up -j release-2000-09-07 -j ulla-order-enter-2000-09-11 tcl/myproj-order-entry-widgets.tcl www/trading/order-enter-n.tcl

$ cvs commit tcl/myproj-order-entry-widgets.tcl www/trading/order-enter-n.tcl

Merging only certain files across branches:

Nothing could be simpler. Just place a tag against certain files, rather than every file in the working copy or rrepository. E.g.:
$ cd /web/myproject-dev/tips/wrapper
$ cvs tag release-tips-holidays-2000-08-28 Makefile libtipswrap.so tipswrap.c tipswrap.h tipswrap.o
Then go merge the files as usual. E.g., continuing the above example (merging from Dev to Staging):
$ cd /web/myproject-staging/
$ cvs up -kk -j release-tips-holidays-2000-08-28
Finally commit, etc. as usual.

Repository Surgery - or, renaming a file:

In CVS, the only normal way to rename or move a file is to use cvs to remove and add it. This is annoying, as CVS can now longer retrieve the full history of the file for you automatically. The useful stopgap to avoid, involving "repository surgery", is discussed in this BBoard article (old broken link), which links to how to do it.

How do you move a branch tag to a different revision of a file?

Note: This section is also a BBoard article (old broken link).

After branching from our Development trunk, I discovered that I'd tagged the wrong revision of a bunch of files. Now, maybe I could have ditched the whole branch and started over, but what if I'd modified a bunch of files on the branch? Discarding the branch then could have been problematic. To fix this, what I really wanted to do was simply take those 6 or 7 files, and move the branch tag to the correct, older revision. So how to do that? Here's how:

First some background: In the project I work on, before branching we always tag the trunk with a root-of_ tag, and when we branch we attach a suffix of _branch to all branch tags. Bug-fix merges from the Staging/Production branch back to the Development trunk are given tags with _fix-n in the tag name. So in the following examples, you'll see tags that look like this:

tag name, e.g. explanation
root-of_release-2000-08-30 Root of tag, placed on the trunk, immediately before doing the branch
release-2000-08-30_branch Branch tag for release-2000-08-30
release-2000-08-30_fix-1 First batch of fixes merged back from the Staging branch to the Dev trunk
release-2000-08-30_fix-2 Second batch of fixes merged back from the Staging branch to the Dev trunk

OK, so you want to move a branch tag from one revision of a file to another. How to do it?

Short answer: 

$ cd /web/dev
$ cvs admin -Nroot-of_release-2000-08-30:1.13            www/doc/sql/stuff.sql
$ cvs tag -l -r root-of_release-2000-08-30 -b tmp_branch www/doc/sql/stuff.sql
$ cvs admin -Nrelease-2000-08-30_branch:tmp_branch       www/doc/sql/stuff.sql
$ cvs tag -d tmp_branch                                  www/doc/sql/stuff.sql
$ cd /web/staging
$ rm www/doc/sql/stuff.sql
$ cvs up www/doc/sql/stuff.sql
In the above example, to make it work for you the only things you'd need to change are the name of the file (stuff.sql), the names of the root-of_release-2000-08-30 and release-2000-08-30_branch tags, and the revision number of the file (1.13) which you want to move the branch to. The rest is boilerplate.

Before trying the above, you'll want to read up on the cvs admin -N command, but basically, it moves tags, much like mv does for unix files.

Also note that you probably don't need to use cvs admin -N at all - you should be able to achieve the same effect with multiple uses of cvs tag. But I find cvs admin -N clearer and more concise.

Long answer:

Here I'm going to walk through the testing I did in order to figure out the above Short Answer. Also, note that I've removed the irrelevent portions of the output from some commands.

First thing, is to dig around my CVS tree and find an unused file to experiment with. /web/dev/www/bannerideas/more.tcl looks perfect. Here is its initial status: 

$ cd /web/dev/www/bannerideas
$ cvs log more.tcl 
RCS file: /cvsweb/myproj/www/bannerideas/more.tcl,v
Working file: more.tcl
symbolic names:
        release-2000-08-30_branch:  1.1.1.2.0.4
        root-of_release-2000-08-30: 1.1.1.2
        acs-3-2-0: 1.1.1.2
Notice the magic branch number of 1.1.1.2.0.4 - the extra zero in the revision number is how CVS actually keeps track of branches. The cvs log commands shows this magic branch number, but cvs status -v does not - which is why I use cvs log throughout these examples.

Let's try moving the the branch back to an older revision of the file:

$ cvs admin -Nrelease-2000-08-30_branch:1.1.1.1.0.2 more.tcl
RCS file: /cvsweb/myproj/www/bannerideas/more.tcl,v
cvs [server aborted]: revision `1.1.1.1.0.2' does not exist
OK, that didn't work, because apparently we have to "create" the branch revision of this file first. If this was a file we cared about, we'd want to make sure we keep our root-of and branch tags in sync, so let's do it that way: 
$ cvs admin -Nroot-of_release-2000-08-30:1.1.1.1 more.tcl
RCS file: /cvsweb/myproj/www/bannerideas/more.tcl,v
done
$ cvs log more.tcl 
symbolic names:
      release-2000-08-30_branch:  1.1.1.2.0.4
      root-of_release-2000-08-30: 1.1.1.1
      acs-3-2-0: 1.1.1.2

$ cvs tag -l -r root-of_release-2000-08-30 -b test-cvs-2000-08-30_branch more.tcl
T more.tcl
$ cvs log more.tcl 
RCS file: /cvsweb/myproj/www/bannerideas/more.tcl,v
symbolic names:
      test-cvs-2000-08-30_branch: 1.1.1.1.0.2
      release-2000-08-30_branch:  1.1.1.2.0.4
      root-of_release-2000-08-30: 1.1.1.1
      acs-3-2-0: 1.1.1.2
OK, the revision number that we need (1.1.1.1.0.2) has been created.

Now we'll actually move the branch tag to the older revision of the file: 

$ cvs admin -Nrelease-2000-08-30_branch:1.1.1.1.0.2 more.tcl
RCS file: /cvsweb/myproj/www/bannerideas/more.tcl,v
done
$ cvs log more.tcl 
symbolic names:
      test-cvs-2000-08-30_branch: 1.1.1.1.0.2
      release-2000-08-30_branch:  1.1.1.1.0.2
      root-of_release-2000-08-30: 1.1.1.1
      acs-3-2-0: 1.1.1.2
Now we go to the Staging server where the release-2000-08-30_branch branch was previously checked out, and we see that the older 1.1.1.1 version of the file has magically become the initial revision on the branch: 
$ cd /web/staging/www/bannerideas
$ cvs status more.tcl 
===================================================================
File: more.tcl          Status: Needs Patch
   Working revision:    1.1.1.2
   Repository revision: 1.1.1.1 /cvsweb/myproj/www/bannerideas/more.tcl,v
   Sticky Tag:          release-2000-08-30_branch (branch: 1.1.1.1.2)
   Sticky Date:         (none)
   Sticky Options:      (none)
Success! At this point, we could simply delete more.tcl and then do cvs update more.tcl to get the 1.1.1.1 rev that we want. But since this is just a test file, we don't want to do that.

Instead, let's put things back the way they were before we started: 

$ cd /web/dev/www/bannerideas
$ cvs admin -Nroot-of_release-2000-08-30:1.1.1.2 more.tcl
RCS file: /cvsweb/myproj/www/bannerideas/more.tcl,v
done
$ cvs admin -Nrelease-2000-08-30_branch:1.1.1.2.0.4 more.tcl
RCS file: /cvsweb/myproj/www/bannerideas/more.tcl,v
cvs [server aborted]: revision `1.1.1.2.0.4' does not exist
Whoops! Moving the root-of tag back where it started worked, but moving the branch tag back failed. Apparently, branch revision numbers have no existance separate from tags placed against them. So, when I moved the release-2000-08-30_branch tag from rev. 1.1.1.2.0.4 to rev. 1.1.1.1.1.2, rev. 1.1.1.2.0.4 simply ceased to exist.

Probably, I should have put a placeholder tag on rev. 1.1.1.2.0.4 to mark my spot, before I moved the release-2000-08-30_branch tag. But since I didn't, let's use the "create a branch revision of this file" trick again: 

$ cvs tag -l -r root-of_release-2000-08-30 -b test-cvs-2_branch more.tcl
T more.tcl
But what revision number was acutally used? Let's find out: 
$ cvs log more.tcl 
symbolic names:
      test-cvs-2_branch:         1.1.1.2.0.4
      release-2000-08-30_branch: 1.1.1.1.0.2
Ah, we see that the original 1.1.1.2.0.4 rev. number was indeed re-used, further confirming our guess that the magic branch revision numbers have no existance independent of branch tags.

Now, we move the branch back to the revision of the file where it originally started: 

$ cvs admin -Nrelease-2000-08-30_branch:1.1.1.2.0.4 more.tcl
RCS file: /cvsweb/myproj/www/bannerideas/more.tcl,v
done
$ cvs log more.tcl 
symbolic names:
      test-cvs-2_branch:         1.1.1.2.0.4
      release-2000-08-30_branch: 1.1.1.2.0.4

Since we're done experimenting, let's get rid of the extra branch tags we created: 

$ cvs tag -d test-cvs-2_branch          more.tcl
D more.tcl
$ cvs tag -d test-cvs-2000-08-30_branch more.tcl
D more.tcl
And we can again go to the branch checked out on Staging, and confirm that everything is back the way it started: 
$ cd /web/staging/www/bannerideas
$ cvs stat more.tcl 
===================================================================
File: more.tcl          Status: Up-to-date
   Working revision:    1.1.1.2
   Repository revision: 1.1.1.2 /cvsweb/myproj/www/bannerideas/more.tcl,v
   Sticky Tag:          release-2000-08-30_branch (branch: 1.1.1.2.4)
   Sticky Date:         (none)
   Sticky Options:      (none)

atp@piskorski.com
 		$Id: cvs-conventions.html,v 1.9 2005/09/09 21:44:45 andy Exp $