head     1.11;
branch   ;
access   ;
symbols  apple:1.1;
locks    ; strict;
comment  @# @;


1.11
date     94.04.11.00.02.05;  author romig;  state Exp;
branches ;
next     1.10;

1.10
date     93.06.12.01.01.31;  author romig;  state Exp;
branches ;
next     1.9;

1.9
date     93.01.07.13.15.40;  author romig;  state Exp;
branches ;
next     1.8;

1.8
date     92.04.21.22.37.05;  author romig;  state Exp;
branches ;
next     1.7;

1.7
date     92.04.21.12.01.00;  author romig;  state Exp;
branches ;
next     1.6;

1.6
date     92.04.20.16.37.47;  author romig;  state Exp;
branches ;
next     1.5;

1.5
date     92.04.17.11.01.22;  author romig;  state Exp;
branches ;
next     1.4;

1.4
date     92.03.18.16.53.07;  author romig;  state Exp;
branches ;
next     1.3;

1.3
date     91.11.16.11.45.34;  author romig;  state Exp;
branches ;
next     1.2;

1.2
date     91.05.31.12.03.34;  author romig;  state Exp;
branches ;
next     1.1;

1.1
date     90.12.17.11.47.29;  author romig;  state Exp;
branches ;
next     ;


desc
@README for newcomers.
@


1.11
log
@updates for 2.6 release
@
text
@BRIEF DESCRIPTION

    The OSU CIS backup system consists of a set of Perl scripts that
    strive to make it easy and safe to manage backups and restores for
    a number of hosts.  The main design constraints were that it
    should be easy to use, robust in handling errors, complementary to
    human skills and shortcomings, and that it should use native
    backup programs.

    The current version supports full and incremental backups on
    multiple hosts, where backups of many file systems from different
    hosts can be put onto one tape.  It maintains a tape library with
    a free tape pool.  There are safety checks that verify that you
    are using the right tape in the right drive, that backups haven't
    been skipped, etc.  There is support for multiple backup chains.
    There is offsite tape storage support.  Restores are easier since
    the system knows what backups are on which files on which tapes,
    and it will ask you to mount tape X and skip to the correct file
    for you.  You can backup to disk or tape or both.  There's an
    event log so you know who has done what.  It comes with support
    for dump and GNU tar, and its easy to add support for other
    archive programs, like cpio or others.

    This version of the system has been ported to and tested on HP/UX
    8.05, 8.0.7, SunOS 4.1.1, 4.1.3, Solaris 2.1, BBN's Mach 2.5,
    Encore's Mach 1.0, and Ultrix 4.2a.  It should be easy to use this
    on other UNIX platforms, provided that you have installed Perl.  We
    are using Perl version 4.36.

    If you've been using previous versions of this backup system, see
    the *IMPORTANT NOTE* below.

FUN STATS

    We've done at least 14,000 backup runs so far, consisting of
    314,000 individual backup (most done with dump).  We backup 40
    hosts with 6 Exabyte 8200 tape drives in two hours, twice a day.

WHAT'S NEW

    This is version 2.6.  Main changes since 2.5: tape type support,
    full support for online backups, wrote a reference manual, and made
    miscellaneous changes that make the system more flexible than
    before.

MISSING PIECES

    We aren't doing anything about multiple tape support or backup
    verification yet.  For multiple tape support, I'll probably
    provide a choice between tape rollover (several mounted tapes,
    roll to next tape as needed - good for unattended backups) and
    asking the user for help.

    It isn't clear how we should deal with tape rollover between tapes
    on several hosts - neither dump nor GNU tar can easily deal with
    moving from one rmt host to another in mid-archive, though it
    would be simple to set up the backup software to allow one to
    shift from host to host between file systems.  

    There's no support for stackers or jukeboxes yet.  

    See the section on future plans in the reference manual for a longer
    discussion of these topics.

GETTING STARTED

    Print a copy of the manual (doc/backup.ps), and follow the
    instructions in the Installation section.  You should also glance
    over the man pages and the Makefile.  If you didn't know, there's
    a mailing list for folks using this stuff, where you can ask
    questions, discuss problems, and take part in designing future
    versions.  Send mail to backup-request@@cis.ohio-state.edu for
    details.  There's also a backup-announce list, which receives
    ONLY announcements about new versions - send mail to
    backup-announce-request to join that.

    *IMPORTANT NOTE* There are many small but important changes since
    version 2.5.  In particular, there are new options in /etc/backups,
    the format of the backup.config file has changed somewhat,
    checkinout has been renamed to move-tape, and there are many new
    options in local.defs and global.defs.  READ the Changes file and
    review your config files carefully!  Please also note that I've
    changed the default names of the database files - you can either
    rename your files to match, or change the definitions in
    global.defs.  I've also renamed the sequence files as well -
    clean-db will automatically deal with this, but you might want to
    rename your *-id* files anyway.  The new names are the same as their
    corresponding database names, with the extension ".seq".
 
WHAT'S WHERE:

    Changes		Describes the change history of the software
    Makefile		Makefile to make it easy to print stuff.
    README		This file.
    Wishlist		List of things to do, bugs to fix, and so on.
    cis/		Misc scripts that you might find interesting.
    doc/		Various documents, including the LISA paper,
			The LISA slides and the backup manual.
    graph/		The optional graph making facilities.
    man/		The man pages.
    misc/		Assorted things you might be missing
			(yagrip.pl). 
    samples/		Sample config files.
    src/		The sources.
    test/		A bunch of test tools.

SPECIAL THANKS

    To the (brave :-) souls who are using this software, and who have
    made so many interesting and useful suggestions.  I'd specifically
    like to thank John Rouillard for his patience, suggestions and bug
    reports, and Sxren Michael Roug for the restore-gtar script and the 
    idea for the pmt script (which has now been rewritten in Perl).

--- Steve Romig, April 10, 1994
@


1.10
log
@various changes for 2.6pre
@
text
@d35 3
a37 7
    We've done at least 7208 backup runs so far, consisting of 81481
    individual backup (most done with dump).  I estimate the total size
    of these backups to be about 3.5 terabytes.  Our collection of
    almost 700 tapes currently contains about 4819 backups from 46
    hosts, with a total size of about 282 gigabytes.  We backup 40 hosts
    with 6 Exabyte 8200 tape drives twice a day, in about 2 hours for
    each run.
d115 1
a115 1
--- Steve Romig, January 7, 1993
@


1.9
log
@final version for 2.6beta
@
text
@a0 2
This is the latest OSU/CIS backup system, such as it is.  

d6 1
a6 1
    should be easy to use, robust in handling errors, complimentary to
d25 4
a28 2
    8.0.7, SunOS 4.1.1, Pyramid OSx 4.4, BBN's Mach 2.5, Encore's Mach
    release, and Ultrix 4.2a.
d45 1
a45 1
    This is version 2.6beta.  Main changes since 2.5: tape type support,
d48 1
a48 2
    before.  Main changes since 2.6alpha: many important bug fixes,
    added 'show scheduled'.  See Changes for details.
d74 6
a79 2
    a mailing list for folks using this stuff.  Send mail to
    backup-request@@cis.ohio-state.edu for details.
d81 12
a92 13
    *IMPORTANT NOTE* There are many small but important changes from
    previous versions.  In particular, there are new options in
    /etc/backups, the format of the backup.config file has changed
    somewhat, checkinout has been renamed to move-tape, and there are
    many new options in local.defs and global.defs.  READ the Changes
    file and review your config files carefully!  

    Please also note that I've changed the default names of the database
    files - you can either rename your files to match, or change the
    definitions in global.defs.  I've also renamed the sequence files as
    well - clean-db will automatically deal with this, but you might
    want to rename your *-id* files anyway.  The new names are the same
    as their corresponding database names, with the extension ".seq".
@


1.8
log
@added note about Ultrix.
@
text
@a25 6
    No, it still doesn't allow backups to span multiple tapes.  Sorry.
    If you are in a situation where you have file systems that take more
    than one tape to backup, then this system is NOT going to help you.
    It was designed for people who want to put multiple backups on one
    tape, rather than people who need multiple tapes for one backup.
    
d27 2
a28 2
    8.0, SunOS 4.0.3 and 4.1.1, Pyramid OSx 4.4, BBN's Mach 2.5 and
    Encore's Mach release.
d35 7
a41 5
    We've done at least 3238 backup runs so far, consisting of 38088
    individual backups (all done with dump).  I estimate the total size
    of these backups to be about 1.59 terabytes.  Our collection of 500
    tapes currently contains 4100 backups from about 30 different hosts,
    with a total size of about 171 gigabytes.  
d45 5
a49 5
    This is version 2.5.  Main additions are tape type support, full
    support for online backups, and misc. changes that make it much
    more flexible than before.  And I made lots of bug fixes.  See
    Changes for details.  Oh, there's a reference manual now - check
    out the doc subdirectory.
d57 1
a57 1
    asking the user.
d63 1
a63 1
    shift from host to host between file systems.  Argh.
d65 5
d78 6
a83 8
    *IMPORTANT NOTE 1* If you've been using previous versions of this
    backup system, you will need to convert your databases to the new
    format.  The changes are: tapes now have "type" and "format"
    fields, and the tape IDs have been changed.  If you don't run the
    conversion script, I have no idea what will happen (though I know
    for sure that clean-db will wipe out the database first chance it
    gets).  See convert/README for more details.  It should be
    painless. 
d85 7
a91 10
    *IMPORTANT NOTE 2* If you have tape devices on hosts that are
    running Ultrix, the backup and recovery stuff won't work as nicely
    as it might since "mt status" on Ultrix systems doesn't report on
    the file position.  Where possible, use tape devices on hosts whose
    "mt status" commands give tape position information.

    You will probably also need to update your /etc/backups files on 
    each of your hosts, since there are some subtle changes there 
    also.  See the backups(5) man page.

a98 2
    convert/		Conversion stuff to convert old style database
			into new style.
d103 3
a105 1
    samples/		Misc samples.
d109 1
a109 1
--- Steve Romig, April 17, 1992
d111 7
@


1.7
log
@Misc fixes
@
text
@d77 1
a77 1
    *IMPORTANT NOTE* If you've been using previous versions of this
d85 6
@


1.6
log
@Added interesting stats.
@
text
@d27 4
d42 4
a45 4
    individual backups (all with dump).  I estimate the total size of
    these backups to be about 1.59 terabytes.  Our collection of 500
    tapes currently contains 4100 backups on about 30 different hosts,
    with a total size of about 171 gigabytes.
@


1.5
log
@Misc updates for 2.5 release
@
text
@d35 8
@


1.4
log
@changes for 2.5
@
text
@a2 6
This is version 2.5.  Main additions are tape type support, full
support for online backups, and misc. changes that make it much more
flexible than before.  And I made lots of bug fixes.  See Changes for
details.  Oh, there's a reference manual now - check out the doc
subdirectory. 

d12 13
a24 5
    Easy to use: the programs that are used most, all-backups and
    getback, are user friendly in that they convey a consistent and
    simple interface to handling backups on many and diverse systems.
    If you fail to provide necessary information on the command line,
    they'll ask for what they want.
a25 36
    Robust: the system tries really, really hard to deal with errors.
    Many of the routines have regression test suites that help to
    verify that the components used to build this thing are working
    properly.  Programs that might damage or change the tapes or the
    online database check to ensure that the right tapes are being
    used in the right drives.  There are all sorts of internal
    consistency checks intended to find bugs and problems in the
    database.  It ain't perfect, but its better than a crufty set of
    shell scripts.

    Complimentary: people are bad at detail work and at doing boring,
    repetitive tasks.  Tasks like free tape management and database
    management are better left to automated tools.  It is easy for the
    backup system to tell when a backup is overdue, so there's a .B
    check-backups routine to bitch about overdue backups.  It is very
    hard to write a program that could easily schedule backups and
    take into account tape drive use, drive status, operator schedules
    and so on, so that's left to the users.

    Native programs: there's a plethora of file system variations.
    You've got short and long name file systems, BSD Fast File
    Systems, file systems that support strange objects like context
    dependent files or access control lists, strange proprietary
    database file systems that have their own wierd backup programs,
    and so on.  This backup system allows you to use the most
    appropriate archiving program to do your backups.  You should be
    able to integrate almost anything into it.  Its designed for
    dump-style incremental backups, but could easily be used with
    programs that only support full saves.  It makes a lot of sense to
    separate the archiving/retrieval format and programs from the
    incremental backup superstructure.  That's what these tools do.

    This version supports incremental backups in multiple chains on
    multiple hosts, tape types, fairly easy restores, backups to tape
    or disk or both, parallel backups, event logs, and more.  

d28 4
d35 1
a35 1
GNU TAR
d37 5
a41 5
    The initial system was designed to use GNU tar, since when I
    started work on things I was dismayed at how unsafe BSD dump was
    on active file systems.  As it turns out, some versions of GNU
    tar have serious shortcomings that (in my opinion) make them
    unacceptable for serious backups.  
a42 41
    In particular:

    1. versions before 1.10 didn't always deal with sparse files
       correctly.  Saving/restoring single sparse files works fine,
       the problems were demonstrated when saving/restoring file
       systems with many sparse files in them.  I never found the
       problem, I just rewrote the code that deals with sparse files.
       Far as I've tested it, 1.10 seems to work correctly.

    2. versions before 1.10 also had a bug wherein using -G to extract
       files from an incremental archive would delete any directories
       that had been renamed between the full save and that
       incremental, due to the way that it dealt with name/inode
       mappings.  This seems to have been fixed.

    3. There used to be strange problems wherein hardlinks would
       sometimes "split off" to become their own files after
       extracting them from tar archives.  I haven't tested this
       lately, I don't know whether its still a problem or not.

    Far as I know, GNU tar 1.10 is pretty solid.  They've fixed the
    bugs that I knew about, and I haven't heard any screaming from the
    people using it.

    *NOTE* that since I use dump for backups, that is more thoroughly
    tested and more featureful.  In particular, although you can use
    getback's pick mode to see the list of backups if you use GNU tar,
    you will have to run your restores "by hand" with "mt fsf" and
    "tar".

DUMP

    Be careful when dumping live (active) file systems - dump was
    designed to work correctly on inactive file systems ONLY, and you
    may have "imperfect" results if you use dump on an active file
    systems.  If you don't know for certain whether your version of
    dump has the Purdue active file system mods added, then I wouldn't
    even think about using it on an active file system.  The
    rescheduling and "backup-while-U-boot" options in all-backups make
    backups on inactive file systems fairly painless.

a56 4
FUTURE WORK

    Described in more detail in Wishlist.

a58 7
    If you will be using this on hosts with different byte orders, and
    want to share the backup database between them, you need to
    construct a version of Perl that uses a DBM look-alike library
    that is byte order independent.  Turns out that the BSD db library
    works well.  You can find a copy plus instructions on how to patch
    it into Perl in ftp.cis.ohio-state.edu:~ftp/pub/backup/db.tar.Z.

a64 2
    The manual contains installation information.

d74 3
a76 1
What's where:
d78 2
a83 1
    backup.config	Sample backup config file.
d89 1
a89 1
    exclude		Sample exclude file (for GNU tar backups).
d91 1
d95 1
a95 1
--- Steve Romig, August 26, 1991
@


1.3
log
@misc updates for 2.2
@
text
@d3 6
d11 6
a16 5
    This backup system is implemented as a set of Perl scripts.  The
    current version (2.2) can use either dump or GNU tar as the
    archive method, selectable on a per filesystem basis.  It should
    work fine with any sort of tape device, although I've only tested
    it extensively with Exabyte 8200 tape drives.
d18 5
a22 7
    Yes, I finally consider this stuff to be production quality.  I've
    been doing lots of backups and restores with it, and it seems to
    work reliably and well.  We're finally switching to full scale
    production use of it this fall.  We've been using it in parallel
    with our old system this summer, and we've been using it almost
    exclusively for restores for a while now.  There don't appear to
    be any major problems with it so far.
d24 9
a32 7
    Although I've tried to design it to be portable and generic,
    you'll probably have to tweak it to get it to suit your
    environment.  I don't think I've run it on anything other than
    Pyramids and Suns, and all the recent testing has been on Suns, so
    it may not work on other platforms right off the bat.  I know that
    one would have to tweak the way it reads 'df' output to get it to
    work on HPs (if you want to use parallel backup mode).
d34 31
d93 10
a104 4
    The current system allows you to use either dump or GNU tar as the
    archive method.  Selectable on a per-filesystem basis.  One could
    "easily" add others, such as cpio - see the manual for details.

a113 6
Restores

    I've finally finished the restore software.  I think it's way
    cool, I hope you like it too.  See the man page for 'getback' for
    details.

d130 1
a130 1
    Described in more detail in Notes.
d134 8
a141 1
    Print a copy of the manual (Doc/backup.ps), and follow the
d147 11
a160 2
    Docs/		Various documents, including the LISA paper,
			The LISA slides and the backup manual.
a161 2
    Notes		List of things to do, bugs to fix, and so on.
			Very terse and cryptic.
d163 1
d165 5
d172 2
a173 1
    src/		The sources
@


1.2
log
@Updated to reflexct current status.
@
text
@d6 1
a6 1
    current version (2.1) can use either dump or GNU tar as the
d9 1
a9 1
    it with Exabyte 8200 tape drives.
d11 7
a17 3
    Yes, I finally consider this stuff to be production quality, or
    nearly so.  I've been doing lots of backups and restores with it,
    and it seems to work reliably and well.
d25 1
a25 1
    work on HPs.
d31 3
a33 14
    on active file systems.  As it turns out, GNU tar has serious
    shortcomings that (in my opinion) make it unacceptable for serious
    backups.  In particular, it doesn't correctly deal with the
    mapping between inodes and file names, which means that it will
    occasionally "split off" files that should be hard links to the
    same inode (so that you end up with 2 separate inodes with the
    same contents, rather than 1).  There's also a serious bug when
    you use the -G switch with -x to extract files from an incremental
    archive - any directories that were renamed between the full save
    and the incremental will *not* appear at all after the incremental
    restore is done. They are still on the full save tape, however,
    and any files that got written to the incremental will be restored
    (I think).  Version 1.09 still didn't deal with sparsely allocated
    files correctly in a reliable fashion. 
d35 20
d58 2
a59 2
    archive method.  One could "easily" add others, such as cpio - ask
    me for details on how to do that.  It really wouldn't be hard.
d72 3
a74 2
    I've finally finished the restore software.  Let me know what
    you think...I probably ought to rename it...
d84 6
d92 1
a92 3
    Described in more detail in Notes - we need a better locking
    system, and it would be nice to have a database editor or some
    sort. 
d96 5
a100 4
    Read INSTALL, glance over the man pages and the Makefile, and set
    to work.  HOWTO describes how to actually do backups.  If you
    didn't know, there's a mailing list for folks using this stuff.
    Send mail to backup-request@@cis.ohio-state.edu for details.
d102 16
a117 1
--- Steve Romig, May 31, 1991
@


1.1
log
@Initial revision
@
text
@d6 4
a9 3
    current version uses GNU tar as the archive method, and should
    work on any sort of tape-like storage device (though its only been
    tested on Exabyte tape drives).  
d11 3
a13 7
    Although I've had great success in doing backups and (and some
    restores, see below) using this software, I don't consider it to
    be production quality yet.  Also, although I've tried to design it
    to be portable and generic, you'll probably have to tweak it to
    get it to suit your environment.  I don't think I've run it on
    anything other than Pyramids and Suns, and all the recent testing
    has been on Suns, so it may not work on other platforms.
d15 8
d25 16
a40 15
    The current system uses GNU tar, since when I started work on
    things I was dismayed at how unsafe BSD dump was on active file
    systems.  As it turns out, GNU tar has serious shortcomings that
    (in my opinion) make it unacceptable for serious backups.  In
    particular, it doesn't correctly deal with the mapping between
    inodes and file names, which means that it will occasionally
    "split off" files that should be hard links to the same inode (so
    that you end up with 2 separate inodes with the same contents,
    rather than 1).  There's also a serious bug when you use the -G
    switch with -x to extract files from an incremental archive - any
    directories that were renamed between the full save and the
    incremental will *not* appear at all after the incremental restore
    is done. They are still on the full save tape, however, and any
    files that got written to the incremental will be restored (I
    think).
d42 1
a42 4
    Anyway, I wouldn't recommend using GNU tar for anything but
    testing at this point, which really shoots down this version of
    the backup stuff, since all it uses is GNU tar.  It isn't hard to
    add support for dump - I'm working on that now.
d44 3
a46 1
MISSING PIECES
d48 8
a55 5
    Aside from adding support for BSD dump, I need to finish the
    restore software.  The Notes file describes somewhat how things
    should look when done - turns out that using BSD restore will
    vastly simplify things in some respects (interactive restores are
    easy, I won't have to write much code for them).
d57 1
a57 3
    This version is missing the "backup during reboot" feature that I
    used to have (and that the paper talks about).  I should be adding
    this again soon.
d59 7
a65 2
    We also aren't doing anything about multiple tape support or
    backup verification yet.  For multiple tape support, I'll probably
d83 1
a83 1
--- Steve Romig, December 12, 1990
@
