TSM & Retention Policies

If you've managed a backup solution before (DPM, BackupExec, UVB, Veamm, etc.), then retention was probably set by answering the simple question of "How long do I want to keep this stuff for?" Most of the time, retention is a simple setting. Set it & forget it right? in Tivoli Storage Manager, or TSM, it's a bit more complex than that.

TSM has versions specifically for other application-specific types of backups such as those for mail servers, databases or VMs but we're just talking about files here. TSM does backups and archives of files. Archives are full backups on a less than frequent schedule (think monthly) while backups are more frequent and do incremental backup jobs (eg, what's changed since the last backup?) on a more frequent schedule (think daily or weekly). An archive just keeps files for a specified number of days (like most other backup tools I mentioned earlier), but backups are handled a bit differently. 

When TSM backs up a file in a backup job, it backs up only the files that have been changed since the last backup. This file, the most recent copy, is regarded as the active file. Let's say the file gets updated and another backup is ran. This more recent backup sees the newly updated copy of our file and makes it the active file. The old active file is now regarded as the inactive file. Your retention settings are set by specifying values for 4 rules which now apply to inactive files. An inactive file will be expired (eg, removed from the TSM database, unavailable to restore*) based on which of the rules are met first. Remember that! Those 4 rules are:

• Versions Existing: The number of historical versions of each file to retain while the file still exists on the client filesystem. (Often short handed to VerExist)
• Versions Deleted: The number of historical versions of each file to retain once the file has been deleted from the client filesystem. (Often short handed to VerDelete)
• Retain Extra: The number of days to retain the historical versions of each file. After this period all older versions will be discarded from the database (more on that later). (Often short handed to RetExtra)
• Retain Only: The number of days to retain the last historical version of each file. (This rule only applies after the file has been deleted from the client filesystem AND the Retain Extra rule value has discarded all other versions). (Often short handed to RetOnly)

These rules only apply to inactive files so don't worry about your active files. Let's work through an example. Let's say you want to keep your backups for 2 weeks, or 14 days. In this case, you can set VerExist & VerDelete to no limit as we care about days rather than versions here. If you have one of these files that changes more often than others, having a lower limit on VerExist or VerDelete than RetExtra or RetOnly rules could cause certain files to expire before we want them to. Not good. At that point, you wouldn't know you didn't have a file until it was too late. 

*It is possible to get an expired file back. An expired file is just a file who's reference in the TSM database has been removed. If you set the reusedelay setting on your TSM storage pool, you can create a multi-day buffer between when an inactive file has expired & when that empty space is reclaimed by TSM. With a healthy reusedelay setting set (a few days or so) and a second TSM instance to restore an older database backup to and a copy of the TSM volume history file, you can get an expired file back by restoring the backed up TSM database to a secondary TSM instance,  & pulling the file you need before it gets actually deleted. In most cases it's far easier to put better thought and planning into your TSM retention policy than it is to pull expired files out of the crypt.