CedarBackup3.actions package¶
Cedar Backup actions.
This package code related to the offical Cedar Backup actions (collect, stage, store, purge, rebuild, and validate).
The action modules consist of mostly “glue” code that uses other lower-level functionality to actually implement a backup. There is one module for each high-level backup action, plus a module that provides shared constants.
All of the public action function implement the Cedar Backup Extension Architecture Interface, i.e. the same interface that extensions implement.
author: | Kenneth J. Pronovici <pronovic@ieee.org> |
---|
Submodules¶
CedarBackup3.actions.collect module¶
Implements the standard ‘collect’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>
-
CedarBackup3.actions.collect.
executeCollect
(configPath, options, config)[source]¶ Executes the collect backup action.
Note: When the collect action is complete, we will write a collect indicator to the collect directory, so it’s obvious that the collect action has completed. The stage process uses this indicator to decide whether a peer is ready to be staged.
Parameters: - configPath (String representing a path on disk) – Path to configuration file on disk
- options (Options object) – Program command-line options
- config (Config object) – Program configuration
Raises: ValueError
– Under many generic error conditionsTarError
– If there is a problem creating a tar file
CedarBackup3.actions.constants module¶
Provides common constants used by standard actions. :author: Kenneth J. Pronovici <pronovic@ieee.org>
CedarBackup3.actions.initialize module¶
Implements the standard ‘initialize’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>
-
CedarBackup3.actions.initialize.
executeInitialize
(configPath, options, config)[source]¶ Executes the initialize action.
The initialize action initializes the media currently in the writer device so that Cedar Backup can recognize it later. This is an optional step; it’s only required if checkMedia is set on the store configuration.
Parameters: - configPath (String representing a path on disk) – Path to configuration file on disk
- options (Options object) – Program command-line options
- config (Config object) – Program configuration
CedarBackup3.actions.purge module¶
Implements the standard ‘purge’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>
-
CedarBackup3.actions.purge.
executePurge
(configPath, options, config)[source]¶ Executes the purge backup action.
For each configured directory, we create a purge item list, remove from the list anything that’s younger than the configured retain days value, and then purge from the filesystem what’s left.
Parameters: - configPath (String representing a path on disk) – Path to configuration file on disk
- options (Options object) – Program command-line options
- config (Config object) – Program configuration
Raises: ValueError
– Under many generic error conditions
CedarBackup3.actions.rebuild module¶
Implements the standard ‘rebuild’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>
-
CedarBackup3.actions.rebuild.
executeRebuild
(configPath, options, config)[source]¶ Executes the rebuild backup action.
This function exists mainly to recreate a disc that has been “trashed” due to media or hardware problems. Note that the “stage complete” indicator isn’t checked for this action.
Note that the rebuild action and the store action are very similar. The main difference is that while store only stores a single day’s staging directory, the rebuild action operates on multiple staging directories.
Parameters: - configPath (String representing a path on disk) – Path to configuration file on disk
- options (Options object) – Program command-line options
- config (Config object) – Program configuration
Raises: ValueError
– Under many generic error conditionsIOError
– If there are problems reading or writing files
CedarBackup3.actions.stage module¶
Implements the standard ‘stage’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>
-
CedarBackup3.actions.stage.
executeStage
(configPath, options, config)[source]¶ Executes the stage backup action.
Note: The daily directory is derived once and then we stick with it, just in case a backup happens to span midnite.
Note: As portions of the stage action is complete, we will write various indicator files so that it’s obvious what actions have been completed. Each peer gets a stage indicator in its collect directory, and then the master gets a stage indicator in its daily staging directory. The store process uses the master’s stage indicator to decide whether a directory is ready to be stored. Currently, nothing uses the indicator at each peer, and it exists for reference only.
Parameters: - configPath (String representing a path on disk) – Path to configuration file on disk
- options (Options object) – Program command-line options
- config (Config object) – Program configuration
Raises: ValueError
– Under many generic error conditionsIOError
– If there are problems reading or writing files
CedarBackup3.actions.store module¶
Implements the standard ‘store’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org> :author: Dmitry Rutsky <rutsky@inbox.ru>
-
CedarBackup3.actions.store.
consistencyCheck
(config, stagingDirs)[source]¶ Runs a consistency check against media in the backup device.
It seems that sometimes, it’s possible to create a corrupted multisession disc (i.e. one that cannot be read) although no errors were encountered while writing the disc. This consistency check makes sure that the data read from disc matches the data that was used to create the disc.
The function mounts the device at a temporary mount point in the working directory, and then compares the indicated staging directories in the staging directory and on the media. The comparison is done via functionality in
filesystem.py
.If no exceptions are thrown, there were no problems with the consistency check. A positive confirmation of “no problems” is also written to the log with
info
priority.@warning: The implementation of this function is very UNIX-specific.
Parameters: - config – Config object
- stagingDirs – Dictionary mapping directory path to date suffix
Raises: ValueError
– If the two directories are not equivalentIOError
– If there is a problem working with the media
-
CedarBackup3.actions.store.
executeStore
(configPath, options, config)[source]¶ Executes the store backup action.
Note: The rebuild action and the store action are very similar. The main difference is that while store only stores a single day’s staging directory, the rebuild action operates on multiple staging directories.
Note: When the store action is complete, we will write a store indicator to the daily staging directory we used, so it’s obvious that the store action has completed.
Parameters: - configPath (String representing a path on disk) – Path to configuration file on disk
- options (Options object) – Program command-line options
- config (Config object) – Program configuration
Raises: ValueError
– Under many generic error conditionsIOError
– If there are problems reading or writing files
-
CedarBackup3.actions.store.
writeImage
(config, newDisc, stagingDirs)[source]¶ Builds and writes an ISO image containing the indicated stage directories.
The generated image will contain each of the staging directories listed in
stagingDirs
. The directories will be placed into the image at the root by date, so staging directory/opt/stage/2005/02/10
will be placed into the disc at/2005/02/10
.Note: This function is implemented in terms of
writeImageBlankSafe
. ThenewDisc
flag is passed in for bothrebuildMedia
andtodayIsStart
.Parameters: - config – Config object
- newDisc – Indicates whether the disc should be re-initialized
- stagingDirs – Dictionary mapping directory path to date suffix
Raises: ValueError
– Under many generic error conditionsIOError
– If there is a problem writing the image to disc
-
CedarBackup3.actions.store.
writeImageBlankSafe
(config, rebuildMedia, todayIsStart, blankBehavior, stagingDirs)[source]¶ Builds and writes an ISO image containing the indicated stage directories.
The generated image will contain each of the staging directories listed in
stagingDirs
. The directories will be placed into the image at the root by date, so staging directory/opt/stage/2005/02/10
will be placed into the disc at/2005/02/10
. The media will always be written with a media label specific to Cedar Backup.This function is similar to
writeImage
, but tries to implement a smarter blanking strategy.First, the media is always blanked if the
rebuildMedia
flag is true. Then, ifrebuildMedia
is false, blanking behavior andtodayIsStart
come into effect:If no blanking behavior is specified, and it is the start of the week, the disc will be blanked If blanking behavior is specified, and either the blank mode is "daily" or the blank mode is "weekly" and it is the start of the week, then the disc will be blanked if it looks like the weekly backup will not fit onto the media. Otherwise, the disc will not be blanked
How do we decide whether the weekly backup will fit onto the media? That is what the blanking factor is used for. The following formula is used:
will backup fit? = (bytes available / (1 + bytes required) <= blankFactor
The blanking factor will vary from setup to setup, and will probably require some experimentation to get it right.
Parameters: - config – Config object
- rebuildMedia – Indicates whether media should be rebuilt
- todayIsStart – Indicates whether today is the starting day of the week
- blankBehavior – Blank behavior from configuration, or
None
to use default behavior - stagingDirs – Dictionary mapping directory path to date suffix
Raises: ValueError
– Under many generic error conditionsIOError
– If there is a problem writing the image to disc
-
CedarBackup3.actions.store.
writeStoreIndicator
(config, stagingDirs)[source]¶ Writes a store indicator file into staging directories.
The store indicator is written into each of the staging directories when either a store or rebuild action has written the staging directory to disc.
Parameters: - config – Config object
- stagingDirs – Dictionary mapping directory path to date suffix
CedarBackup3.actions.util module¶
Implements action-related utilities :author: Kenneth J. Pronovici <pronovic@ieee.org>
-
CedarBackup3.actions.util.
buildMediaLabel
()[source]¶ Builds a media label to be used on Cedar Backup media. :returns: Media label as a string
-
CedarBackup3.actions.util.
checkMediaState
(storeConfig)[source]¶ Checks state of the media in the backup device to confirm whether it has been initialized for use with Cedar Backup.
We can tell whether the media has been initialized by looking at its media label. If the media label starts with MEDIA_LABEL_PREFIX, then it has been initialized.
The check varies depending on whether the media is rewritable or not. For non-rewritable media, we also accept a
None
media label, since this kind of media cannot safely be initialized.Parameters: storeConfig – Store configuration Raises: ValueError
– If media is not initialized
-
CedarBackup3.actions.util.
createWriter
(config)[source]¶ Creates a writer object based on current configuration.
This function creates and returns a writer based on configuration. This is done to abstract action functionality from knowing what kind of writer is in use. Since all writers implement the same interface, there’s no need for actions to care which one they’re working with.
Currently, the
cdwriter
anddvdwriter
device types are allowed. An exception will be raised if any other device type is used.This function also checks to make sure that the device isn’t mounted before creating a writer object for it. Experience shows that sometimes if the device is mounted, we have problems with the backup. We may as well do the check here first, before instantiating the writer.
Parameters: config – Config object
Returns: Writer that can be used to write a directory to some media
Raises: ValueError
– If there is a problem getting the writerIOError
– If there is a problem creating the writer object
-
CedarBackup3.actions.util.
findDailyDirs
(stagingDir, indicatorFile)[source]¶ Returns a list of all daily staging directories that do not contain the indicated indicator file.
Parameters: stagingDir – Configured staging directory (config.targetDir) Returns: List of absolute paths to daily staging directories
-
CedarBackup3.actions.util.
getBackupFiles
(targetDir)[source]¶ Gets a list of backup files in a target directory.
Files that match INDICATOR_PATTERN (i.e.
"cback.store"
,"cback.stage"
, etc.) are assumed to be indicator files and are ignored.Parameters: targetDir – Directory to look in Returns: List of backup files in the directory Raises: ValueError
– If the target directory does not exist
-
CedarBackup3.actions.util.
initializeMediaState
(config)[source]¶ Initializes state of the media in the backup device so Cedar Backup can recognize it.
This is done by writing an mostly-empty image (it contains a “Cedar Backup” directory) to the media with a known media label.
Note: Only rewritable media (CD-RW, DVD+RW) can be initialized. It doesn’t make any sense to initialize media that cannot be rewritten (CD-R, DVD+R), since Cedar Backup would then not be able to use that media for a backup.
Parameters: config – Cedar Backup configuration
Raises: ValueError
– If media could not be initializedValueError
– If the configured media type is not rewritable
-
CedarBackup3.actions.util.
writeIndicatorFile
(targetDir, indicatorFile, backupUser, backupGroup)[source]¶ Writes an indicator file into a target directory. :param targetDir: Target directory in which to write indicator :param indicatorFile: Name of the indicator file :param backupUser: User that indicator file should be owned by :param backupGroup: Group that indicator file should be owned by
Raises: IOException
– If there is a problem writing the indicator file
CedarBackup3.actions.validate module¶
Implements the standard ‘validate’ action. :author: Kenneth J. Pronovici <pronovic@ieee.org>
-
CedarBackup3.actions.validate.
executeValidate
(configPath, options, config)[source]¶ Executes the validate action.
This action validates each of the individual sections in the config file. This is a “runtime” validation. The config file itself is already valid in a structural sense, so what we check here that is that we can actually use the configuration without any problems.
There’s a separate validation function for each of the configuration sections. Each validation function returns a true/false indication for whether configuration was valid, and then logs any configuration problems it finds. This way, one pass over configuration indicates most or all of the obvious problems, rather than finding just one problem at a time.
Any reported problems will be logged at the ERROR level normally, or at the INFO level if the quiet flag is enabled.
Parameters: - configPath (String representing a path on disk) – Path to configuration file on disk
- options (Options object) – Program command-line options
- config (Config object) – Program configuration
Raises: ValueError
– If some configuration value is invalid