CedarBackup3.tools package¶
Official Cedar Backup Tools
This package provides official Cedar Backup tools. Tools are things that feel a little like extensions, but don’t fit the normal mold of extensions. For instance, they might not be intended to run from cron, or might need to interact dynamically with the user (i.e. accept user input).
Tools are usually scripts that are run directly from the command line, just
like the main cback3
script. Like the cback3
script, the majority of a
tool is implemented in a .py module, and then the script just invokes the
module’s cli()
function. The actual scripts for tools are distributed in
the util/ directory.
author: | Kenneth J. Pronovici <pronovic@ieee.org> |
---|
Submodules¶
CedarBackup3.tools.amazons3 module¶
Synchonizes a local directory with an Amazon S3 bucket.
No configuration is required; all necessary information is taken from the command-line. The only thing configuration would help with is the path resolver interface, and it doesn’t seem worth it to require configuration just to get that.
author: | Kenneth J. Pronovici <pronovic@ieee.org> |
---|
-
class
CedarBackup3.tools.amazons3.
Options
(argumentList=None, argumentString=None, validate=True)[source]¶ Bases:
object
Class representing command-line options for the cback3-amazons3-sync script.
The
Options
class is a Python object representation of the command-line options of the cback3-amazons3-sync script.The object representation is two-way: a command line string or a list of command line arguments can be used to create an
Options
object, and then changes to the object can be propogated back to a list of command-line arguments or to a command-line string. AnOptions
object can even be created from scratch programmatically (if you have a need for that).There are two main levels of validation in the
Options
class. The first is field-level validation. Field-level validation comes into play when a given field in an object is assigned to or updated. We use Python’sproperty
functionality to enforce specific validations on field values, and in some places we even use customized list classes to enforce validations on list members. You should expect to catch aValueError
exception when making assignments to fields if you are programmatically filling an object.The second level of validation is post-completion validation. Certain validations don’t make sense until an object representation of options is fully “complete”. We don’t want these validations to apply all of the time, because it would make building up a valid object from scratch a real pain. For instance, we might have to do things in the right order to keep from throwing exceptions, etc.
All of these post-completion validations are encapsulated in the
Options.validate
method. This method can be called at any time by a client, and will always be called immediately after creating aOptions
object from a command line and before exporting aOptions
object back to a command line. This way, we get acceptable ease-of-use but we also don’t accept or emit invalid command lines.Note: Lists within this class are “unordered” for equality comparisons.
-
__init__
(argumentList=None, argumentString=None, validate=True)[source]¶ Initializes an options object.
If you initialize the object without passing either
argumentList
orargumentString
, the object will be empty and will be invalid until it is filled in properly.No reference to the original arguments is saved off by this class. Once the data has been parsed (successfully or not) this original information is discarded.
The argument list is assumed to be a list of arguments, not including the name of the command, something like
sys.argv[1:]
. If you passsys.argv
instead, things are not going to work.The argument string will be parsed into an argument list by the
util.splitCommandLine
function (see the documentation for that function for some important notes about its limitations). There is an assumption that the resulting list will be equivalent tosys.argv[1:]
, just likeargumentList
.Unless the
validate
argument isFalse
, theOptions.validate
method will be called (with its default arguments) after successfully parsing any passed-in command line. This validation ensures that appropriate actions, etc. have been specified. Keep in mind that even ifvalidate
isFalse
, it might not be possible to parse the passed-in command line, so an exception might still be raised.Note: The command line format is specified by the
_usage
function. Call_usage
to see a usage statement for the cback3-amazons3-sync script.Note: It is strongly suggested that the
validate
option always be set toTrue
(the default) unless there is a specific need to read in invalid command line arguments.Parameters: - argumentList (List of arguments, i.e.
sys.argv
) – Command line for a program - argumentString (String, i.e. "cback3-amazons3-sync --verbose stage store") – Command line for a program
- validate (Boolean true/false) – Validate the command line after parsing it
Raises: getopt.GetoptError
– If the command-line arguments could not be parsedValueError
– If the command-line arguments are invalid
- argumentList (List of arguments, i.e.
-
buildArgumentList
(validate=True)[source]¶ Extracts options into a list of command line arguments.
The original order of the various arguments (if, indeed, the object was initialized with a command-line) is not preserved in this generated argument list. Besides that, the argument list is normalized to use the long option names (i.e. –version rather than -V). The resulting list will be suitable for passing back to the constructor in the
argumentList
parameter. UnlikebuildArgumentString
, string arguments are not quoted here, because there is no need for it.Unless the
validate
parameter isFalse
, theOptions.validate
method will be called (with its default arguments) against the options before extracting the command line. If the options are not valid, then an argument list will not be extracted.Note: It is strongly suggested that the
validate
option always be set toTrue
(the default) unless there is a specific need to extract an invalid command line.Parameters: validate (Boolean true/false) – Validate the options before extracting the command line Returns: List representation of command-line arguments Raises: ValueError
– If options within the object are invalid
-
buildArgumentString
(validate=True)[source]¶ Extracts options into a string of command-line arguments.
The original order of the various arguments (if, indeed, the object was initialized with a command-line) is not preserved in this generated argument string. Besides that, the argument string is normalized to use the long option names (i.e. –version rather than -V) and to quote all string arguments with double quotes (
"
). The resulting string will be suitable for passing back to the constructor in theargumentString
parameter.Unless the
validate
parameter isFalse
, theOptions.validate
method will be called (with its default arguments) against the options before extracting the command line. If the options are not valid, then an argument string will not be extracted.Note: It is strongly suggested that the
validate
option always be set toTrue
(the default) unless there is a specific need to extract an invalid command line.Parameters: validate (Boolean true/false) – Validate the options before extracting the command line Returns: String representation of command-line arguments Raises: ValueError
– If options within the object are invalid
-
debug
¶ Command-line debug (
-d,--debug
) flag.
-
diagnostics
¶ Command-line diagnostics (
-D,--diagnostics
) flag.
-
help
¶ Command-line help (
-h,--help
) flag.
-
ignoreWarnings
¶ Command-line ignoreWarnings (
-w,--ignoreWarnings
) flag
-
logfile
¶ Command-line logfile (
-l,--logfile
) parameter.
-
mode
¶ Command-line mode (
-m,--mode
) parameter.
-
output
¶ Command-line output (
-O,--output
) flag.
-
owner
¶ Command-line owner (
-o,--owner
) parameter, as tuple(user,group)
.
-
quiet
¶ Command-line quiet (
-q,--quiet
) flag.
-
s3BucketUrl
¶ Command-line s3BucketUrl, target of sync.
-
sourceDir
¶ Command-line sourceDir, source of sync.
-
stacktrace
¶ Command-line stacktrace (
-s,--stack
) flag.
-
validate
()[source]¶ Validates command-line options represented by the object.
Unless
--help
or--version
are supplied, at least one action must be specified. Other validations (as for allowed values for particular options) will be taken care of at assignment time by the properties functionality.Note: The command line format is specified by the
_usage
function. Call_usage
to see a usage statement for the cback3-amazons3-sync script.Raises: ValueError
– If one of the validations fails
-
verbose
¶ Command-line verbose (
-b,--verbose
) flag.
-
verifyOnly
¶ Command-line verifyOnly (
-v,--verifyOnly
) flag.
-
version
¶ Command-line version (
-V,--version
) flag.
-
-
CedarBackup3.tools.amazons3.
cli
()[source]¶ Implements the command-line interface for the
cback3-amazons3-sync
script.Essentially, this is the “main routine” for the cback3-amazons3-sync script. It does all of the argument processing for the script, and then also implements the tool functionality.
This function looks pretty similiar to
CedarBackup3.cli.cli()
. It’s not easy to refactor this code to make it reusable and also readable, so I’ve decided to just live with the duplication.A different error code is returned for each type of failure:
1
: The Python interpreter version is < 3.42
: Error processing command-line arguments3
: Error configuring logging5
: Backup was interrupted with a CTRL-C or similar6
: Error executing other parts of the script
Note: This script uses print rather than logging to the INFO level, because it is interactive. Underlying Cedar Backup functionality uses the logging mechanism exclusively.
Returns: Error code as described above
CedarBackup3.tools.span module¶
Spans staged data among multiple discs
This is the Cedar Backup span tool. It is intended for use by people who stage more data than can fit on a single disc. It allows a user to split staged data among more than one disc. It can’t be an extension because it requires user input when switching media.
Most configuration is taken from the Cedar Backup configuration file, specifically the store section. A few pieces of configuration are taken directly from the user.
author: | Kenneth J. Pronovici <pronovic@ieee.org> |
---|
-
class
CedarBackup3.tools.span.
SpanOptions
(argumentList=None, argumentString=None, validate=True)[source]¶ Bases:
CedarBackup3.cli.Options
Tool-specific command-line options.
Most of the cback3 command-line options are exactly what we need here – logfile path, permissions, verbosity, etc. However, we need to make a few tweaks since we don’t accept any actions.
Also, a few extra command line options that we accept are really ignored underneath. I just don’t care about that for a tool like this.
-
CedarBackup3.tools.span.
cli
()[source]¶ Implements the command-line interface for the
cback3-span
script.Essentially, this is the “main routine” for the cback3-span script. It does all of the argument processing for the script, and then also implements the tool functionality.
This function looks pretty similiar to
CedarBackup3.cli.cli()
. It’s not easy to refactor this code to make it reusable and also readable, so I’ve decided to just live with the duplication.A different error code is returned for each type of failure:
1
: The Python interpreter version is < 3.42
: Error processing command-line arguments3
: Error configuring logging4
: Error parsing indicated configuration file5
: Backup was interrupted with a CTRL-C or similar6
: Error executing other parts of the script
Note: This script uses print rather than logging to the INFO level, because it is interactive. Underlying Cedar Backup functionality uses the logging mechanism exclusively.
Returns: Error code as described above