Nick’s Python Toolbox¶
Nxpy is an etherogeneous collection of libraries, dealing with diverse topics such as wrapping complex commands with API’s, automation of backup files, support for writing your own file-like objects and many other things.
abstract
- Additions to the abc standard module¶
Helpers for the standard abc
module.
-
class
abstractstatic
(function)[source]¶ Decorator that combines
staticmethod
andabc.abstractmethod
.Copied from this answer to this StackOverflow question.
backup_file
- File objects with automated backup¶
Backup a file or directory to make editing reversible.
Implement the context manager protocol, so as to be suitable to be used with the with statement. When used in this fashion changes are discarded when an exception is thrown.
-
class
BackupDir
(dir_, ext='.BAK', mode=1)[source]¶ Move or copy a directory that needs to be recreated or modified.
-
__exit__
(exc_type, exc_val, exc_tb)[source]¶ When the controlling with statement is exited normally discard the backup directory, otherwise restore it to its original place.
-
__init__
(dir_, ext='.BAK', mode=1)[source]¶ Prepare to backup the dir_ directory.
The backup will be created in dir_’s parent directory, which must be writable, with extension ext. If mode is MOVE, the default, the original directory will be moved to the backup destination; if mode is COPY it will be copied there.
-
-
class
BackupFile
(file_, ext='.BAK', dir='.', mode=2)[source]¶ Implements a read only file object used to automatically back up a file that has to be modified.
-
__exit__
(exc_type, exc_val, exc_tb)[source]¶ When the controlling with statement is exited normally discard the backup file, otherwise restore it to its original place.
-
__init__
(file_, ext='.BAK', dir='.', mode=2)[source]¶ Prepare to backup file_, either a file-like object or a path.
The backup file will be created in directory dir with extension ext. If mode is COPY the original file will be copied to the backup destination; if mode is MOVE it will be moved there.
-
close
()[source]¶ Close the backup file and release the corresponding reference.
The backup file may not be reopened.
-
name
¶ The name of the file to be backed up.
-
-
exception
NotSavedError
[source]¶ Raised when commit or rollback is called on an inactive BackUpFile or BackUpDirectory.
command
- Wrap complex commands in Python objects¶
Tools to wrap a Python API around interactive and non-interactive programs.
The command.Command
and interpreter.Interpreter
classes handle batch and
interactive commands respectively. They can be provided with option.Config
instances
which describe the options available to the programs being wrapped. The option.Parser
class can then be used to validate option sets and construct the corresponding command lines.
See the svn.svn
module for a concrete example.
command
- Drive batch commands with function calls¶
Non interactive command driver.
-
class
Command
(cmd, debug=False)[source]¶ Represents the command to be executed. Typically you would derive from this class and provide a different method for each alternative way of invoking the program. If the program you want to execute has many sub-commands you might provide a different method for each sub-command. You can use the
option.Config
class to declare the options supported by your command and then use theoption.Parser
class to validate your methods’ arguments and generate the resulting command line. A debug mode is available in which commands are echoed rather than run. This can be enabled globally or separately for each invocation.
error
- The command
package exception hierarchy¶
Exception classes for the nxpy.command package.
interpreter
- Wrap interactive programs in Python classes¶
Interactive program driver.
-
class
BaseInterpreter
(popen)[source]¶ Controls the execution of an interactive program in a sub-process. Provides means to send input to the controlled process and to check different conditions on its output and error streams.
-
__init__
(popen)[source]¶ Creates an interpreter instance. popen is a
Popen
-like object which must support non-blocking I/O.
-
expect
(cond=None, timeout=0, retries=0, interval=0.01, quantum=0.01, raise_on_error=True, log=None)[source]¶ Express expectations on the outcome of a command.
cond is a two argument callable which will be passed the command’s standard output and standard error, and which should return True if the expectation is satisfied. For the other arguments see the documentation for the
Timer
class.
-
expect_regexp
(regexp, where=0, **kwargs)[source]¶ Expect to find a match for the regexp regular expression within the where stream.
-
run
(cmd, log=None, **kwargs)[source]¶ Executes the command and waits for the expected outcome or an error.
-
-
class
Interpreter
(cmd)[source]¶ The actual Interpreter class.
This implementation uses a
core.nonblocking_subprocess.NonblockingPopen
instance.
-
class
RegexpWaiter
(regexp, where)[source]¶ Wait for a match to a given regexp, passed either compiled or as a string.
-
class
Timer
(timeout=0, retries=0, interval=0.1, quantum=0.01)[source]¶ A collaborative timer class. Support a polling mechanism by keeping track of the amount of time to wait before the next attempt, according to different policies.
-
__init__
(timeout=0, retries=0, interval=0.1, quantum=0.01)[source]¶ Specify an overall timeout, a number of retries and/or an interval between them. The next attempt will not take place before a quantum has passed. Timings are expressed in seconds. If a timeout is specified it will take precedence over the other arguments; in that case the number of retries will take precedence over the interval. If neither a timeout nor a number of retries are specified the overall timer will never expire.
-
expired
()[source]¶ Indicate whether the current timer expired. Use as polling loop control condition.
-
option
- Describe complex command lines¶
Function argument to command line option conversion. Provides means to describe commands with complicated syntaxes, which often combine sub-commands, options and arguments. Typical examples include subversion and ftp.
-
class
Config
(prefix='--', separator=' ', bool_opts=(), value_opts=(), iterable_opts=(), format_opts={}, mapped_opts={}, opposite_opts={})[source]¶ Command option definitions. Provides a single definition point for all the options supported by a command.
-
__init__
(prefix='--', separator=' ', bool_opts=(), value_opts=(), iterable_opts=(), format_opts={}, mapped_opts={}, opposite_opts={})[source]¶ Constructor. Its arguments are used to specify all the valid options. Each option is prefixed by prefix. When an option takes multiple arguments these are separated by a separator. bool_opts must be specified on the command line when they are True. value_opts take a single argument; iterable_opts take multiple arguments; format_opts have their syntax specified by means of a format string; mapped_opts require some form of translation, usually because they are not valid Python identifiers; opposite_opts must be specified on the command line when they are False.
-
-
class
Parser
(config, command, arguments, options, **defaults)[source]¶ Constructs a complex command line from the provided command and its options and arguments. Uses a
Config
instance, config, to provide means to check conditions on the supplied options. Other constraints on how options should be used may be expressed and verified by means of the check methods.-
__init__
(config, command, arguments, options, **defaults)[source]¶ Takes an instance of
Config
, a command to execute, an iterable of arguments and a mapping of options and their actual values. The remaining keyword arguments indicate the options supported by command with their default values.
-
checkExactlyOneOption
(*options)[source]¶ Checks that one and only one in a set of mutually exclusive options has been specified.
-
checkExclusiveOptions
(*options)[source]¶ Checks that at most one in a set of mutually exclusive options has been specified.
-
checkNotBothOptsAndArgs
(*options)[source]¶ Checks that options incompatible with arguments haven’t been specified if any argument is present.
-
file_object
- Stubs for read-only and modifiable file-like objects¶
Helper classes for the implementation of read-only and writable file objects that forward calls to an actual file object variable.
file
- File related utilities¶
File related utilities.
maven
- Tools to execute the Maven build tool and manipulate its configuration¶
Tools to drive the Maven build tool and to manipulate its configuration files.
artifact
- Representation of a Maven artifact¶
assembly_descriptor
- Representation of a Maven Assembly plugin’s descriptor¶
mvn
- Wrapper class for the mvn
command line tool¶
Maven wrapper.
pom
- Representation of a Maven POM
file¶
memo
- Memoize objects according to a given key¶
Memoize class instances according to a given key.
By default the key only assumes the True value, thus implementing a singleton.
nonblocking_subprocess
- Subprocesses with non-blocking I/O¶
Allow non-blocking interaction with a subprocess.
This module was taken from this recipe in the ActiveState Code Recipes website, with only minor modifications. This is the original description:
Title: Module to allow Asynchronous subprocess use on Windows and Posix platforms
Submitter: Josiah Carlson (other recipes)
Last Updated: 2006/12/01
Version no: 1.9
Category: System
On Windows pywin32 is required.
-
class
NonblockingPopen
(cmd, encoding=None, **kwargs)[source]¶ An asynchronous variant to
subprocess.Popen
, which doesn’t block on incomplete I/O operations.Note that the terms input, output and error refer to the controlled program streams, so we receive from output or error and we send to input.
-
__init__
(cmd, encoding=None, **kwargs)[source]¶ Execute cmd in a subprocess, using encoding to convert to and from binary data written or read from/to the subprocess’s input, output and error streams.
Additional keyword arguments are as specified by
subprocess.Popen.__init__()
method.
-
-
recv_some
(p, t=0.1, e=1, tr=5, stderr=0)[source]¶ Try and receive data from
NonblockingPopen
object p’s stdout in at most tr tries and with a timeout of t. If stderr is True receive from the subprocess’s stderr instead.
-
send_all
(p, data)[source]¶ Send all of data to
NonblockingPopen
object p’s stdin.
past
- Python version support enforcement¶
Identification and enforcement of supported Python releases.
-
class
Version
(version)[source]¶ Identifies a Python release in a way that is convenient for comparison and printing.
path
- File system related utilities¶
filesystem related utilities.
-
class
CurrentDirectory
(path)[source]¶ A context manager that allows changing the current directory temporarily.
-
current
¶ Return the current directory.
-
-
blasttree
(dir_)[source]¶ Remove a directory more stubbornly than
shutil.rmtree()
.Required on filesystems that do not allow removal of non-writable files
ply
- Add-ons for the PLY lexer & parser generator¶
Wrapper classes for the PLY parser generator.
parser
- A class wrapper for PLY parsers¶
scanner
- A class wrapper for PLY scanners¶
sequence
- Sequence related utilities¶
Utility functions that deal with non-string sequences.
sort
- Sorting functions¶
Sort functions.
svn
- High level API for the Subversion version control tool¶
A Python API for the Subversion version control tool.
A lazy, ahem, agile person’s answer to the official svn bindings.
svn
- Wrapper for the svn client tool¶
Subversion client wrapper.
Only supports versions 1.6, 1.7 and 1.8, others might work but have not been tested. Requires at least Python 2.6.
-
class
Parser
(command, arguments, options, **defaults)[source]¶ Allows passing nxpy.svn.url.Url instances as arguments to Svn’s methods.
-
class
Status
(line)[source]¶ Represents the output of one line of the
svn status
command in a structured way.
-
class
Svn
(debug=False)[source]¶ The actual wrapper.
-
__init__
(debug=False)[source]¶ Takes as arguments the command name and a boolean value indicating whether debug mode should be activated for all executions of this command.
-
getexternals
(d)[source]¶ Return d’s
svn:externals
property as a dictionary of directory - URL pairs.Note that only a limited subset of the externals syntax is supported: either the pre-svn 1.5 one (directory - URL) or the same with inverted elements. Throw nxpy.svn.url.BadUrlError if an external URL is malformed.
-
svnadmin
- Wrapper for the svnadmin administration tool¶
Subversion administration tool wrapper.
url
- Models a URL adhering to the trunk/tags/branches convention¶
Subversion URL manipulation.
wcopy
- Models a working copy¶
Working copy manipulation.
-
exception
ModifiedError
[source]¶ Raised when attempting to tag or branch a working copy that contains changes.
-
exception
NotOnBranchError
[source]¶ Raised when attempting to delete a working copy that is not on the requested branch.
-
exception
NotOnTagError
[source]¶ Raised when attempting to delete a working copy that is not on the requested tag.
temp_file
- Temporary files that support the context protocol¶
Temporary files and directories.
Requires at least Python 2.6
-
class
TempDir
(*args, **kwargs)[source]¶ A temporary directory that implements the context manager protocol.
The directory is removed when the context is exited from. Uses
tempfile.mkdtemp()
to create the actual directory.-
name
¶ Return the directory name.
-
-
class
TempFile
(*args, **kwargs)[source]¶ A temporary file that implements the context manager protocol.
Wrap a
tempfile.NamedTemporaryFile()
generated file-like object, to ensure it is not deleted on close, but rather when the underlying context is closed.-
name
¶ Return the actual file name.
-
test
- Test support utilities¶
Testing related utilities.
env
- Access to the testing environment for the svn, maven and msvs packages¶
Environment configuration for tests that interact with the system.
log
- Log configuration for tests¶
Logging configuration for tests.
Running the tests¶
Nxpy tests are based on the standard unittest
module. As recent features are used the
unittest2
backport is required with Python 2.6. Tests reside in _test
subdirectories of the
library package directory. For each module
module tests should be found in a test_module
module.
Tests may be run for all supported Python versions installed on your system and present in your
PATH
environment variable with tox
and pytest
. Dependencies from libraries available
from PyPI are automatically installed by tox. The following additional requirements should also be
fulfilled:
- The
nxpy-maven
library requires that a recent version of Maven be present in yourPATH
. - The
nxpy-svn
library requires that a recent version of Subversion be present in yourPATH
.
Generating the documentation¶
Nxpy’s documentation is written in reStructuredText and rendered with Sphinx.
Creating new releases¶
Libraries should only be released when needed. Although a combined package is not released, its configuration should be updated to reflect library changes.
Assuming all changes to code and documentation have been pushed to the upstream repository, the basic steps for the creation of a new library release are:
Run
tox
in the root directory of your development checkout. Ideally you should be developing against the most recent supported Python release on one of the supported platforms. As long as Python 2.7 is supported tests should be run against it too.Update any release related configuration file, e.g.:
CHANGES.txt
README.rst
setup.py
Commit to
master
any remaining change and push upstream. This should trigger GitHub Actions tests against all the supported Python versions.Bump the library version number according to semantic versioning: increment the minor version element if the new release includes API breaking changes, the increment version element otherwise. Add
rc1
to the release number to mark the fact that this is a release candidate.Run
python setup.py sdist bdist_wheel
in the library’s root directory. Check the contents of the resulting packages in thedist
directory.Run
twine check dist/*
. Fix any resulting problem.Run
twine upload --repository-url https://test.pypi.org/legacy/ dist/*
to upload the library to Test PyPI. You will need a Test PyPI account for that.Create a new virtualenv and install the new library in it with
pip install --index-url https://test.pypi.org/simple/ --no-deps --pre <<Library>>
.Perform a minimal test.
Remove the
build
,dist
and<<Library>>.egg-info
directories.Remove the
rc1
prefix from the version number in thesetup.py
file.Commit and push all outstanding changes.
Run
python setup.py sdist bdist_wheel
again and check the contents of the resulting packages.Run
twine check dist/*
.Run
twine upload dist/*
to upload the library to PyPI. You will need a PyPI account.Create another virtualenv and install the new library in it with
pip install <<Library>>
.Perform a last test.
Supported Python versions 2.7 3.6 3.7 3.8 3.9 3.10 Supported platforms Linux MacOS Windows 7 or later
The libraries are being developed with Python 3.9 so as to be compatible with Python 2.7. Tests are run and most modules work also with 3.6, 3.7 and 3.8. Some should still work with versions as early as 3.2 and 2.5. There is no immediate plan to remove Python 2.x support, but in general earlier releases will only be supported as long as external tools, such as Travis or pip, keep supporting them.
Originally the libraries resided on SourceForge and were distributed as a single package. Starting from release 1.0.0 each library is being packaged separately even though they are all hosted within the same project on GitHub.
The Nxpy logo was drawn by Claudia Romano.