nestly Package

nestly Package

nestly is a collection of functions designed to make running software with combinatorial choices of parameters easier.

core Module

Core functions for building nests.

class nestly.core.Nest(control_name='control.json', indent=2, fail_on_clash=False, warn_on_clash=True, base_dict=None, include_outdir=True)

Bases: object

Nests are used to build nested parameter selections, culminating in a directory structure representing choices made, and a JSON dictionary with all selections.

Build parameter combinations with Nest.add(), then create a nested directory structure with Nest.build().

Parameters:

• control_name – Name JSON file to be created in each leaf
• indent – Indentation level in json file
• fail_on_clash – Error if a nest level attempts to overwrite a previous value
• warn_on_clash – Print a warning if a nest level attempts ot overwrite a previous value
• base_dict – Base dictionary to start all control dictionaries from (default: {})
• include_outdir – If true, include an OUTDIR key in every control indicating the directory this control would be written to.

add(name, nestable, create_dir=True, update=False, label_func=<type 'str'>, template_subs=False)

Add a level to the nest

Parameters:

• name (string) – Name of the level. Forms the key in the output dictionary.
• nestable – Either an iterable object containing values, _or_ a function which takes a single argument (the control dictionary) and returns an iterable object containing values
• create_dir (boolean) – Should a directory level be created for this nestable?
• update (boolean) – Should the control dictionary be updated with the results of each value returned by the nestable? Only valid for dictionary results; useful for updating multiple values. At a minimum, a key-value pair corresponding to name must be returned.
• label_func – Function to be called to convert each value to a directory label.
• template_subs (boolean) – Should the strings in / returned by nestable be treated as templates? If true, str.format is called with the current values of the control dictionary.

build(root='runs')

Build a nested directory structure, starting in root

Parameters:root – Root directory for structure

iter(root=None)

Create an iterator of (directory, control_dict) tuples for all valid parameter choices in this Nest.

Parameters:root – Root directory Return type:Generator of (directory, control_dictionary) tuples.

nestly.core.control_iter(base_dir, control_name='control.json')

Generate the names of all control files under base_dir

nestly.core.nest_map(control_iter, map_fn)

Apply map_fn to the directories defined by control_iter

For each control file in control_iter, map_fn is called with the directory and control file contents as arguments.

Example:

>>> list(nest_map(['run1/control.json', 'run2/control.json'],
...               lambda d, c: c['run_id']))
[1, 2]

Parameters:

• control_iter – Iterable of paths to JSON control files
• map_fn (function) – Function to run for each control file. It should accept two arguments: the directory of the control file and the json-decoded contents of the control file.

Returns:

A generator of the results of applying map_fn to elements in control_iter

nestly.core.stripext(path)

Return the basename, minus extension, of a path.

Parameters:path (string) – Path to file

scons Module

SCons integration for nestly.

class nestly.scons.SConsWrap(nest, dest_dir='.')

Bases: object

A Nest wrapper to add SCons integration.

This class wraps a Nest in order to provide methods which are useful for using nestly with SCons.

A Nest passed to SConsWrap must have been created with include_outdir=True, which is the default.

add(*a, **kw)

Call .add on the wrapped Nest.

add_aggregate(data_fac, name=None)

Add an aggregate target to this nest.

The first argument is a nullary factory function which will be called immediately for each of the current control dictionaries and stored in each dictionary with the given name like in add_target. After finalize_aggregate or finalize_all_aggregates are called, the decorated function will then be called in the same way as add_target, except with an additional argument: the value which was returned by the factory function.

Since nests added after the aggregate can access the factory function’s value, it can be mutated to provide additional values for use when the decorated function is called.

add_nest(name=None, **kw)

A simple decorator which wraps nest.add.

add_target(name=None)

Add an SCons target to this nest.

The function decorated will be immediately called with each of the output directories and current control dictionaries. Each result will be added to the respective control dictionary for later nests to access.

finalize_aggregate(aggregate)

Call the finalizers for one particular aggregate.

Finalizing an aggregate this way means that it will not be finalized by any future calls to finalize_all_aggregates.

finalize_all_aggregates()

Call the finalizers for all defined aggregates.

If any aggregates have been specifically finalized by finalize_aggregate, they will not be finalized again. This function itself calls finalize_aggregate; if finalize_all_aggregates is called twice, aggregates will not be finalized twice.

Aggregates will be finalized in the same order in which they were defined.

nestly.scons.name_targets(func)

Wrap a function such that returning 'a', 'b', 'c', [1, 2, 3] transforms the value into dict(a=1, b=2, c=3).

This is useful in the case where the last parameter is an SCons command.

Subpackages

• scripts Package
  • nestrun Module
  • nestagg Module