nestly is a collection of functions designed to make running software with combinatorial choices of parameters easier.
Core functions for building nests.
Nest(control_name='control.json', indent=2, fail_on_clash=False, warn_on_clash=True, base_dict=None, include_outdir=True)¶
Nests are used to build nested parameter selections, culminating in a directory structure representing choices made, and a JSON dictionary with all selections.
- control_name – Name JSON file to be created in each leaf
- indent – Indentation level in json file
- fail_on_clash – Error if a nest level attempts to overwrite a previous value
- warn_on_clash – Print a warning if a nest level attempts ot overwrite a previous value
- base_dict – Base dictionary to start all control dictionaries from
- include_outdir – If true, include an OUTDIR key in every control indicating the directory this control would be written to.
add(name, nestable, create_dir=True, update=False, label_func=<type 'str'>, template_subs=False)¶
Add a level to the nest
- name (string) – Name of the level. Forms the key in the output dictionary.
- nestable – Either an iterable object containing values, _or_ a function which takes a single argument (the control dictionary) and returns an iterable object containing values
- create_dir (boolean) – Should a directory level be created for this nestable?
- update (boolean) – Should the control dictionary be updated with
the results of each value returned by the nestable? Only valid for
dictionary results; useful for updating multiple values. At a
minimum, a key-value pair corresponding to
namemust be returned.
- label_func – Function to be called to convert each value to a directory label.
- template_subs (boolean) – Should the strings in / returned by nestable be treated as templates? If true, str.format is called with the current values of the control dictionary.
Build a nested directory structure, starting in
Parameters: root – Root directory for structure
Generate the names of all control files under base_dir
map_fnto the directories defined by
For each control file in control_iter, map_fn is called with the directory and control file contents as arguments.
>>> list(nest_map(['run1/control.json', 'run2/control.json'], ... lambda d, c: c['run_id'])) [1, 2]
- control_iter – Iterable of paths to JSON control files
- map_fn (function) – Function to run for each control file. It should accept two arguments: the directory of the control file and the json-decoded contents of the control file.
A generator of the results of applying
map_fnto elements in
Return the basename, minus extension, of a path.
Parameters: path (string) – Path to file
SCons integration for nestly.
SConsEncoder(skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, encoding='utf-8', default=None)¶
JSON Encoder which handles SCons objects.
SConsWrap(nest, dest_dir='.', alias_environment=None)¶
A Nest wrapper to add SCons integration.
This class wraps a
Nestin order to provide methods which are useful for using nestly with SCons.
A Nest passed to SConsWrap must have been created with
include_outdir=True, which is the default.
add(name, nestable, **kw)¶
Adds a level to the nesting and creates a checkpoint that can be reverted to later for aggregation by calling
Add an aggregate target to this nest.
Since nests added after the aggregate can access the construct returned by the factory function value, it can be mutated to provide additional values for use when the decorated function is called.
To do something with the aggregates, you must
SConsWrap.pop()nest levels created between addition of the aggregate and then can add any normal targets you would like which take advantage of the targets added to the data structure.
- name – Name for the target in the nest
- data_fac – a nullary factory function which will be called
immediately for each of the current control dictionaries and stored
in each dictionary with the given name as in
add_controls(env, target_name='control', file_name='control.json', encoder_cls=<class 'nestly.scons.SConsEncoder'>)¶
Adds a target to build a control file at each of the current leaves.
- env – SCons Environment object
- target_name – Name for target in nest
- file_name – Name for output file.
Add an SCons target to this nest.
The function decorated will be immediately called with each of the output directories and current control dictionaries. Each result will be added to the respective control dictionary for later nests to access.
Parameters: name – Name for the target in the name (default: function name).
Add an SCons target to this nest, with an SCons Environment
The function decorated will be immediately called with three arguments:
environment: A clone of the SCons environment, with variables populated for all values in the control dictionary, plus a variable
outdir: The output directory
control: The control dictionary
Each result will be added to the respective control dictionary for later nests to access.
SConsWrap.add_target()only by the addition of the
Reverts to the nest stage just before the corresponding call of
SConsWrap.add_aggregate(). However, any aggregate collections which have been worked on will still be accessible, and can be called operated on together after calling this method. If no name is passed, will revert to the last nest level.
Parameters: name – Name of the nest level to pop.
Wrap a function such that returning
'a', 'b', 'c', [1, 2, 3]transforms the value into
dict(a=1, b=2, c=3).
This is useful in the case where the last parameter is an SCons command.