xpipe.config package¶

Submodules¶

xpipe.config.config module¶

xpipe.config.config.load_yaml(config_file: str, args: Optional[dict] = None)¶

Loads a configuration file and return a dictionary. This loader is able to load custom tags.

Parameters:

config_file (str) – The path of the yaml config file
args (dict) – Dictionary of arguments that will be passed to jinja

Returns:

A dictionary containing a representation of the configuration.

Return type:

dict

xpipe.config.config.load_config(config_file: str, args: Optional[dict] = None)¶

Loads a configuration file and return a Config Object which can instantiate the wanted objects.

Parameters:

config_file (str) – The path of the yaml config file
args (dict) – Dictionary of arguments that will be passed to jinja

Returns:

A Config object

Return type:

Config

xpipe.config.config.load_config_from_str(conf: str, args: Optional[dict] = None, base_path: Optional[str] = None)¶

Loads a configuration from a string and return a Config Object which can instantiate the wanted objects.

Parameters:

conf (str) – A configuration
base_path (str) – Simulate that ‘conf’ comes from this given path. It will be used if your conf contains relative paths.

Returns:

A Config object

Return type:

Config

xpipe.config.config.to_yaml(conf)¶

Converts a Config object to a yaml string

Parameters:: conf (Config) – A configuration
Returns:: The corresponding yaml string
Return type:: str

xpipe.config.config.to_dict(conf)¶

Converts a Config object to a dictionary.

Parameters:: conf (Config) – A Config object
Returns:: A multi-level dictionary containing a representation of the configuration.
Return type:: dict

xpipe.config.config.to_objects(conf)¶

Converts a Config object to a dictionary of real python objects.

Parameters:: conf (Config) – A Config object
Returns:: A multi-level dictionary containing a representation of the configuration.
Return type:: dict

xpipe.config.config.merge(*confs, inplace=False)¶

Merges multiple configurations.

Parameters:: confs (Config) – The configurations to merge
Returns:: The merged configuration
Return type:: Config

xpipe.config.config.get_node(config, path, delimiter='/', parent='..', attribute='.')¶

Gets a node from a configuration.

Parameters:

config (Config) – The configuration
path (str) – The path of the node to get
delimiter (str) – The delimiter used to separate the path

Returns:

The node

Return type:

Config

xpipe.config.config.get_base(node)¶

Return the root node of the given node (highest level node). If the node is in an included configuration, the base node is the root node of the included configuration.

Errors:: ValueError: In case there is a circular reference

Parameters:: node (Node) – The node to get the base node from

xpipe.config.config.set_parent(node, parent)¶

Sets the parent of a node.

Parameters:

node (Node) – The node to set the parent of
parent (Node) – The parent node

xpipe.config.config.set_name(node, name)¶

Sets the name of a node.

Parameters:

node (Node) – The node to set the name of
name (str) – The name of the node

xpipe.config.config.set_config_dict(node, config_dict)¶

Sets the node config dict

Parameters:

node (Node) – Node
config_dict (dict) – The config dict of the node

xpipe.config.loader module¶

xpipe.config.loader.parse_path(class_path, aliases=None)¶

Get the path of a class from a given path using importlib library. If the path first part is an alias, the path is replaced with the real path. exemple: class_path = “my_alias.Class” aliases = {“my_alias”: “package.module”} return “package.module.Class”

Parameters:

class_path (str) – Class path (e.g. “package.module.Class”)
aliases (dict<str, str>, optional) – A dictionary of aliases. The key is the alias and the value is the path. Defaults to None.

Returns:

_description_

Return type:

_type_

xpipe.config.loader.load_class(class_path, aliases=None)¶

Loads a class from a given path using importlib library.

Parameters:

class_path (str) – Class path (e.g. “package.module.Class”)
aliases (dict<str, str>, optional) – A dictionary of aliases. The key is the alias and the value is the path. Defaults to None.

Raises:

ImportError – If the module cannot be imported.
AttributeError – If the module does not contain the class.

Returns:

The class.

Return type:

Object

xpipe.config.node module¶

class xpipe.config.node.Node¶

Bases: object

classmethod from_yaml(loader, node)¶

classmethod to_yaml(dumper, data)¶

xpipe.config.objects module¶

class xpipe.config.objects.Config¶

Bases: xpipe.config.node.Node, dict

A configuration is like a dictionary containing elements like SingleObject, Config and so on.

class xpipe.config.objects.FromIncludes¶

Bases: xpipe.config.node.Node

yaml_tag = '!from'¶

class xpipe.config.objects.List¶: Bases: xpipe.config.node.Node, list

class xpipe.config.objects.SingleObject¶

Bases: xpipe.config.node.Node

Allow the instantiation of an object defined in a yaml configuration file.

yaml_tag = '!obj'¶

class xpipe.config.objects.ObjectsList¶

Bases: xpipe.config.objects.List

Create a list of SingleObject from a yaml configuration file.

xpipe.config.objects.construct(name, config_dict, parent=None, **kwargs)¶

Build a tree from a dictionary

Parameters:

name (str) – Name of the node
config_dict (dict) – The dictionary
parent (Node) – Parent node
**kwargs – Complementary arguments to pass to _xpipe_construct method of the node being built.

Returns:

The build tree element

Return type:

Node | Variable

xpipe.config.tag module¶

xpipe.config.tag.register(yaml_tag)¶

Register a class such that it can be instantiated from a yaml tag

Parameters:: yaml_tag (str) – The corresponding tag to instantiate the registered class

xpipe.config.tag.save_tags(yaml)¶

Register all classes that have @tags.register to the given yaml library. Yaml will then be able to load them when needed.

Parameters:: yaml (lib) – The yaml library

xpipe.config.utils module¶

xpipe.config.utils.is_objects_list(name, config_dict)¶

Check if the given configuration is an objects list.

Parameters:: config_dict (any) – A configuration
Returns:: True if ‘config_dict’ is a dictionary that defines an objects list
Return type:: bool

xpipe.config.utils.is_var(name, config_dict)¶

Checks if the given configuration defines a Variable

Parameters:: config_dict (any) – A configuration
Returns:: True if ‘config_dict’ defines a Variable
Return type:: bool

xpipe.config.utils.is_list(name, config_dict)¶

Check if the given configuration is an objects.List.

Parameters:

name (str) – Name of the variable
config_dict (dict) – A configuration

Returns:

True if ‘config_dict’ is an objects.List.

Return type:

bool

xpipe.config.utils.is_config(name, config_dict)¶

Check if the given configuration is a objects.Config.

Parameters:

name (str) – Name of the variable
config_dict (dict) – A configuration

Returns:

True if ‘config_dict’ is a objects.Config

Return type:

bool

xpipe.config.utils.valid_var_name(name: str)¶

Raise an error if ‘name’ is not a valid Variable name. A valid variable name is a string that starts with a letter or an underscore and is followed by letters, numbers, or underscores.

Parameters:

name (str) – Name of the variable

Raises:

ValueError – If name contains caracters that are not alphabetical or numerical
ValueError – If name begin with a number

xpipe.config.utils.apply_jinja_templating(text: str, args: Optional[dict] = None)¶

Apply jinja templating on ‘text’

Parameters:

text (str) – A jinja template to render
args (Optional[dict], optional) – All needed arguments to render the template. Defaults to None.

Returns:

Rendered template

Return type:

str

xpipe.config.variables module¶

class xpipe.config.variables.Variable(value)¶

Bases: xpipe.config.node.Node

property value¶

property name¶

class xpipe.config.variables.SimpleVariable¶

Bases: xpipe.config.variables.Variable

A basic variable with no additional functionality.

class xpipe.config.variables.EnvVariable(value)¶

Bases: xpipe.config.variables.Variable

This class defines a yaml tag. It will load an environment variable.

yaml_tag = '!env'¶

class xpipe.config.variables.ReferenceVariable(value)¶

Bases: xpipe.config.variables.Variable

This class defines a yaml tag. It is a reference to another node.

property value¶

yaml_tag = '!ref'¶

class xpipe.config.variables.FormatStrVariable(value)¶

Bases: xpipe.config.variables.Variable

This class defines a yaml tag. The class will automatically replace substrings $ENV_VAR or ${ENV_VAR} with the corresponding environment variables.

yaml_tag = '!f'¶

class xpipe.config.variables.ClassTag(class_path)¶

Bases: xpipe.config.variables.Variable

This class defines a yaml tag It store a class (not an instance)

yaml_tag = '!class'¶

Module contents¶

xpipe.config is a package implementing main classes needed to load the yaml tree file and load objects from it.

xpipe.config.load_config(config_file: str, args: Optional[dict] = None)¶

Loads a configuration file and return a Config Object which can instantiate the wanted objects.

Parameters:

config_file (str) – The path of the yaml config file
args (dict) – Dictionary of arguments that will be passed to jinja

Returns:

A Config object

Return type:

Config

xpipe.config.load_config_from_str(conf: str, args: Optional[dict] = None, base_path: Optional[str] = None)¶

Loads a configuration from a string and return a Config Object which can instantiate the wanted objects.

Parameters:

conf (str) – A configuration
base_path (str) – Simulate that ‘conf’ comes from this given path. It will be used if your conf contains relative paths.

Returns:

A Config object

Return type:

Config

xpipe.config.to_dict(conf)¶

Converts a Config object to a dictionary.

Parameters:: conf (Config) – A Config object
Returns:: A multi-level dictionary containing a representation of the configuration.
Return type:: dict

xpipe.config.to_yaml(conf)¶

Converts a Config object to a yaml string

Parameters:: conf (Config) – A configuration
Returns:: The corresponding yaml string
Return type:: str

xpipe.config.merge(*confs, inplace=False)¶

Merges multiple configurations.

Parameters:: confs (Config) – The configurations to merge
Returns:: The merged configuration
Return type:: Config