Classes

The EchoBay Library is built using multiple modular classes under the same namespace that could be used together in the Limbo Optimization framework or separately to add Echo State Network functionalities to your project.

Reservoir

class Reservoir

Implementation of Echo State Network Reservoir unit

Reservoir is defined by the actual reservoir Wr, a input matrix Win and a non-linear state. Initialization of the Reservoir is defined by a high-level configuration structure which supports also differentiated input scaling, particular topologies, and deep networks.

Public Functions

Reservoir(const int nLayers = 1, const int Nu = 2, const int type = 0)

Construct a Reservoir class with a given topology, number of layers and inputs.

Parameters

  • nLayers: Number of layers in deep reservoirs (= 1 in shallow networks)

  • Nu: Number of input channels

  • type: Topology of the network see also EchoBay::esn_config(const Eigen::VectorXd &optParams, YAML::Node confParams, int Nu, const std::string folder) for details

~Reservoir()

Destroy the Reservoir object cleaning internal layers.

SparseBO init_Wr(const int Nr, const double density, const double scalingFactor, const double leaky, const int extraParams)

Initialize the Reservoir’s weights according to user-defined parameters.

Return

SparseBO Scaled Wr

Parameters

  • Nr: Reservoir size

  • density: Reservoir density

  • scalingFactor: Custom scaling factor TODO add reference

  • leaky: Leaky factor

  • extraParam: Extra parameter defining edges (SW topology) or jumps (CRJ topology)

void init_network()

Initialize the reservoir according to internal topology.

void init_WinLayers()

Initialize random input layers.

void init_WrLayers()

Initialize the weights of the Reservoir.

void init_stateMat()

Reset state matrix.

void init_WinLayers_swt()

Initialize input layers according to Small World Topology selection.

void init_WinLayers_crj()

Initialize input layers according to Cyclic Reservoir Jumps selection.

void load_network(const std::string &folder)

Load the whole Reservoir from files.

Parameters

  • folder: Path to the folder containing the files

void load_WinLayers(const std::string &folder)

Load input layers from files.

Parameters

  • folder: Path to the folder containing the files

void load_WrLayers(const std::string &folder)

Load Reservoir’s weight layers from files.

Parameters

  • folder: Path to the folder containing the files

void load_stateMat(const std::string &folder)

Load Reservoir’s states from files.

Parameters

  • folder: Path to the folder containing the files

void init_LayerConfig(const Eigen::VectorXd &optParams, const YAML::Node confParams)

Initialize configuration structure according to optimization vector and parameters file.

Parameters

  • optParams: Eigen Vector used by Limbo to define hyper-parameters

  • confParams: YAML Node containing hyper-parameters at high level

void init_LayerConfig(std::vector<stringdouble_t> paramValue)

Initialize configuration structure according to vector of key/value pairs.

Parameters

  • paramValue: vector of stringdouble_t pairs with key and value

std::vector<layerParameter> get_LayerConfig() const

Getter function for layerConfig as vector of layerParameter.

Return

std::vector<layerParameter> Layers configuration

int get_nLayers() const

Getter function for SWT WinIndex.

Return

std::vector<int> Number of layers

int get_fullNr(const int layer = -1) const

Return the sum of Nr across layers.

Return

int sum of Nr across layers

Parameters

  • layer: number of layers. Default -1 counts all layers

std::vector<ArrayI> get_WinIndex() const

Getter function for SWT WinIndex.

Return

std::vector<std::vector<int>> Win Index

std::vector<ArrayI> get_WoutIndex() const

Getter function for SWT WoutIndex.

Return

std::vector<std::vector<int>> Wout Index

int get_NrSWT(const int layer = -1) const

Return the sum of Nr across layers for SWT topology.

Return

int sum of SWT Nr across layers

Parameters

  • layer: number of layers. Default -1 counts all layers

int get_ReservoirType() const

Getter function for Reservoir type.

Return

int internal Reservoir type

void print_params(const int nLayers, const int nWashout, const double lambda)

Print optimizable parameters with a pretty table.

Parameters

  • nLayers: Number of layers

  • nWashout: Sample washout

  • lambda: Ridge regression factor see also EchoBay::Wout_ridge(int rows, int cols, double lambda, Eigen::Ref<MatrixBO> biasedState, Eigen::Ref<MatrixBO> target) for details

floatBO return_net_dimension(const YAML::Node confParams) const

Return memory occupation of the Reservoir object.

confParams YAML Node containing hyper-parameters at high level

Return

floatBO The sum along all the layers of the product between Nr and Density

DataStorage

class DataStorage

Data management class

DataStorage objects embeds all input data, labels and sampling vectors. It also includes data movement methods

Public Functions

void load_data(const std::string dataFile, const std::string labelFile, const uint8_t type)

Load data from files.

Parameters

  • dataFile: path to the data file

  • labelFile: path to the label file

  • type: differentiate training data (“train”) or validation/test (“valid”)

void copy_data(Eigen::Ref<MatrixBO> data, Eigen::Ref<MatrixBO> label, const uint8_t type)

Copy data from existing matrices.

Parameters

  • data: Eigen Matrix of data

  • label: Eigen Matrix of labels

  • type: differentiate training data (“train”) or validation/test (“valid”)

ArrayI8 set_sampleArray(Eigen::Ref<MatrixBO> samplingData, int nWashout, bool init_flag, const std::string &problemType, const uint8_t type)

Normalize sampling vector to be used in other functions.

Return

ArrayBO Cleaned sampling array

Parameters

  • samplingData: Eigen Matrix with sampling format TODO explain this

  • nWashout: Number of washout samples

  • init_flag: Flag that controls if washout resets also ESN states

  • problemType: Classification or Regression/Memory

  • type: train or validation

MatrixBO get_data(const uint8_t type, const uint8_t select) const

Return data or labels based on parameters.

Return

MatrixBO Selection

Parameters

  • type: train or validation

  • select: data or label

int get_dataCols(const uint8_t type) const

Get columns of _trainData or _evalData.

Return

int number of columns

Parameters

  • type: train or validation

int get_dataLength(const uint8_t type) const

Get length (rows) of _trainData or _evalData.

Return

int number of samples

Parameters

  • type: train or validation

ArrayI8 get_sampleArray(const uint8_t type) const

Return sampling array.

Return

ArrayBO Output sampling Array

Parameters

  • type: Train or validation

std::vector<ArrayI8> get_samplingBatches(const uint8_t type) const

Getter function for internal sampling vector.

Return

std::vector<ArrayBO> Internal sampling vector

int get_dataOffset(const uint8_t type, const int batch) const

Get linear offset to access data using batches.

Return

int Starting offset for that batch

Parameters

  • type: Train or validation

  • batch: 0-indexed batch position

int get_maxSamples(const uint8_t type, const int batch = -1) const

get number of sampled state across all batches

Return

int number of sampled states

Parameters

  • type: Train or validation

Comparator

class Comparator

Comparator class to calculate various fitness functions

Other than various label transformations, a Comparator object provides a common API (see EchoBay::Comparator::get_fitness(Eigen::Ref<MatrixBO> predict)) to template the calls to fitness functions.

Public Functions

Comparator(const std::string &problemType, const std::string &fitnessRule)

Construct a Comparator class for a given problem with a specific fitness function.

Parameters

  • problemType: Classification, Regression or MemoryCapacity

  • fitnessRule: fitness function dependent from the chosen problemType

void set_label_size(int rows, int cols)

Resize the internal outputLabel according to problemType.

Parameters

  • rows: Rows of the outputLabel

  • cols: Columns of the outputLabel, multiplied by nClasses in Classification problem (see also get_targetMatrix(const Eigen::Ref<const MatrixBO> label, const Eigen::Ref<const ArrayBO> sampleState))

floatBO get_fitness(Eigen::Ref<MatrixBO> predict)

Evaluate fitness function against internal targetLabel.

see also set_targetLabel(const Eigen::Ref<const MatrixBO> label, const Eigen::Ref<const ArrayBO> sampleState) for details on internal targetLabel

Return

floatBO Value of the fitness function evaluation

Parameters

  • predict: Input matrix with predicted values

void set_targetLabel(const Eigen::Ref<const MatrixBO> label, const Eigen::Ref<const ArrayI8> sampleState)

Set internal targetLabel according to Comparator problemType.

Classification problems use the same label and then calculate the number of classes. Regression and MemoryCapacity problems resize targetLabel and manage washout samples

Parameters

  • label: Eigen Matrix containing the desired target label

  • sampleState: Eigen Array containing the sampling index to manage ESN washout see also EchoBay::DataStorage::set_sampleArray(Eigen::Ref<MatrixBO> samplingData, int nWashout, bool init_flag, const std::string &problemType, const std::string type)

MatrixBO get_targetMatrix(const Eigen::Ref<const MatrixBO> label, const Eigen::Ref<const ArrayI8> sampleState)

Transform the training targetMatrix according to the problemType.

Classification problems use the same label and then perform one-hot encoding Regression and MemoryCapacity problems resize targetLabel and manage washout samples

Return

MatrixBO Problem dependent target matrix used for readout calculation see also MatrixBO EchoBay::Wout_ridge(int rows, int cols, double lambda, Eigen::Ref<MatrixBO> biasedState, Eigen::Ref<MatrixBO> target)

Parameters

  • label: Eigen Matrix containing the desired target label

  • sampleState: Eigen Array containing the sampling index to manage ESN washout see also EchoBay::DataStorage::set_sampleArray(Eigen::Ref<MatrixBO> samplingData, int nWashout, bool init_flag, const std::string &problemType, const std::string type)

void one_hot_encoding(Eigen::Ref<MatrixBO> Dst, MatrixBO Src)

Transform linear labeling in one-hot encoding format.

Parameters

  • Dst: Encoding destination matrix

  • Src: Input source label

int get_nClasses()

Return the internal nClasses variable.

Return

int Number of unique classes in the label

EchoBay namespace

ComputeState

These functions update the non-linear states of Echo State Networks.

Warning

doxygenfile: Found multiple matches for file “ComputeState.hpp

ESN

namespace EchoBay

Functions

ArrayBO esn_caller(const Eigen::VectorXd &optParams, YAML::Node confParams, const std::string outputFolder, const DataStorage &store, const bool guessEval, const std::string &computationType, const std::string &matrixFolder)

Main optimization function. Configure or load an Echo State Network.

Return

ArrayBO Return fitness value to the Bayesian Optimizer

Parameters

  • optParams: Eigen Vector used by Limbo to define hyper-parameters

  • confParams: YAML Node containing hyper-parameters at high level

  • outputFolder: Path to save all output files

  • store: DataStorage object containing training data and labels. See also EchoBay::DataStorage

  • guessEval: If true evaluates multiple guesses as the worst case, otherwise, get the mean of guesses

  • computationType: “train” performs ESN training, otherwise, validation and testing

  • matrixFolder: Path to the folder where Reservoir matrices will be saved or loaded

Reservoir esn_config(const Eigen::VectorXd &optParams, YAML::Node confParams, const int Nu, const int guess, const std::string folder)

Configure or load an Echo State Network.

Return

Reservoir Configured Reservoir object

Parameters

  • optParams: Eigen Vector used by Limbo to define hyper-parameters

  • confParams: YAML Node containing hyper-parameters at high level

  • Nu: Number of input channels

  • guess: Number of guess (used for output)

  • folder: Path to the folder where Reservoir matrices will be saved or loaded

ArrayBO esn_train(YAML::Node confParams, Reservoir &ESN, const double lambda, const std::string problemType, const std::string fitnessRule, const std::string outputFolder, const DataStorage &store, const bool saveflag, const int guesses)

Perform Echo State Network training.

Return

ArrayBO Return fitness value to the Bayesian Optimizer

Parameters

  • confParams: YAML Node containing hyper-parameters at high level

  • ESN: Reservoir Object

  • lambda: Regression lambda factor see EchoBay::readout_train(Reservoir &ESN, const MatrixBO &trainData, const Eigen::Ref<const ArrayBO> sampleState, Eigen::Ref<MatrixBO> target, double lambda, int blockStep)

  • problemType: Classification, Regression or MemoryCapacity

  • fitnessRule: Fitness function dependent from the chosen problemType

  • outputFolder: Path to the folder where the output will be saved

  • store: DataStorage object containing training data and labels. See also EchoBay::DataStorage

  • saveflag: If true, save the output and trained ESN matrices on a file

  • guesses: Number of guesses to be evaluated (used for output naming)

ArrayBO esn_compute(YAML::Node confParams, Reservoir &ESN, const DataStorage &store, const std::string &problemType, const std::string &fitnessRule, const int blockStep, Eigen::Ref<MatrixBO> Wout, const std::string &outputFolder, const bool saveflag, const int guesses)

Compute the output of a loaded Echo State Network.

Return

ArrayBO Return fitness value to the Bayesian Optimizer

Parameters

  • confParams: YAML Node containing hyper-parameters at high level

  • ESN: Reservoir Object

  • store: Datastorage container with the data to be processed

  • problemType: Classification, Regression or MemoryCapacity

  • fitnessRule: Fitness function dependent from the chosen problemType

  • blockStep: Dimension of the blocks used in Readout prediction

  • Wout: Readout matrix

  • outputFolder: Path to the folder where the output will be saved

  • saveflag: If true, save the output on a file

  • guesses: Number of guesses to be evaluated (used for output naming)

floatBO esn_compute(const MatrixBO &input_data, const std::string &folder)
MatrixBO compute_prediction(const int outNr, const int outDimension, const Eigen::Ref<MatrixBO> Wout, Reservoir &ESN, const MatrixBO &src, const Eigen::Ref<const ArrayBO> sampleState)

Utilities

Other headers are used for all the methods related to ESN management, like Eigen configuration, data IO, configuration utilities, CLI printing

EigenConfig

Modify this file to change floating point precision of the entire Library

Typedefs

typedef double floatBO

floatBO defines the floating point precision to be used in EchoBay

typedef Eigen::ArrayXd ArrayBO
typedef Eigen::ArrayXi ArrayI
typedef Eigen::Array<int8_t, Eigen::Dynamic, 1> ArrayI8
typedef Eigen::MatrixXd MatrixBO
typedef Eigen::VectorXcd VComplexBO
typedef Eigen::Triplet<floatBO> TripletBO
typedef Eigen::SparseMatrix<floatBO, 0x1> SparseBO
typedef std::unordered_set<std::pair<int, int>, pair_hash>::iterator UOMIterator
struct pair_hash
#include <EigenConfig.hpp>

Public Functions

template<class T1, class T2>
std::size_t operator()(const std::pair<T1, T2> &pair) const

LimboParams

Structures related to Limbo Bayesian Optimization

Functions

void save_time_samples(std::string folder, time_samples input)

Variables

YAML::Node configData
time_samples time_stats
struct time_samples
#include <LimboParams.hpp>

Public Members

std::chrono::high_resolution_clock::time_point tStart
std::vector<std::chrono::high_resolution_clock::time_point> tSamples
struct Params
#include <LimboParams.hpp>
struct init_gridsampling : public init_gridsampling
#include <LimboParams.hpp>

Public Functions

Params::init_gridsampling::BO_PARAM(int, bins, 10)
struct init_randomsampling : public init_randomsampling
#include <LimboParams.hpp>

Public Functions

BO_DYN_PARAM(int, samples)
struct kernel : public kernel
#include <LimboParams.hpp>

Public Functions

Params::kernel::BO_PARAM(double, noise, 0. 001)
struct opt_gridsearch : public opt_gridsearch
#include <LimboParams.hpp>

Public Functions

Params::opt_gridsearch::BO_PARAM(int, bins, 20)
struct stop_maxfitness
#include <LimboParams.hpp>

Public Functions

BO_DYN_PARAM(float, threshold)
struct stop_maxiterations : public stop_maxiterations
#include <LimboParams.hpp>

Public Functions

Params::stop_maxiterations::BO_PARAM(int, iterations, 40)
template<typename Params>
struct MaxFitness
#include <LimboParams.hpp>

Public Functions

MaxFitness()
template<typename BO, typename AggregatorFunction>
bool operator()(const BO &bo, const AggregatorFunction &afun)
template<typename Params>
struct NoVariation
#include <LimboParams.hpp>

Public Functions

NoVariation()
template<typename BO, typename AggregatorFunction>
bool operator()(const BO &bo, const AggregatorFunction &afun)
struct AggregatorMemoryOpt
#include <LimboParams.hpp>

Public Types

using result_type = double

Public Functions

double operator()(const Eigen::VectorXd &x) const
struct AggregatorNaive
#include <LimboParams.hpp>

Public Types

using result_type = double

Public Functions

double operator()(const Eigen::VectorXd &x) const
template<typename Params>
struct ParSampler
#include <LimboParams.hpp>

Public Functions

template<typename StateFunction, typename AggregatorFunction, typename Opt>
void operator()(const StateFunction &seval, const AggregatorFunction&, Opt &opt) const
template<typename StateFunction, typename Opt>
void _explore(int dim_in, const StateFunction &seval, const Eigen::VectorXd &current, Opt &opt) const

IOUtils

Manage input and output from disk

Defines

CSV_IO_NO_THREAD

Typedefs

typedef std::pair<std::string, std::string> stringpair_t

Functions

template<class Matrix>
void write_matrix(const std::string &filename, const Matrix &matrix)

Write a Eigen Matrix to file in binary format.

Template Parameters

  • Matrix: Class of Eigen Matrix to be written

Parameters

  • filename: Path to the file or filename

  • matrix: Input matrix

template<class Matrix>
void read_matrix(const std::string &filename, Matrix &matrix)

Read a Eigen Matrix from a file in binary format.

Template Parameters

  • Matrix: Class of Eigen Matrix to be written

Parameters

  • filename: Path to the file or filename

  • matrix: Output matrix

template<class Array>
void write_array(const std::string &filename, const Array &array)

Write a Eigen Array to file in binary format.

Template Parameters

  • Array: Class of Eigen Array to be written

Parameters

  • filename: Path to the file or filename

  • array: Input Array

template<class Array>
void read_array(const std::string &filename, Array &array)

Read a Eigen Array from a file in binary format.

Template Parameters

  • Array: Class of Eigen Array to be read

Parameters

  • filename: Path to the file or filename

  • array: Output Array

template<typename T>
std::vector<T> cat_vector(const std::vector<T> &a, const std::vector<T> &b)

Concatenate two std::vector.

Return

std::vector<T> Concatenated vector

Template Parameters

  • T: Template of the std::vector

Parameters

  • a: Vector a

  • b: Vector b

template<typename T>
std::vector<size_t> sort_indexes(const std::vector<T> &v)

Sort the indexes of a std::vector.

Return

std::vector<size_t> Sorted indexes

Template Parameters

  • T: Template of the std::vector

Parameters

  • v: Input vector

void save_config(const std::string &filename, YAML::Node YAML_CONF, const Eigen::VectorXd x)

Save current configuration on a portable YAML file.

Parameters

  • filename: Path to the file

  • YAML_CONF: Current configuration strucutred as YAML Node

  • x: Limbo configuration vector of hyper-parameters

std::string replace_tag(const std::string &input_string, int number)

Replace a {{x}} tag with a number in a string.

Return

std::string Output string

Parameters

  • inputString: Input string with or without {{x}} tag

  • number: Number to be applied

void load_csv(std::string filenameData, MatrixBO &arrValue)

Load a Eigen Matrix from a csv file.

Parameters

  • filename: Path to the input file

  • output: Eigen Matrix with loaded data

MatrixBO cat_matrix(const MatrixBO &a, const MatrixBO &b)

Concatenate two Eigen Matrices.

Return

MatrixBO Concatenated matrices

Parameters

  • a: Matrix a

  • b: Matrix b

ArrayBO cat_array(const ArrayBO &a, const ArrayBO &b)

Concatenate two Eigen Arrays.

Return

ArrayBO Concatenated arrays

Parameters

  • a: Array a

  • b: Array b

Variables

const std::vector<stringpair_t> tMapLayer{stringpair_t("Nr", ), , , , , }
const std::vector<stringpair_t> tMapGeneral{stringpair_t("Nl", ), , }

ConfigUtils

Map parameters in the YAML configuration to values in the Limbo vector

Typedefs

typedef std::map<std::string, Eigen::VectorXd> stringdouble_t

Functions

template<class T>
T parse_config(const std::string &paramName, const int layer, const int extra, const Eigen::VectorXd &optParams, const YAML::Node confParams, T defaultValue)

Map a value in the Limbo vector to proper range from the YAML configuration Node.

Return

T Mapped value

Template Parameters

  • T: type of returned value, int or float

Parameters

  • paramName: Name of the parameter in the YAML Node

  • layer: Layer position in a deep ESN configuration

  • extra: Number fo extra parameters see TODO add reference to Limbo vector structure

  • optParams: Eigen Vector used by Limbo to define hyper-parameters

  • confParams: YAML Node containing hyper-parameters at high level

  • defaultValue: Default value to be returned in the case that the parameter is not available in the YAML Node