DataFile.hpp

DataFile objects use DataNode objects to dynamically fill out file contents.

Functions

template<typename CONTAINER> ContainerDataFile<CONTAINER> MakeContainerDataFile(std::function<CONTAINER(void)> fun, const std::string &filename, const std::string &b = "", const std::string &s = ",", const std::string &e = "\n")

Convenience function for building a container data file.

Parameters:

fun – is the function to call to update the container
filename – is the name of the file to output to
b – character to print at the beggininning of each line
s – delimeter (i.e. what character to print between entries on each line)
e – character to print at the end of each line

class DataFile

#include <DataFile.hpp>

Keep track of everything associated with periodically printing data to a file. Maintain a set of functions for calculating the desired measurements at each point in time that they are required. It also handles the formating of the file.

Public Functions

inline DataFile(const std::string &in_filename, const std::string &b = "", const std::string &s = ",", const std::string &e = "\n")

inline DataFile(std::ostream &in_os, const std::string &b = "", const std::string &s = ",", const std::string &e = "\n")

DataFile(const DataFile&) = default

DataFile(DataFile&&) = default

inline virtual ~DataFile()

DataFile &operator=(const DataFile&) = default

DataFile &operator=(DataFile&&) = default

inline const std::string &GetFilename() const: Get the filename used for this file.

inline const std::string &GetLineBegin() const: Returns the string that is printed at the beginning of each line.

inline const std::string &GetSpacer() const: Returns the string that is printed between elements on each line (i.e. the delimeter).

inline const std::string &GetLineEnd() const: Returns the string that is printed at the end of each line.

inline void SetTiming(time_fun_t fun): Provide a timing function with a bool(size_t update) signature. The timing function is called with the update, and returns a bool to indicate if files should print this update.

inline void SetTimingOnce(size_t print_time)

Setup this file to print only once, at the specified update. Note that this timing function can be replaced at any time, even after being triggered.

Parameters:: print_time – The update the file should print at

inline void SetTimingRepeat(size_t step): Setup this file to print every ‘step’ updates.

inline void SetTimingRange(size_t first, size_t step, size_t last): Setup this file to print only in a specified time range, and a given frequency (step).

inline void SetLineBegin(const std::string &_in)

Print a string at the beginning of each line.

Parameters:: _in – the string to print

inline void SetSpacer(const std::string &_in)

Print a string between elements.

Parameters:: _in – the string (i.e. the delimeter).

inline void SetLineEnd(const std::string &_in)

Print a string at the end of each line.

Parameters:: _in – the string to print

inline void SetupLine(const std::string &b, const std::string &s, const std::string &e)

Set line begin, delimeter, and line end characters.

Parameters:

b – line-begin character.
s – column delimeter
e – line end character

inline virtual void PrintHeaderKeys(): Print a header containing the name of each column.

inline virtual void PrintHeaderComment(const std::string &cstart = "# "): Print a header containing comments describing all of the columns.

inline virtual void Update(): Run all of the functions and print the results as a new line in the file.

inline virtual void Update(size_t update)

If Update is called with an update number, the full version of update is called only if the update value passes the timing function (that is, the timing function returns true).

Parameters:: update – the current time step

inline void AddPreFun(pre_fun_t fun)

Add a function to run before each time data is calculated. You can add as many functions as you want.

Parameters:: fun – the function to add

inline size_t Add(const std::function<void(std::ostream&)> &fun, const std::string &key, const std::string &desc)

Generic function for adding a column to the DataFile. In practice, you probably want to call one of the more specific ones.

Parameters:

fun – function to call to get the value to print
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)

template<typename T> inline size_t AddFun(const std::function<T()> &fun, const std::string &key = "", const std::string &desc = "")

Add a function that returns a value to be printed to the file.

Parameters:

fun – function to call to get the value to print
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)

template<typename T> inline size_t AddVar(const T &var, const std::string &key = "", const std::string &desc = "")

Add a function that always prints the current value of var.

Parameters:

var – variable to print value of
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)

template<typename VAL_TYPE, data... MODS> inline size_t AddCurrent(DataNode<VAL_TYPE, MODS...> &node, const std::string &key = "", const std::string &desc = "", const bool &reset = false, const bool &pull = false)

Add a function that always pulls the current value from the DataNode node. Requires that node have the data::Current modifier.

Parameters:

node – DataNode to get data from
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)
reset – If this parameter is set true, we will call Reset on that DataNode after pulling the current value from the node
pull – Should the node pull data before this statistic is calculated?

template<typename VAL_TYPE, data... MODS> inline size_t AddMean(DataNode<VAL_TYPE, MODS...> &node, const std::string &key = "", const std::string &desc = "", const bool &reset = false, const bool &pull = false)

Add a function that always pulls the mean value from the DataNode node. Requires that node have the data::Range or data::FullRange modifier.

Parameters:

node – DataNode to get data from
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)
reset – If this parameter is set true, we will call Reset on that DataNode after pulling the current value from the node
pull – Should the node pull data before this statistic is calculated? template <typename VAL_TYPE, data… MODS>

template<typename VAL_TYPE, data... MODS> inline size_t AddMedian(DataNode<VAL_TYPE, MODS...> &node, const std::string &key = "", const std::string &desc = "", const bool &reset = false, const bool &pull = false)

Add a function that always pulls the median value from the DataNode node. Requires that node have the data::Range or data::FullRange modifier.

Parameters:

node – DataNode to get data from
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)
reset – If this parameter is set true, we will call Reset on that DataNode after pulling the current value from the node
pull – Should the node pull data before this statistic is calculated?

template<typename VAL_TYPE, data... MODS> inline size_t AddPercentile(DataNode<VAL_TYPE, MODS...> &node, const double pct = 100.0, const std::string &key = "", const std::string &desc = "", const bool &reset = false, const bool &pull = false)

Add a function that always pulls the Percentile value from the DataNode node. Requires that node have the data::Range or data::FullRange modifier.

Parameters:

node – DataNode to get data from
pct – the percentile value to return
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)
reset – If this parameter is set true, we will call Reset on that DataNode after pulling the current value from the node
pull – Should the node pull data before this statistic is calculated?

template<typename VAL_TYPE, data... MODS> inline size_t AddTotal(DataNode<VAL_TYPE, MODS...> &node, const std::string &key = "", const std::string &desc = "", const bool &reset = false, const bool &pull = false)

Add a function that always pulls the total value from the DataNode node. Requires that node have the data::Range or data::FullRange modifier.

Parameters:

node – DataNode to get data from
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)
reset – If this parameter is set true, we will call Reset on that DataNode after pulling the current value from the node
pull – Should the node pull data before this statistic is calculated?

template<typename VAL_TYPE, data... MODS> inline size_t AddMin(DataNode<VAL_TYPE, MODS...> &node, const std::string &key = "", const std::string &desc = "", const bool &reset = false, const bool &pull = false)

Add a function that always pulls the minimum value from the DataNode node. Requires that node have the data::Range or data::FullRange modifier.

Parameters:

node – DataNode to get data from
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)
reset – If this parameter is set true, we will call Reset on that DataNode after pulling the current value from the node
pull – Should the node pull data before this statistic is calculated?

template<typename VAL_TYPE, data... MODS> inline size_t AddMax(DataNode<VAL_TYPE, MODS...> &node, const std::string &key = "", const std::string &desc = "", const bool &reset = false, const bool &pull = false)

Add a function that always pulls the maximum value from the DataNode node. Requires that node have the data::Range or data::FullRange modifier.

Parameters:

node – DataNode to get data from
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)
reset – If this parameter is set true, we will call Reset on that DataNode after pulling the current value from the node
pull – Should the node pull data before this statistic is calculated?

template<typename VAL_TYPE, data... MODS> inline size_t AddVariance(DataNode<VAL_TYPE, MODS...> &node, const std::string &key = "", const std::string &desc = "", const bool &reset = false, const bool &pull = false)

Add a function that always pulls the variance from the DataNode node Requires that node have the data::Stats or data::FullStats modifier.

Parameters:

node – DataNode to get data from
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)
reset – If this parameter is set true, we will call Reset on that DataNode after pulling the current value from the node
pull – Should the node pull data before this statistic is calculated?

template<typename VAL_TYPE, data... MODS> inline size_t AddStandardDeviation(DataNode<VAL_TYPE, MODS...> &node, const std::string &key = "", const std::string &desc = "", const bool &reset = false, const bool &pull = false)

Add a function that always pulls the standard deviation from the DataNode node Requires that node have the data::Stats or data::FullStats modifier.

Parameters:

node – DataNode to get data from
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)
reset – If this parameter is set true, we will call Reset on that DataNode after pulling the current value from the node
pull – Should the node pull data before this statistic is calculated?

template<typename VAL_TYPE, data... MODS> inline size_t AddSkew(DataNode<VAL_TYPE, MODS...> &node, const std::string &key = "", const std::string &desc = "", const bool &reset = false, const bool &pull = false)

Add a function that always pulls the skewness from the DataNode node Requires that node have the data::Stats or data::FullStats modifier.

Parameters:

node – DataNode to get data from
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)
reset – If this parameter is set true, we will call Reset on that DataNode after pulling the current value from the node
pull – Should the node pull data before this statistic is calculated?

template<typename VAL_TYPE, data... MODS> inline size_t AddKurtosis(DataNode<VAL_TYPE, MODS...> &node, const std::string &key = "", const std::string &desc = "", const bool &reset = false, const bool &pull = false)

Add a function that always pulls the kurtosis from the DataNode node Requires that node have the data::Stats or data::FullStats modifier.

Parameters:

node – DataNode to get data from
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)
reset – If this parameter is set true, we will call Reset on that DataNode after pulling the current value from the node
pull – Should the node pull data before this statistic is calculated?

template<typename VAL_TYPE, data... MODS> inline void AddStats(DataNode<VAL_TYPE, MODS...> &node, const std::string &key = "", const std::string &desc = "", const bool &reset = false, const bool &pull = false)

Add multiple functions that pull common stats measurements (mean, variance, min, and max) from the DataNode node. Requires that node have the data::Stats or data::FullStats modifier. key and desc will have the name of the stat appended to the beginning. Note: excludes standard deviation, because it is easily calculated from variance Note: Setting pull and/or reset to true only pulls on first statistic calculated and only resets on the last. Otherwise there would be a risk of data loss or at least massive replication of computational effort. Even still, think carefully before setting either of these to true when you’re drawing varied information from the same node.

Parameters:

node – DataNode to get data from
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)
reset – If this parameter is set true, we will call Reset on that DataNode after pulling the current value from the node
pull – Should the node pull data before this statistic is calculated?

template<typename VAL_TYPE, data... MODS> inline void AddAllStats(DataNode<VAL_TYPE, MODS...> &node, const std::string &key = "", const std::string &desc = "", const bool &reset = false, const bool &pull = false)

Add multiple functions that pull all stats measurements (mean, variance, min, max, skew, and kurtosis) from the DataNode node Requires that node have the data::Stats or data::FullStats modifier. key and desc will have the name of the stat appended to the beginning. Note: excludes standard deviation, because it is easily calculated from variance Note: Setting pull and/or reset to true only pulls on first statistic calculated and only resets on the last. Otherwise there would be a risk of data loss or at least massive replication of computational effort. Even still, think carefully before setting either of these to true when you’re drawing varied information from the same node.

Parameters:

node – DataNode to get data from
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)
reset – If this parameter is set true, we will call Reset on that DataNode after pulling the current value from the node
pull – Should the node pull data before this statistic is calculated?

template<typename VAL_TYPE, data... MODS> inline size_t AddHistBin(DataNode<VAL_TYPE, MODS...> &node, size_t bin_id, const std::string &key = "", const std::string &desc = "", const bool &reset = false, const bool &pull = false)

Add a function that always pulls the count of the bin_id ‘th bin of the histogram from node. Requires that node have the data::Histogram modifier and at least bin_id bins.

Parameters:

node – DataNode to get data from
bin_id – id of bin to log data from (0-indexed)
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)
reset – If this parameter is set true, we will call Reset on that DataNode after pulling the current value from the node
pull – Should the node pull data before this statistic is calculated?

template<typename VAL_TYPE, data... MODS> inline void AddAllHistBins(DataNode<VAL_TYPE, MODS...> &node, const std::string &key = "", const std::string &desc = "", const bool &reset = false, const bool &pull = false)

Add a set of functions that always pull the count of each bin of the histogram from node. Requires that node have the data::Histogram modifier. Note: Setting pull and/or reset to true only pulls on first statistic calculated and only resets on the last. Otherwise there would be a risk of data loss or at least massive replication of computational effort. Even still, think carefully before setting either of these to true when you’re drawing varied information from the same node.

Parameters:

node – DataNode to get data from
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)
reset – If this parameter is set true, we will call Reset on that DataNode after pulling the current value from the node
pull – Should the node pull data before this statistic is calculated?

template<typename VAL_TYPE, data... MODS> inline size_t AddInferiority(DataNode<VAL_TYPE, MODS...> &node, const std::string &key = "", const std::string &desc = "", const bool &reset = false, const bool &pull = false)

Add a function that always pulls the inferiority (mean divided by max) from the specified DataNode. Requires that node have the data::Range or data::FullRange modifier.

Parameters:

node – DataNode to get data from
key – Keyword associated with this column (gets used as a column name for this data)
desc – Full description of this data (used by PrintHeaderComment)
reset – If this parameter is set true, we will call Reset on that DataNode after pulling the current value from the node
pull – Should the node pull data before this statistic is calculated?

Protected Types

using fun_t = void(std::ostream&)

using time_fun_t = std::function<bool(size_t)>

using pre_fun_t = std::function<void()>

Protected Attributes

std::string filename: Name of the file that we are printing to (if one exists)

std::ostream *os: Stream to print to.

FunctionSet<fun_t> funs: Set of functions to call, one per column in the file.

FunctionSet<void()> pre_funs: Set of functions to call before calculating data.

vector<std::string> keys: Keywords associated with each column.

vector<std::string> descs: Full description for each column.

time_fun_t timing_fun: Function to determine updates to print on (default: all)

std::string line_begin: What should we print at the start of each line?

std::string line_spacer: What should we print between entries?

std::string line_end: What should we print at the end of each line?

template<typename CONTAINER> class ContainerDataFile

#include <DataFile.hpp>

Sometimes you may want a data file where a set of functions is run on every item in a container every time you write to the file. ContainerDataFiles do that.

Note: CONTAINER type can be a pointer to a container and the datafile will handle dereferencing it appropriately.

Public Functions

template<typename ...ARGS> inline explicit ContainerDataFile(ARGS&&... arguments)

inline ~ContainerDataFile()

inline void SetUpdateContainerFun(const fun_update_container_t fun): Tell this file what function to run to update the contents of the container that data is being calculated on.

inline void PrintHeaderKeys() override: Print a header containing the name of each column.

inline void PrintHeaderComment(const std::string &cstart = "# ") override: Print a header containing comments describing all of the columns.

inline const container_t GetCurrentRows() const

inline void OutputLine(const data_t d)

inline void Update() override: Run all of the functions and print the results as a new line in the file.

inline void Update(size_t update) override: Update the file with an additional set of lines.

inline size_t Add(const std::function<void(std::ostream&, data_t)> &fun, const std::string &key, const std::string &desc): If a function takes an ostream, pass in the correct one. Generic function for adding a column to the DataFile. In practice, you probably want to call one of the more specific ones.

template<typename T> inline size_t AddContainerFun(const std::function<T(const data_t)> &fun, const std::string &key = "", const std::string &desc = ""): Add a function that returns a value to be printed to the file.

Protected Types

using container_t = typename std::remove_reference<CONTAINER>::type

using raw_container_t = typename remove_ptr_type<container_t>::type

using data_t = typename raw_container_t::value_type

using container_fun_t = void(std::ostream&, data_t)

using fun_update_container_t = std::function<container_t(void)>

Protected Attributes

fun_update_container_t update_container_fun

container_t current_rows

FunctionSet<container_fun_t> container_funs

vector<std::string> container_keys

vector<std::string> container_descs

template<typename CONTAINER> class ContainerDataFile

#include <DataFile.hpp>

Sometimes you may want a data file where a set of functions is run on every item in a container every time you write to the file. ContainerDataFiles do that.

Note: CONTAINER type can be a pointer to a container and the datafile will handle dereferencing it appropriately.

Public Functions

template<typename ...ARGS> inline explicit ContainerDataFile(ARGS&&... arguments)

inline ~ContainerDataFile()

inline void SetUpdateContainerFun(const fun_update_container_t fun): Tell this file what function to run to update the contents of the container that data is being calculated on.

inline void PrintHeaderKeys() override: Print a header containing the name of each column.

inline void PrintHeaderComment(const std::string &cstart = "# ") override: Print a header containing comments describing all of the columns.

inline const container_t GetCurrentRows() const

inline void OutputLine(const data_t d)

inline void Update() override: Run all of the functions and print the results as a new line in the file.

inline void Update(size_t update) override: Update the file with an additional set of lines.

inline size_t Add(const std::function<void(std::ostream&, data_t)> &fun, const std::string &key, const std::string &desc): If a function takes an ostream, pass in the correct one. Generic function for adding a column to the DataFile. In practice, you probably want to call one of the more specific ones.

template<typename T> inline size_t AddContainerFun(const std::function<T(const data_t)> &fun, const std::string &key = "", const std::string &desc = ""): Add a function that returns a value to be printed to the file.

Protected Types

using container_t = typename std::remove_reference<CONTAINER>::type

using raw_container_t = typename remove_ptr_type<container_t>::type

using data_t = typename raw_container_t::value_type

using container_fun_t = void(std::ostream&, data_t)

using fun_update_container_t = std::function<container_t(void)>

Protected Attributes

fun_update_container_t update_container_fun

container_t current_rows

FunctionSet<container_fun_t> container_funs

vector<std::string> container_keys

vector<std::string> container_descs