Convenience Loader#

In our research, we frequently load data structured in a specific way from files. To be more precise, usually, one parameter is varied over one experiment, while the other parameters are fixed. So, the files are separated by the value of the parameters that are fixed in each experiment, and the first column of each file is the independently varied parameter, while the other columns are the results of the experiment. This is a very common case in our research, and this class helps facilitate the repetitive task of loading such data.

class elecboltz.Loader(x_vary_label: str | Sequence[str] = None, x_search: Mapping[str, Sequence[int | float]] = {}, y_label: Sequence[str] = None, split_by: str = None, save_new_labels: bool = False, save_new_values: bool = False)#

Convenience class for loading data split into multiple files.

Prepares the data for fit_model by processing multiple files labeled with values of independent variables and containing the variation of one variable. For example, the files can be labeled with the values of phi and field, and contain the variation of theta.

Parameters:

x_vary_label (Union[str, Sequence[str]], optional) – Label of the independent variable(s) that varies inside the files. If not provided, it will be inferred from the file contents (column headers).
x_search (Mapping[str, Sequence[Union[int, float]]], optional) – Dictionary mapping labels of independent variables to the values to be searched for in the file names. Example: {'phi': [30, 60], 'field': [1.3, 2.7]}.
y_label (Sequence[str], optional) – Labels of the dependent variables, used for fitting routine. See elecboltz.fit.fit_model for details about the allowed labels. If None, will be inferred from the file contents (column headers).
split_by (str, optional) – If provided, the processed data will be split into separate sequences by this label. This is useful for fitting models that require separate data sets for different values of a variable (e.g. when fitting band parameters to different temperatures).
save_new_labels (bool, optional) – If True, new labels found in the file names will be saved to x_search. If there are multiple values for the new labels for each indicated label in x_search, then the values of those labels in x_search will be repeated for each new value of the new label. For example, if x_search is {'phi': [30, 60]} and files with label phi=30_field=1, phi=60_field=1, phi=30_field=2, and phi=60_field=2 are found, then x_search will be updated to {'phi': [30, 60, 30, 60], 'field': [1, 1, 2, 2]} (probably; it depends on the order that the files are read).
save_new_values (bool, optional) – If True, new values for existing labels in x_search will be added to the list of values for that label.

x_data#

x_data variable for the fitting procedure.

Type:: defaultdict[str, list[np.ndarray]]

y_data#

y_data variable for the fitting procedure.

Type:: defaultdict[str, list[np.ndarray]]

x_data_raw#

Raw data of the independent variable(s) collected from the files. Each x_vary_label is mapped to the corresponding arrays. Each array corresponds to a different value of the labels in x_search.

Type:: defaultdict[str, list[np.ndarray]]

y_data_raw#

Raw data of the dependent variable collected from the files. Each sequence corresponds to a different dependent variable in y_label, and each array inside that corresponds to a different value in x_search_values.

Type:: deefaultdict[str, list[np.ndarray]]

x_data_interpolated#

Interpolated, but unprocessed, data of the independent variable varying inside the files.

Type:: defaultdict[str, list[np.ndarray]]

y_data_interpolated#

Interpolated, but unprocessed, data of the dependent variable collected from the files.

Type:: defaultdict[str, list[np.ndarray]]

x_vary_label#

Label of the independent variable(s) that varies inside the files.

Type:: Union[str, Sequence[str]], optional

x_label#

Dictionary mapping labels of independent variables inside file names to their values.

Type:: Mapping[str, Union[int, float]]

loaded_files#

List of the files that were loaded.

Type:: list[pathlib.Path]

interpolate(n_points: int = 50, x_min: float = None, x_max: float = None, x_shift: float = None, x_normalize: float = None, y_shift: float = 0.0, y_normalize: float = 1.0)#

Interpolate the loaded data to the specified number of points.

For now, only supports linear interpolation for a single independent variable varying inside the files.

Parameters:

npoints (int, optional) – Number of points to interpolate to.
x_min (float, optional) – Lower boundary of the range for the independent variable (varying inside the files). If not provided, it is set to the minimum value of the independent variable in the loaded data.
x_max (float, optional) – Upper boundary of the range for the independent variable (varying inside the files). If not provided, it is set to the maximum value of the independent variable in the loaded data.
x_shift (float, optional) – If provided, the data will be shifted by the value at this point.
x_normalize (float, optional) – If provided, the data will be normalized by the value at this point. Note that shifts are applied before normalization.
y_shift (float, optional) – The data will be shifted to this value (if x_shift is provided).
y_normalize (float, optional) – The data will be normalized to this value (if x_normalize is provided).

load(folder_path: str = '.', prefix: str = '', recursive: bool = True, x_columns: Sequence[int] | Sequence[str] = None, y_columns: Sequence[int] | Sequence[str] = None, x_units: Sequence[float] | float = 1.0, y_units: Sequence[float] | float = 1.0, **kwargs)#

Load the data from files in the specified folder.

The function automatically determines which files to load based on the given parameters. The file names should be underline separated, with each part representing a different independent variable. You can put an equal sign or an underline between the variable name and its value, but it is not necessary if there is no ambiguity. If there is ambiguity, you must put an equal sign (an underline is not sufficient). For example, the file name can be admr_lsco_phi=30_field10_rho0=0.1.csv or admr_lsco_phi_30_field=10_rho0=0.1.csv.

Parameters:

folder_path (str, optional) – Path to the folder containing the data files.
prefix (str, optional) – Prefix for the data files.
recursive (bool, optional) – If True, search for files recursively in the folder and its subfolders. If False, search only in the specified folder.
x_columns (Union[Sequence[int], Sequence[str]], optional) – Override which columns to load for the independent variable. If a sequence of integers, it specifies the column indices to load. If a sequence of strings, it specifies the column names (headers) to load. If not provided, the first column is loaded as the independent variable.
y_columns (Union[Sequence[int], Sequence[str]], optional) – Override which columns to load for the dependent variables. If a sequence of integers, it specifies the column indices to load. If a sequence of strings, it specifies the column names (headers) to load. If not provided, all columns except the first one (independent variable) are loaded.
x_units (Union[Sequence[float], float], optional) – Units for the independent variable(s). If a single float is provided, it is applied to all independent variables. If a sequence is provided, it must match the number of independent variables.
y_units (Union[Sequence[float], float], optional) – Units for the dependent variable(s). If a single float is provided, it is applied to all dependent variables. If a sequence is provided, it must match the number of dependent variables.
**kwargs (dict, optional) – Additional keyword arguments to pass to numpy.loadtxt.

process_data()#

Process the loaded data to prepare it for fitting.

Fills the x_data and y_data attributes with the correct values. This is run at the end of the load and interpolate methods. You can use it if you do extra processing on the data after loading or interpolating.