Read Data

modulo_vki.utils.read_db.ReadData._data_processing(D: array, FOLDER_OUT: str = './', N_PARTITIONS: int = 1, MR: bool = False, SAVE_D: bool = False, FOLDER_IN: str = './', filename: str = '', h: int = 0, f: int = 0, c: int = 0, N: int = 0, N_S: int = 0, N_T: int = 0)

First, if the D matrix is not provided, this method attempts to load the data and assembles the D matrix. Then, it performs pre-processing operations on the data matrix, D. if MR=True, the mean (per each column - i.e.: snapshot at time t_i) is removed; If the MEMORY_SAVING=True the data matrix is splitted to optimize memory usage. Moreover, D is stored on disk and removed from the live memory. Finally, if in this condition, also the data type of the matrix is self is changed: from float64 -> float32, with the same purpose.

Parameters:

  • D – np.array data matrix D

  • FOLDER_OUT – str folder in which the data (partitions and/or data matrix itself) will be eventually saved.

  • MEMORY_SAVING – bool, optional If True, memory saving feature is activated. Passed through __init__

  • N_PARTITIONS – int In memory saving environment, this parameter refers to the number of partitions to be applied to the data matrix. If the number indicated by the user is not a multiple of the N_T i.e.: if (N_T % N_PARTITIONS) !=0 - then an additional partition is introduced, that contains the remaining columns

  • MR – bool, optional If True, it removes the mean (per column) from each snapshot

  • SAVE_D – bool, optional If True, the matrix D is saved into memory. If the Memory Saving feature is active, this is performed by default.

  • FOLDER_IN – str, optional. Needed only if database=None If the D matrix is not provided (database = None), read it from the path FOLDER_IN

  • filename – str, optional. Needed only if database=None If the database is not provided, read it from the files filename The files must be named “filenamexxxx.dat” where x is the number of the file that goes from 0 to the number of time steps saved

  • h – int, optional. Needed only if database=None Lines to be skipped from the header of filename

  • f – int, optional. Needed only if database=None Lines to be skipped from the footer of filename

  • c – int, optional. Needed only if database=None Columns to be skipped (for example if the first c columns contain the mesh grid.)

  • N – int, optional. Needed only if database=None Components to be analysed.

  • N_S – int, optional. Needed only if database=None Number of points in space.

  • N_T – int, optional. Needed only if database=None components to be analysed.

Returns:

There are four possible scenario:

  1. if N_Partitions ==1 and MR = True, return is D,D_MEAN (the mean snapshot!)

  2. if N_Partitions ==1 and MR = False, return is D.

  3. if N_Partitions >1 and MR = True, return is D_MEAN

  4. if N_Partitions >1 and MR=False, return is None