TS_Normalize
. This involves filtering out operations or time series that produced many errors or special-valued outputs, and then normalizing of the output of all operations, which is typically done in-sample, according to an outlier-robust sigmoidal transform (although other normalizing transformations can be selected). Both of these tasks are performed using the function TS_Normalize
. The TS_Normalize
function writes the new, filtered, normalized matrix to a local file called HCTSA_N.mat
. This contains normalized, and trimmed versions of the information in HCTSA.mat
.'mixedSigmoid'
. The second input controls the filtering of time series and operations based on minimum thresholds for good values in the corresponding rows (corresponding to time series; filtered first) and columns (corresponding to operations; filtered second) of the data matrix.HCTSA_N.mat
.HCTSA_N.mat
file as normalizationInfo
, a structure that contains the normalization function, filtering options used, and the corresponding TS_Normalize
code that can be used to re-run the normalization.BF_NormalizeMatrix
.[0.7,0.9]
, removes time series with less than 70% good values, and then removes operations with less than 90% good values. Some applications can tolerate some special-valued outputs from operations (like some clustering methods, where distances are simply calculated using those operations that are did not produce special-valued outputs for each pair of objects), but others cannot (like Principal Components Analysis); the filtering parameters should be specified accordingly.HCTSA_N.mat
in the knowledge that different settings for filtering and normalizing the results can be applied at any time by simply rerunning TS_Normalize
, which will overwrite the existing HCTSA_N.mat
with the results of the new normalization and filtration settings.TS_FilterData
'lengthdep'
, 'locdep'
, and 'spreaddep'
, you can easily filter these out to check the robustness of your analysis.'locdep'
or 'spreaddep'
features (or any other combination of keyword labels). You can then go ahead with analyzing the filtered HCTSA dataset as above, except using your new filename, HCTSA_locFilt
.