Multivariate Entropies

Functions for estimating the entropy of a multivariate dataset.

The following functions also form the base entropy method used by multivariate multiscale entropy functions.

MvCoSiEn(Data, varargin)

MvCoSiEn estimates the multivariate cosine similarity entropy of a multivariate dataset.

[MCoSi, Bm] = MvCoSiEn(Data)

Returns the multivariate cosine similarity entropy estimate (MCoSi) and the corresponding global probabilities (Bm) estimated for the M multivariate sequences in Data using the default parameters: embedding dimension = 2*ones(M,1), time delay = ones(M,1), angular threshold = 0.1, logarithm = 2, data normalization = none,

Note

To maximize the number of points in the embedding process, this algorithm uses N-max(m*tau) delay vectors and not N-max(m)*max(tau) as employed in [1][2][3].

[MCoSi, Bm] = MvCoSiEn(Data, name, value, …)

Returns the multivariate cosine similarity entropy estimates (MCoSi) estimated from the M multivariate data sequences in Data using the specified name/value pair arguments:

Data:

Multivariate dataset, NxM matrix of N (>10) observations (rows) and M (cols) univariate data sequences

m:

Embedding Dimension, a vector of M positive integers

tau:

Time Delay, a vector of M positive integers

r:

Angular threshold, a value in range [0 < r < 1]

Logx:

Logarithm base, a positive scalar (enter 0 for natural log)

Norm:

Normalisation of Data, one of the following integers

0:

no normalisation (default)

1:

remove median(Data) to get zero-median series

2:

remove mean(Data) to get zero-mean series

3:

normalises each sequence in Data to unit variance and zero mean

4:

normalises each sequence in Data values to range [-1 1]

See also:

CoSiEn, MvMSEn, MvSampEn, MvDispEn, MvFuzzEn, MvPermEn, cMvMSEn

References:

[1] H. Xiao, T. Chanwimalueang and D. P. Mandic,: “Multivariate Multiscale Cosine Similarity Entropy” IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5997-6001, doi: 10.1109/ICASSP43922.2022.9747282.
[2] Xiao, H.; Chanwimalueang, T.; Mandic, D.P.,: “Multivariate Multiscale Cosine Similarity Entropy and Its Application to Examine Circularity Properties in Division Algebras.” Entropy 2022, 24, 1287.
[3] Ahmed Mosabber Uddin, Danilo P. Mandic: “Multivariate multiscale entropy: A tool for complexity analysis of multichannel data.” Physical Review E 84.6 (2011): 061918.
[4] Theerasak Chanwimalueang and Danilo Mandic,: “Cosine similarity entropy: Self-correlation-based complexity analysis of dynamical systems.” Entropy 19.12 (2017): 652.

MvDispEn(Data, varargin)

MvDispEn estimates the multivariate dispersion entropy of a multivariate dataset.

[MDisp, RDE] = MvDispEn(Data)

Returns the multivariate dispersion entropy estimate (MDisp) and the reverse dispersion entropy (RDE) for the M multivariate sequences in Data using the default parameters: embedding dimension = 2*ones(M,1), time delay = ones(M,1), # symbols = 3, algorithm method = ‘v1’ (see below), data transform = normalised cumulative density function (ncdf) logarithm = natural, entropy normalization = true,

Important

By default, MvDispEn uses the method termed mvDEii in [1], which follows the original multivariate embedding approach of Ahmed & Mandic [2]. The v1 method therefore returns a singular entropy estimate.

If the v2 method is selected (Methodx=='v2'), the main method outlined in [1] termed mvDE is applied. In this case, entropy is estimated using each combination of multivariate delay vectors with lengths 1:max(m), with each entropy value returned accordingly. See [1] for more info.

[MDisp, RDE] = MvDispEn(Data, name, value, …)

Returns the multivariate dispersion entropy estimate (MDisp) for the M multivariate data sequences in Data using the specified name/value pair arguments:

Data:

Multivariate dataset, NxM matrix of N (>10) observations (rows) and M (cols) univariate data sequences

m:

Embedding Dimension, a vector of M positive integers

tau:

Time Delay, a vector of M positive integers

c:

Number of symbols in transform, an integer > 1

Methodx:

The method of multivariate dispersion entropy estimation as outlined in [1], either

‘v1’:

employs the method consistent with the original multivariate embedding approach of Ahmed & Mandic [2], termed mvDEii in [1]. (default)

‘v2’:

employs the main method derived in [1], termed mvDE.

Typex:

Type of data-to-symbolic sequence transform, one of the following {linear, kmeans, ncdf, equal}

See the EntropyHub Guide for more info on these transforms.

Logx:

Logarithm base, a positive scalar

Norm:

Normalisation of MDisp and RDE values, a boolean:

false:

no normalisation (default)

true:

normalises w.r.t number of possible dispersion patterns (c^m).

See also:

DispEn, DispEn2D, MvSampEn, MvFuzzEn, MvPermEn, MvMSEn, cMvMSEn,

References:

[1] H Azami, A Fernández, J Escudero: “Multivariate Multiscale Dispersion Entropy of Biomedical Times Series” Entropy 2019, 21, 913.
[2] Ahmed Mosabber Uddin, Danilo P. Mandic: “Multivariate multiscale entropy: A tool for complexity analysis of multichannel data.” Physical Review E 84.6 (2011): 061918.
[3] Mostafa Rostaghi and Hamed Azami,: “Dispersion entropy: A measure for time-series analysis.” IEEE Signal Processing Letters 23.5 (2016): 610-614.
[4] Hamed Azami and Javier Escudero,: “Amplitude-and fluctuation-based dispersion entropy.” Entropy 20.3 (2018): 210.
[5] Li Yuxing, Xiang Gao and Long Wang,: “Reverse dispersion entropy: A new complexity measure for sensor signal.” Sensors 19.23 (2019): 5203.

MvFuzzEn(Data, varargin)

MvFuzzEn estimates the multivariate fuzzy entropy of a multivariate dataset.

[MFuzz, B0, Bt, B1] = MvFuzzEn(Data)

Returns the multivariate fuzzy entropy estimate (MFuzz) and the average vector distances (m: B0; joint total m+1 subspace: Bt; all possible m+1 subspaces: B1), from the M multivariate sequences in Data using the default parameters: embedding dimension = 2*ones(M,1), time delay = ones(M,1), fuzzy membership function = “default”, fuzzy function parameters= [0.2, 2], logarithm = natural, data normalization = false,

Attention

The entropy value returned as MFuzz is estimated using the “full” method [i.e. -log(Bt/B0)] which compares delay vectors across all possible m+1 expansions of the embedding space as applied in [1][3]. Contrary to conventional definitions of fuzzy entropy, this method does not provide a lower bound of 0!! Thus, it is possible to obtain negative entropy values for multivariate fuzzy entropy, even for stochastic processes…

Alternatively, one can calculate MFuzz via the “naive” method, which ensures a lower bound of 0, by using the average vector distances for an individual m+1 subspace (B1) [e.g. -log(B1(1)/B0)], or the average for all m+1 subspaces [i.e. -log(mean(B1)/B0)].

Note

To maximize the number of points in the embedding process, this algorithm uses N-max(m*tau) delay vectors and not N-max(m)*max(tau) as employed in [1] and [3].

[MFuzz, B0, Bt, B1] = MvFuzzEn(Data, name, value, …)

Returns the multivariate fuzzy entropy estimates (MFuzz) estimated from the M multivariate data sequences in Data using the specified name/value pair arguments:

Data:

Multivariate dataset, NxM matrix of N (>10) observations (rows) and M (cols) univariate data sequences

m:

Embedding Dimension, a vector of M positive integers

tau:

Time Delay, a vector of M positive integers

Fx:

Fuzzy function name, one of the following strings, {'sigmoid', 'modsampen', 'default', 'gudermannian', 'bell', 'triangular', 'trapezoidal1', 'trapezoidal2', 'z_shaped', 'gaussian', 'constgaussian'}

r:

Fuzzy function parameters, a 1 element scalar or a 2 element vector of positive values. The r parameters for each fuzzy function are defined as follows: (default: [.2 2])

sigmoid:

r(1) = divisor of the exponential argument

r(2) = value subtracted from argument (pre-division)

modsampen:

r(1) = divisor of the exponential argument

r(2) = value subtracted from argument (pre-division)

default:

r(1) = divisor of the exponential argument

r(2) = argument exponent (pre-division)

gudermannian:

r = a scalar whose value is the numerator of

argument to gudermannian function: GD(x) = atan(tanh(r/x)). GD(x) is normalised to have a maximum value of 1.

triangular:

r = a positive scalar whose value is the threshold (corner point) of the triangular function.

trapezoidal1:

r = a positive scalar whose value corresponds to the upper (2r) and lower (r) corner points of the trapezoid.

trapezoidal2:

r(1) = a value corresponding to the upper corner point of the trapezoid.

r(2) = a value corresponding to the lower corner point of the trapezoid.

z_shaped:

r = a scalar whose value corresponds to the upper (2r) and lower (r) corner points of the z-shape.

bell:

r(1) = divisor of the distance value

r(2) = exponent of generalized bell-shaped function

gaussian:

r = a scalar whose value scales the slope of the Gaussian curve.

constgaussian:

r = a scalar whose value defines the lower threshod and shape of the Gaussian curve.

Norm:

Normalisation of all M sequences to unit variance, a boolean

Logx:

Logarithm base, a positive scalar [default: natural]

For further information on the name/value paire arguments, see the EntropyHub guide

See also:

MvSampEn, FuzzEn, XFuzzEn, FuzzEn2D, MSEn, MvPermEn.

References:

[1] Ahmed, Mosabber U., et al.: “A multivariate multiscale fuzzy entropy algorithm with application to uterine EMG complexity analysis.” Entropy 19.1 (2016): 2.
[2] Azami, Alberto Fernández, Javier Escudero.: “Refined multiscale fuzzy entropy based on standard deviation for biomedical signal analysis.” Medical & biological engineering & computing 55 (2017): 2037-2052.
[3] Ahmed Mosabber Uddin, Danilo P. Mandic: “Multivariate multiscale entropy analysis.” IEEE signal processing letters 19.2 (2011): 91-94.

MvPermEn(Data, varargin)

MvPermEn estimates the multivariate permutation entropy of a multivariate dataset.

[MPerm MPnorm] = MvPermEn(Data)

Returns the multivariate permutation entropy estimate (MPerm) and the normalized permutation entropy for the M multivariate sequences in Data using the default parameters: embedding dimension = 2*ones(M,1), time delay = ones(M,1), logarithm = 2, normalisation = w.r.t #symbols (sum(m-1))

Attention

The multivariate permutation entropy algorithm implemented here uses multivariate embedding based on Takens’ embedding theorem, and follows the methods for multivariate entropy estimation through shared spatial reconstruction as originally presented by Ahmed & Mandic [1].

This function does NOT use the multivariate permutation entropy algorithm of Morabito et al. (Entropy, 2012) where the entropy values of individual univariate sequences are averaged because such methods do not follow the definition of multivariate embedding and therefore do not consider cross-channel statistical complexity.

Note

To maximize the number of points in the embedding process, this algorithm uses N-max(tau*m) delay vectors and not N-max(m)*max(tau) as employed in [1].

[MPerm, MPnorm] = MvPermEn(Data, name, value, …)

Returns the multivariate permutation entropy estimate (MPerm) for the M multivariate data sequences in Data using the specified name/value pair arguments:

Data:

Multivariate dataset, NxM matrix of N (>10) observations (rows) and M (cols) univariate data sequences

m:

Embedding Dimension, a vector of M positive integers

tau:

Time Delay, a vector of M positive integers

Typex:

Permutation entropy variation, can be one of the following strings: {'modified', 'ampaware', 'weighted', 'edge', 'phase'}

See the EntropyHub guide for more info on PermEn variants.

tpx:

Tuning parameter for associated permutation entropy variation.

ampaware:

tpx is the A parameter, a value in range [0 1] (default = 0.5)

edge:

tpx is the r sensitivity parameter, a scalar > 0 (default = 1)

phase:

tpx is the option to unwrap the phase angle of Hilbert-transformed signal, either [] or 1 (default = 0)

Norm:

Normalisation of MPnorm value, a boolean operator:

false:

normalises w.r.t log(# of permutation symbols [sum(m)-1]) - default

true:

normalises w.r.t log(# of all possible permutations [sum(m)!])

Logx:

Logarithm base, a positive scalar (defualt = 2; enter 0 for natural logarithm)

See also:

PermEn, PermEn2D, XPermEn, MvMSEn, MvFuzzEn, MvSampEn, MvDispEn, cMvMSEn.

References:

[1] Ahmed Mosabber Uddin, Danilo P. Mandic: “Multivariate multiscale entropy: A tool for complexity analysis of multichannel data.” Physical Review E 84.6 (2011): 061918.
[2] Christoph Bandt and Bernd Pompe,: “Permutation entropy: A natural complexity measure for time series.” Physical Review Letters, 88.17 (2002): 174102.
[3] Chunhua Bian, et al.,: “Modified permutation-entropy analysis of heartbeat dynamics.” Physical Review E 85.2 (2012) : 021906
[4] Bilal Fadlallah, et al.,: “Weighted-permutation entropy: A complexity measure for time series incorporating amplitude information.” Physical Review E 87.2 (2013): 022911.
[5] Hamed Azami and Javier Escudero,: “Amplitude-aware permutation entropy: Illustration in spike detection and signal segmentation.” Computer methods and programs in biomedicine, 128 (2016): 40-51.
[6] Zhiqiang Huo, et al.,: “Edge Permutation Entropy: An Improved Entropy Measure for Time-Series Analysis,” 45th Annual Conference of the IEEE Industrial Electronics Soc, (2019), 5998-6003
[7] Maik Riedl, Andreas Müller, and Niels Wessel,: “Practical considerations of permutation entropy.” The European Physical Journal Special Topics 222.2 (2013): 249-262.
[8] Kang Huan, Xiaofeng Zhang, and Guangbin Zhang,: “Phase permutation entropy: A complexity measure for nonlinear time series incorporating phase information.” Physica A: Statistical Mechanics and its Applications 568 (2021): 125686.

MvSampEn(Data, varargin)

MvSampEn estimates the multivariate sample entropy of a multivariate dataset.

[MSamp, B0, Bt, B1] = MvSampEn(Data)

Returns the multivariate sample entropy estimate (MSamp) and the average number of matched delay vectors (m: B0; joint total m+1 subspace: Bt; all possible m+1 subspaces: B1), from the M multivariate sequences in Data using the default parameters: embedding dimension = 2*ones(M,1), time delay = ones(M,1), radius threshold = 0.2, logarithm = natural, data normalization = false,

Attention

The entropy value returned as MSamp is estimated using the “full” method [i.e. -log(Bt/B0)] which compares delay vectors across all possible m+1 expansions of the embedding space as applied in [1][2]. Contrary to conventional definitions of sample entropy, this method does not provide a lower bound of 0!! Thus, it is possible to obtain negative entropy values for multivariate sample entropy, even for stochastic processes…

Alternatively, one can calculate MSamp via the “naive” method, which ensures a lower bound of 0, by using the average number of matched vectors for an individual m+1 subspace (B1) [e.g. -log(B1(1)/B0)], or the average for all m+1 subspaces [i.e. -log(mean(B1)/B0)].

Note

To maximize the number of points in the embedding process, this algorithm uses N-max(m*tau) delay vectors and not N-max(m)*max(tau) as employed in [1], [2].

[MSamp, B0, Bt, B1] = MvSampEn(Data, name, value, …)

Returns the multivariate sample entropy estimates (MSamp) estimated from the M multivariate data sequences in Data using the specified name/value pair arguments:

Data:

Multivariate dataset, NxM matrix of N (>10) observations (rows) and M (cols) univariate data sequences

m:

Embedding Dimension, a vector of M positive integers

tau:

Time Delay, a vector of M positive integers

r:

Radius Distance threshold, a positive scalar

Norm:

Normalisation of all M sequences to unit variance, a boolean

Logx:

Logarithm base, a positive scalar

See also:

SampEn, XSampEn, SampEn2D, MvMSEn, MvFuzzEn, MvPermEn, MvDispEn, MvCoSiEn.

References:

[1] Ahmed Mosabber Uddin, Danilo P. Mandic: “Multivariate multiscale entropy: A tool for complexity analysis of multichannel data.” Physical Review E 84.6 (2011): 061918.
[2] Ahmed Mosabber Uddin, Danilo P. Mandic: “Multivariate multiscale entropy analysis.” IEEE signal processing letters 19.2 (2011): 91-94.