Cross Entropies

Functions for estimating the entropy between two univariate time series.

The following functions also form the cross-entropy method used by multiscale cross-entropy functions.

These functions are directly available when EntropyHub is imported:

import EntropyHub as EH

dir(EH)
XApEn(*Sig, m=2, tau=1, r=None, Logx=numpy.exp)

XApEn estimates the cross-approximate entropy between two univariate data sequences.

XAp, Phi = XApEn(Sig1, Sig2)

Returns the cross-approximate entropy estimates (XAp) and the average number of matched vectors (Phi) for m = [0,1,2], estimated for the data sequences contained in Sig1 and Sig2 using the default parameters: embedding dimension = 2, time delay = 1, radius threshold = 0.2*SDpooled(Sig1,``Sig2``), logarithm = natural

**NOTE: XApEn is direction-dependent. Thus, Sig1 is used as the template data sequence, and Sig2 is the matching sequence.

XAp, Phi = XApEn(Sig1, Sig2, keyword = value, ...)

Returns the cross-approximate entropy estimates (XAp) between the data sequences contained in Sig1 and Sig2 using the specified ‘keyword’ arguments:

m:

  • Embedding Dimension, a positive integer [default: 2]

tau:

  • Time Delay, a positive integer [default: 1]

r:

  • Radius Distance Threshold, a positive scalar [default: 0.2*SDpooled(Sig1,``Sig2``)]

Logx:

  • Logarithm base, a positive scalar [default: natural]

See also:

XSampEn, XFuzzEn, XMSEn, ApEn, SampEn, MSEn

References:
[1] Steven Pincus and Burton H. Singer,

“Randomness and degrees of irregularity.” Proceedings of the National Academy of Sciences 93.5 (1996): 2083-2088.

[2] Steven Pincus,

“Assessing serial irregularity and its implications for health.” Annals of the New York Academy of Sciences 954.1 (2001): 245-267.

XCondEn(*Sig, m=2, tau=1, c=6, Logx=numpy.exp, Norm=False)

XCondEn estimates the corrected cross-conditional entropy between two univariate data sequences.

XCond, SEw, SEz = XCondEn(Sig1, Sig2)

Returns the corrected cross-conditional entropy estimates (XCond) and the corresponding Shannon entropies (m: SEw, m+1: SEz) for m = [1,2] estimated for the data sequences contained in Sig1 and Sig2 using the default parameters: embedding dimension = 2, time delay = 1, number of symbols = 6, logarithm = natural ** Note: XCondEn is direction-dependent. Therefore, the order of the data sequences Sig1 and Sig2 matters. If Sig1 is the sequence ‘y’, and Sig2 is the sequence ‘u’, then XCond is the amount of information carried by y(i) when the pattern u(i) is found.

XCond, SEw, SEz = XCondEn(Sig1, Sig2, keyword = value, ...)

Returns the corrected cross-conditional entropy estimates (XCond) for the data sequences contained in Sig1 and Sig2 using the specified ‘keyword’ arguments:

m:

  • Embedding Dimension, an integer > 1 [default: 2]

tau:

  • Time Delay, a positive integer [default: 1]

c:

  • Number of symbols, an integer > 1 [default: 6]

Logx:

  • Logarithm base, a positive scalar [default: natural]

Norm:

  • Normalisation of XCond value, one of the following integers:

  • [False] no normalisation [default]

  • [True] normalises w.r.t cross-Shannon entropy.

See also:

XFuzzEn, XSampEn, XApEn, XPermEn, CondEn, XMSEn

References:
[1] Alberto Porta, et al.,

“Conditional entropy approach for the evaluation of the coupling strength.” Biological cybernetics 81.2 (1999): 119-129.

XDistEn(*Sig, m=2, tau=1, Bins='Sturges', Logx=2, Norm=True)

XDistEn estimates the cross-distribution entropy between two univariate data sequences.

XDist, Ppi = XDistEn(Sig1, Sig2)

Returns the cross-distribution entropy estimate (XDist) and the corresponding distribution probabilities (Ppi) estimated between the data sequences contained in Sig1 and Sig2 using the default parameters: embedding dimension = 2, time delay = 1, binning method = 'Sturges', logarithm = base 2, normalisation = w.r.t # of histogram bins

XDist, Ppi = XDistEn(Sig1, Sig2, keyword = value, ...)

Returns the cross-distribution entropy estimate (XDist) estimated between the data sequences contained in Sig1 and Sig2 using the specified ‘keyword’ = arguments:

m:

  • Embedding Dimension, a positive integer [default: 2]

tau:

  • Time Delay, a positive integer [default: 1]

Bins:

  • Histogram bin selection method for distance distribution,

  • an integer > 1 indicating the number of bins, or one of the

  • following strings {'sturges', 'sqrt', 'rice', 'doanes'} [default: ‘sturges’]

Logx:

  • Logarithm base, a positive scalar [default: 2] (enter 0 for natural log)

Norm:

  • Normalisation of DistEn value, a boolean value:

  • [False] no normalisation.

  • [True] normalises w.r.t # of histogram bins [default]

See also:

XSampEn, XApEn, XPermEn, XCondEn, DistEn, DistEn2D, XMSEn

References:
[1] Yuanyuan Wang and Pengjian Shang,

“Analysis of financial stock markets through the multiscale cross-distribution entropy based on the Tsallis entropy.” Nonlinear Dynamics 94.2 (2018): 1361-1376.

XFuzzEn(*Sig, m=2, tau=1, r=(0.2, 2.0), Fx='default', Logx=numpy.exp)

XFuzzEn estimates the cross-fuzzy entropy between two univariate data sequences.

XFuzz, Ps1, Ps2 = XFuzzEn(Sig1,Sig2)

Returns the cross-fuzzy entropy estimates (XFuzz) and the average fuzzy distances (m: Ps1, m+1: Ps2) for m = [1,2] estimated for the data sequences contained in Sig1 and Sig2, using the default parameters: embedding dimension = 2, time delay = 1, fuzzy function (Fx) = ‘default’, fuzzy function parameters (r) = (0.2, 2), logarithm = natural

XFuzz, Ps1, Ps2 = XFuzzEn(Sig1, Sig2, keyword = value, ...)

Returns the cross-fuzzy entropy estimates (XFuzz) for dimensions = [1, …, m] estimated for the data sequences in Sig1 and Sig2 using the specified ‘keyword’ arguments:

m:

  • Embedding Dimension, a positive integer [default: 2]

tau:

  • Time Delay, a positive integer [default: 1]

Fx:

  • Fuzzy function name, one of the following strings:

{'sigmoid', 'modsampen', 'default', 'gudermannian', 'bell', 'triangular', 'trapezoidal1', 'trapezoidal2', 'z_shaped', 'gaussian', 'constgaussian'}

r:

  • Fuzzy function parameters, a 1 element scalar or a 2 element vector of positive values.

The r parameters for each fuzzy function are defined as follows: [default: (.2 2)]
  • default: [Tuple]

    • r(1) = divisor of the exponential argument

    • r(2) = argument exponent (pre-division)

  • sigmoid: [Tuple]

    • r(1) = divisor of the exponential argument

    • r(2) = value subtracted from argument (pre-division)

  • modsampen: [Tuple]

    • r(1) = divisor of the exponential argument

    • r(2) = value subtracted from argument (pre-division)

  • gudermannian:

    • r = a scalar whose value is the numerator of argument to gudermannian function: GD(x) = atan(tanh(r/x)). GD(x) is normalised to have a maximum value of 1.

  • triangular:

    • r = a scalar whose value is the threshold (corner point) of the triangular function.

  • trapezoidal1:

    • r = a scalar whose value corresponds to the upper (2r) and lower (r) corner points of the trapezoid.

  • trapezoidal2: [Tuple]

    • r(1) = a value corresponding to the upper corner point of the trapezoid.

    • r(2) = a value corresponding to the lower corner point of the trapezoid.

  • z_shaped:

    • r = a scalar whose value corresponds to the upper (2r) and lower (r) corner points of the z-shape.

  • bell:

    • r(1) = divisor of the distance value

    • r(2) = exponent of generalized bell-shaped function

  • gaussian:

    • r = a scalar whose value scales the slope of the Gaussian curve.

  • constgaussian:

    • r = a scalar whose value defines the lower threshod and shape of the Gaussian curve.

  • [DEPRICATED] linear:

    r = an integer value. When r = 0, the argument of the exponential function is normalised between [0 1]. When r = 1, the minimuum value of the exponential argument is set to 0.

Logx:

  • Logarithm base, a positive scalar [default: natural]

For further information on the keyword arguments, see the EntropyHub guide.

See also:

FuzzEn, XSampEn, XApEn, FuzzEn2D, XMSEn, MSEn

References:
[1] Hong-Bo Xie, et al.,

“Cross-fuzzy entropy: A new method to test pattern synchrony of bivariate time series.” Information Sciences 180.9 (2010): 1715-1724.

[2] Hamed Azami, et al.

“Fuzzy Entropy Metrics for the Analysis of Biomedical Signals: Assessment and Comparison” IEEE Access 7 (2019): 104833-104847

XK2En(*Sig, m=2, tau=1, r=None, Logx=numpy.exp)

XK2En estimates the cross-Kolmogorov entropy between two univariate data sequences.

XK2, Ci = XK2En(Sig1, Sig2)

Returns the cross-Kolmogorov entropy estimates (XK2) and the correlation integrals (Ci) for m = [1, 2] estimated between the data sequences contained in Sig1 and``Sig2`` using the default parameters: embedding dimension = 2, time delay = 1, distance threshold (r) = 0.2*SDpooled(Sig1,``Sig2``), logarithm = natural

XK2, Ci = XK2En(Sig1,Sig2, keyword = value, ...)

Returns the cross-Kolmogorov entropy estimates (XK2) estimated between the data sequences contained in Sig1 and Sig2 using the specified ‘keyword’ arguments:

m:

  • Embedding Dimension, a positive integer [default: 2]

tau:

  • Time Delay, a positive integer [default: 1]

r:

  • Radius Distance Threshold, a positive scalar [default: 0.2*SDpooled(Sig1,``Sig2``),]

Logx:

  • Logarithm base, a positive scalar [default: natural]

See also:

XSampEn, XFuzzEn, XApEn, K2En, XMSEn, XDistEn

References:
[1] Matthew W. Flood,

“XK2En - EntropyHub Project” (2021) https://github.com/MattWillFlood/EntropyHub

XPermEn(*Sig, m=3, tau=1, Logx=numpy.exp)

XPermEn estimates the cross-permutation entropy between two univariate data sequences.

XPerm = XPermEn(Sig1, Sig2)

Returns the cross-permuation entropy estimates (XPerm) estimated betweeen the data sequences contained in Sig1 and Sig2 using the default parameters: embedding dimension = 3, time delay = 1, logarithm = base 2,

XPerm = XPermEn(Sig1, Sig2, keyword = value, ...)

Returns the permutation entropy estimates (Perm) estimated between the data sequences contained in Sig1 and Sig2 using the specified ‘keyword’ arguments:

m:

  • Embedding Dimension, an integer > 2 [default: 3]

**Note: XPerm is undefined for embedding dimensions < 3.

tau:

  • Time Delay, a positive integer [default: 1]

Logx:

  • Logarithm base, a positive scalar [default: 2] (enter 0 for natural log).

See also:

PermEn, XApEn, XSampEn, XFuzzEn, XMSEn

References:
[1] Wenbin Shi, Pengjian Shang, and Aijing Lin,

“The coupling analysis of stock market indices based on cross-permutation entropy.” Nonlinear Dynamics 79.4 (2015): 2439-2447.

XSampEn(*Sig, m=2, tau=1, r=None, Logx=numpy.exp, Vcp=False)

XSampEn Estimates the cross-sample entropy between two univariate data sequences.

XSamp, A, B = XSampEn(Sig1, Sig2)

Returns the cross-sample entropy estimates (XSamp) and the number of matched vectors (m: B, m+1: A) for m = [0,1,2] estimated for the two univariate data sequences contained in Sig1 and Sig2 using the default parameters: embedding dimension = 2, time delay = 1, radius = 0.2*SDpooled(Sig1,``Sig2``), logarithm = natural

XSamp, A, B, (Vcp, Ka, Kb) = XSampEn(Sig1, Sig2, ..., Vcp = True)

If Vcp == True, an additional tuple (Vcp, Ka, Kb) is returned with the cross-sample entropy estimates (XSamp) and the number of matched state vectors (m: B, m+1: A). (Vcp, Ka, Kb) contains the variance of the conditional probabilities (Vcp, i.e. CP = A/B), and the number of overlapping matching vector pairs of lengths m+1 (Ka) and m (Kb), respectively. Note Vcp is undefined for the zeroth embedding dimension (m = 0) and due to computational demand, will take substantially more time to return function outputs. See Appendix B in [2] for more info.

XSamp, A, B = XSampEn(Sig1, Sig2, keyword = value, ...)

Returns the cross-sample entropy estimates (XSamp) for dimensions [0,1,…, m] estimated between the data sequences Sig1 and Sig2 using the specified ‘keyword’ arguments:

m:

  • Embedding Dimension, a positive integer [default: 2]

tau:

  • Time Delay, a positive integer [default: 1]

r:

  • Radius, a positive scalar [default: 0.2*SDpooled(Sig1,``Sig2``)]

Logx:

  • Logarithm base, a positive scalar [default: natural]

See also:

XFuzzEn, XApEn, SampEn, SampEn2D, XMSEn, ApEn

References:
[1] Joshua S Richman and J. Randall Moorman.

“Physiological time-series analysis using approximate entropy and sample entropy.” American Journal of Physiology-Heart and Circulatory Physiology (2000)

[2] Douglas E Lake, Joshua S Richman, M.P. Griffin, J. Randall Moorman

“Sample entropy analysis of neonatal heart rate variability.” American Journal of Physiology-Regulatory, Integrative and Comparative Physiology 283, no. 3 (2002): R789-R797.

XSpecEn(*Sig, N=None, Freqs=(0, 1), Logx=numpy.exp, Norm=True)

XSpecEn estimates the cross-spectral entropy between two univariate data sequences.

XSpec, BandEn = XSpecEn(Sig1, Sig2)

Returns the cross-spectral entropy estimate (XSpec) of the full cross- spectrum and the within-band entropy (BandEn) estimated for the data sequences contained in Sig1 and Sig2 using the default parameters: N-point FFT = 2 * max length of Sig1/Sig2, normalised band edge frequencies = [0 1], logarithm = base 2, normalisation = w.r.t # of spectrum/band frequency values.

XSpec, BandEn = XSpecEn(Sig1, Sig2, keyword = value, ...)

Returns the cross-spectral entropy (XSpec) and the within-band entropy (BandEn) estimate for the data sequences contained in Sig1 and Sig2 using the following specified ‘keyword’ arguments:

N:

  • Resolution of spectrum (N-point FFT), an integer > 1

Freqs:

  • Normalised and edge frequencies, a scalar in range [0 1] where 1 corresponds to the Nyquist frequency (Fs/2).

  • Note: When no band frequencies are entered, BandEn == SpecEn

Logx:

  • Logarithm base, a positive scalar [default: natural]

Norm:

  • Normalisation of XSpec value, one of the following integers:

[false] no normalisation. [true] normalises w.r.t # of frequency values within the spectrum/band [default]

See the EntropyHub guide for more info.

See also:

SpecEn, fft, XDistEn, periodogram, XSampEn, XApEn

References:
[1] Matthew W. Flood,

“XSpecEn - EntropyHub Project” (2021) https://github.com/MattWillFlood/EntropyHub