# Cross Entropies

## Functions for estimating the entropy between two univariate time series.

The following functions also form the cross-entropy method used by multiscale cross-entropy functions.

These functions are directly available when EntropyHub is imported:

```import EntropyHub as EH

dir(EH)
```

XApEn(*Sig, m=2, tau=1, r=None, Logx=numpy.exp)

XApEn estimates the cross-approximate entropy between two univariate data sequences.

```XAp, Phi = XApEn(Sig1, Sig2)
```

Returns the cross-approximate entropy estimates (`XAp`) and the average number of matched vectors (`Phi`) for `m` = [0,1,2], estimated for the data sequences contained in `Sig1` and `Sig2` using the default parameters: embedding dimension = 2, time delay = 1, radius threshold = 0.2*SDpooled(`Sig1`,``Sig2``), logarithm = natural

**NOTE: `XApEn` is direction-dependent. Thus, `Sig1` is used as the template data sequence, and `Sig2` is the matching sequence.

```XAp, Phi = XApEn(Sig1, Sig2, keyword = value, ...)
```

Returns the cross-approximate entropy estimates (`XAp`) between the data sequences contained in `Sig1` and `Sig2` using the specified ‘keyword’ arguments:

m:
• Embedding Dimension, a positive integer [default: 2]

tau:
• Time Delay, a positive integer [default: 1]

r:
• Radius Distance Threshold, a positive scalar [default: 0.2*SDpooled(`Sig1`,``Sig2``)]

Logx:
• Logarithm base, a positive scalar [default: natural]

`XSampEn`, `XFuzzEn`, `XMSEn`, `ApEn`, `SampEn`, `MSEn`

References:
[1] Steven Pincus and Burton H. Singer,

“Randomness and degrees of irregularity.” Proceedings of the National Academy of Sciences 93.5 (1996): 2083-2088.

[2] Steven Pincus,

“Assessing serial irregularity and its implications for health.” Annals of the New York Academy of Sciences 954.1 (2001): 245-267.

XCondEn(*Sig, m=2, tau=1, c=6, Logx=numpy.exp, Norm=False)

XCondEn estimates the corrected cross-conditional entropy between two univariate data sequences.

```XCond, SEw, SEz = XCondEn(Sig1, Sig2)
```

Returns the corrected cross-conditional entropy estimates (`XCond`) and the corresponding Shannon entropies (`m: SEw`, `m+1: SEz`) for `m` = [1,2] estimated for the data sequences contained in `Sig1` and `Sig2` using the default parameters: embedding dimension = 2, time delay = 1, number of symbols = 6, logarithm = natural ** Note: `XCondEn` is direction-dependent. Therefore, the order of the data sequences `Sig1` and `Sig2` matters. If `Sig1` is the sequence ‘y’, and `Sig2` is the sequence ‘u’, then `XCond` is the amount of information carried by y(i) when the pattern u(i) is found.

```XCond, SEw, SEz = XCondEn(Sig1, Sig2, keyword = value, ...)
```

Returns the corrected cross-conditional entropy estimates (`XCond`) for the data sequences contained in `Sig1` and `Sig2` using the specified ‘keyword’ arguments:

m:
• Embedding Dimension, an integer > 1 [default: 2]

tau:
• Time Delay, a positive integer [default: 1]

c:
• Number of symbols, an integer > 1 [default: 6]

Logx:
• Logarithm base, a positive scalar [default: natural]

Norm:
• Normalisation of XCond value, one of the following integers:

• [False] no normalisation [default]

• [True] normalises w.r.t cross-Shannon entropy.

`XFuzzEn`, `XSampEn`, `XApEn`, `XPermEn`, `CondEn`, `XMSEn`

References:
[1] Alberto Porta, et al.,

“Conditional entropy approach for the evaluation of the coupling strength.” Biological cybernetics 81.2 (1999): 119-129.

XDistEn(*Sig, m=2, tau=1, Bins='Sturges', Logx=2, Norm=True)

XDistEn estimates the cross-distribution entropy between two univariate data sequences.

```XDist, Ppi = XDistEn(Sig1, Sig2)
```

Returns the cross-distribution entropy estimate (`XDist`) and the corresponding distribution probabilities (`Ppi`) estimated between the data sequences contained in `Sig1` and `Sig2` using the default parameters: embedding dimension = 2, time delay = 1, binning method = `'Sturges'`, logarithm = base 2, normalisation = w.r.t # of histogram bins

```XDist, Ppi = XDistEn(Sig1, Sig2, keyword = value, ...)
```

Returns the cross-distribution entropy estimate (`XDist`) estimated between the data sequences contained in `Sig1` and `Sig2` using the specified ‘keyword’ = arguments:

m:
• Embedding Dimension, a positive integer [default: 2]

tau:
• Time Delay, a positive integer [default: 1]

Bins:
• Histogram bin selection method for distance distribution,

• an integer > 1 indicating the number of bins, or one of the

• following strings {`'sturges'`, `'sqrt'`, `'rice'`, `'doanes'`} [default: ‘sturges’]

Logx:
• Logarithm base, a positive scalar [default: 2] (enter 0 for natural log)

Norm:
• Normalisation of DistEn value, a boolean value:

• [False] no normalisation.

• [True] normalises w.r.t # of histogram bins [default]

`XSampEn`, `XApEn`, `XPermEn`, `XCondEn`, `DistEn`, `DistEn2D`, `XMSEn`

References:
[1] Yuanyuan Wang and Pengjian Shang,

“Analysis of financial stock markets through the multiscale cross-distribution entropy based on the Tsallis entropy.” Nonlinear Dynamics 94.2 (2018): 1361-1376.

XFuzzEn(*Sig, m=2, tau=1, r=(0.2, 2.0), Fx='default', Logx=numpy.exp)

XFuzzEn estimates the cross-fuzzy entropy between two univariate data sequences.

```XFuzz, Ps1, Ps2 = XFuzzEn(Sig1,Sig2)
```

Returns the cross-fuzzy entropy estimates (`XFuzz`) and the average fuzzy distances (`m: Ps1`, `m+1: Ps2`) for `m` = [1,2] estimated for the data sequences contained in `Sig1` and `Sig2`, using the default parameters: embedding dimension = 2, time delay = 1, fuzzy function (`Fx`) = ‘default’, fuzzy function parameters (`r`) = (0.2, 2), logarithm = natural

```XFuzz, Ps1, Ps2 = XFuzzEn(Sig1, Sig2, keyword = value, ...)
```

Returns the cross-fuzzy entropy estimates (`XFuzz`) for dimensions = [1, …, `m`] estimated for the data sequences in `Sig1` and `Sig2` using the specified ‘keyword’ arguments:

m:
• Embedding Dimension, a positive integer [default: 2]

tau:
• Time Delay, a positive integer [default: 1]

Fx:
• Fuzzy function name, one of the following strings:

{`'sigmoid'`, `'modsampen'`, `'default'`, `'gudermannian'`, `'bell'`, `'triangular'`, `'trapezoidal1'`, `'trapezoidal2'`, `'z_shaped'`, `'gaussian'`, `'constgaussian'`}

r:
• Fuzzy function parameters, a 1 element scalar or a 2 element vector of positive values.

The `r` parameters for each fuzzy function are defined as follows: [default: (.2 2)] - default: [Tuple]

• r(1) = divisor of the exponential argument

• r(2) = argument exponent (pre-division)

• sigmoid: [Tuple]
• r(1) = divisor of the exponential argument

• r(2) = value subtracted from argument (pre-division)

• modsampen: [Tuple]
• r(1) = divisor of the exponential argument

• r(2) = value subtracted from argument (pre-division)

• gudermannian:
• r = a scalar whose value is the numerator of argument to gudermannian function: GD(x) = atan(tanh(r/x)). GD(x) is normalised to have a maximum value of 1.

• triangular:
• r = a scalar whose value is the threshold (corner point) of the triangular function.

• trapezoidal1:
• r = a scalar whose value corresponds to the upper (2r) and lower (r) corner points of the trapezoid.

• trapezoidal2: [Tuple]
• r(1) = a value corresponding to the upper corner point of the trapezoid.

• r(2) = a value corresponding to the lower corner point of the trapezoid.

• z_shaped:
• r = a scalar whose value corresponds to the upper (2r) and lower (r) corner points of the z-shape.

• bell:
• r(1) = divisor of the distance value

• r(2) = exponent of generalized bell-shaped function

• gaussian:
• r = a scalar whose value scales the slope of the Gaussian curve.

• constgaussian:
• r = a scalar whose value defines the lower threshod and shape of the Gaussian curve.

• [DEPRICATED] linear:

r = an integer value. When r = 0, the argument of the exponential function is normalised between [0 1]. When r = 1, the minimuum value of the exponential argument is set to 0.

Logx:
• Logarithm base, a positive scalar [default: natural]

For further information on the keyword arguments, see the EntropyHub guide.

`FuzzEn`, `XSampEn`, `XApEn`, `FuzzEn2D`, `XMSEn`, `MSEn`

References:
[1] Hong-Bo Xie, et al.,

“Cross-fuzzy entropy: A new method to test pattern synchrony of bivariate time series.” Information Sciences 180.9 (2010): 1715-1724.

[2] Hamed Azami, et al.

“Fuzzy Entropy Metrics for the Analysis of Biomedical Signals: Assessment and Comparison” IEEE Access 7 (2019): 104833-104847

XK2En(*Sig, m=2, tau=1, r=None, Logx=numpy.exp)

XK2En estimates the cross-Kolmogorov entropy between two univariate data sequences.

```XK2, Ci = XK2En(Sig1, Sig2)
```

Returns the cross-Kolmogorov entropy estimates (`XK2`) and the correlation integrals (`Ci`) for `m` = [1, 2] estimated between the data sequences contained in `Sig1` and``Sig2`` using the default parameters: embedding dimension = 2, time delay = 1, distance threshold (`r`) = 0.2*SDpooled(`Sig1`,``Sig2``), logarithm = natural

```XK2, Ci = XK2En(Sig1,Sig2, keyword = value, ...)
```

Returns the cross-Kolmogorov entropy estimates (`XK2`) estimated between the data sequences contained in `Sig1` and `Sig2` using the specified ‘keyword’ arguments:

m:
• Embedding Dimension, a positive integer [default: 2]

tau:
• Time Delay, a positive integer [default: 1]

r:
• Radius Distance Threshold, a positive scalar [default: 0.2*SDpooled(`Sig1`,``Sig2``),]

Logx:
• Logarithm base, a positive scalar [default: natural]

`XSampEn`, `XFuzzEn`, `XApEn`, `K2En`, `XMSEn`, `XDistEn`

References:
[1] Matthew W. Flood,

“XK2En - EntropyHub Project” (2021) https://github.com/MattWillFlood/EntropyHub

XPermEn(*Sig, m=3, tau=1, Logx=numpy.exp)

XPermEn estimates the cross-permutation entropy between two univariate data sequences.

```XPerm = XPermEn(Sig1, Sig2)
```

Returns the cross-permuation entropy estimates (`XPerm`) estimated betweeen the data sequences contained in `Sig1` and `Sig2` using the default parameters: embedding dimension = 3, time delay = 1, logarithm = base 2,

```XPerm = XPermEn(Sig1, Sig2, keyword = value, ...)
```

Returns the permutation entropy estimates (`Perm`) estimated between the data sequences contained in `Sig1` and `Sig2` using the specified ‘keyword’ arguments:

m:
• Embedding Dimension, an integer > 2 [default: 3]

**Note: `XPerm` is undefined for embedding dimensions < 3.

tau:
• Time Delay, a positive integer [default: 1]

Logx:
• Logarithm base, a positive scalar [default: 2] (enter 0 for natural log).

`PermEn`, `XApEn`, `XSampEn`, `XFuzzEn`, `XMSEn`

References:
[1] Wenbin Shi, Pengjian Shang, and Aijing Lin,

“The coupling analysis of stock market indices based on cross-permutation entropy.” Nonlinear Dynamics 79.4 (2015): 2439-2447.

XSampEn(*Sig, m=2, tau=1, r=None, Logx=numpy.exp, Vcp=False)

XSampEn Estimates the cross-sample entropy between two univariate data sequences.

```XSamp, A, B = XSampEn(Sig1, Sig2)
```

Returns the cross-sample entropy estimates (`XSamp`) and the number of matched vectors (`m: B`, `m+1: A`) for `m` = [0,1,2] estimated for the two univariate data sequences contained in `Sig1` and `Sig2` using the default parameters: embedding dimension = 2, time delay = 1, radius = 0.2*SDpooled(`Sig1`,``Sig2``), logarithm = natural

```XSamp, A, B, (Vcp, Ka, Kb) = XSampEn(Sig1, Sig2, ..., Vcp = True)
```

If `Vcp == True`, an additional tuple `(Vcp, Ka, Kb)` is returned with the cross-sample entropy estimates (`XSamp`) and the number of matched state vectors (`m: B`, `m+1: A`). `(Vcp, Ka, Kb)` contains the variance of the conditional probabilities (`Vcp`, i.e. CP = A/B), and the number of overlapping matching vector pairs of lengths m+1 (`Ka`) and m (`Kb`), respectively. Note `Vcp` is undefined for the zeroth embedding dimension (m = 0) and due to computational demand, will take substantially more time to return function outputs. See Appendix B in [2] for more info.

```XSamp, A, B = XSampEn(Sig1, Sig2, keyword = value, ...)
```

Returns the cross-sample entropy estimates (`XSamp`) for dimensions [0,1,…, `m`] estimated between the data sequences `Sig1` and `Sig2` using the specified ‘keyword’ arguments:

m:
• Embedding Dimension, a positive integer [default: 2]

tau:
• Time Delay, a positive integer [default: 1]

r:
• Radius, a positive scalar [default: 0.2*SDpooled(`Sig1`,``Sig2``)]

Logx:
• Logarithm base, a positive scalar [default: natural]

`XFuzzEn`, `XApEn`, `SampEn`, `SampEn2D`, `XMSEn`, `ApEn`

References:
[1] Joshua S Richman and J. Randall Moorman.

“Physiological time-series analysis using approximate entropy and sample entropy.” American Journal of Physiology-Heart and Circulatory Physiology (2000)

[2] Douglas E Lake, Joshua S Richman, M.P. Griffin, J. Randall Moorman

“Sample entropy analysis of neonatal heart rate variability.” American Journal of Physiology-Regulatory, Integrative and Comparative Physiology 283, no. 3 (2002): R789-R797.

XSpecEn(*Sig, N=None, Freqs=(0, 1), Logx=numpy.exp, Norm=True)

XSpecEn estimates the cross-spectral entropy between two univariate data sequences.

```XSpec, BandEn = XSpecEn(Sig1, Sig2)
```

Returns the cross-spectral entropy estimate (`XSpec`) of the full cross- spectrum and the within-band entropy (`BandEn`) estimated for the data sequences contained in `Sig1` and `Sig2` using the default parameters: N-point FFT = 2 * max length of `Sig1`/`Sig2`, normalised band edge frequencies = [0 1], logarithm = base 2, normalisation = w.r.t # of spectrum/band frequency values.

```XSpec, BandEn = XSpecEn(Sig1, Sig2, keyword = value, ...)
```

Returns the cross-spectral entropy (`XSpec`) and the within-band entropy (`BandEn`) estimate for the data sequences contained in `Sig1` and `Sig2` using the following specified ‘keyword’ arguments:

N:
• Resolution of spectrum (N-point FFT), an integer > 1

Freqs:
• Normalised and edge frequencies, a scalar in range [0 1] where 1 corresponds to the Nyquist frequency (Fs/2).

• Note: When no band frequencies are entered, `BandEn == SpecEn`

Logx:
• Logarithm base, a positive scalar [default: natural]

Norm:
• Normalisation of `XSpec` value, one of the following integers:

[false] no normalisation. [true] normalises w.r.t # of frequency values within the spectrum/band [default]

`SpecEn`, `fft`, `XDistEn`, `periodogram`, `XSampEn`, `XApEn`