LTI Systems on Random Inputs

Convolution, correlation propagation, spectra, and stationarity through linear filters.

The PSD page asks what a stationary process looks like in frequency. This page asks what a system does to such a process. The same filter has two faces: convolution in time and multiplication in frequency.

1. Impulse response and convolution

An LTI system is determined by its impulse response $h(t)$. For an input random process $X(t)$, each realization is filtered by convolution:

$$ Y(t) = (X*h)(t)=\int X(u)h(t-u)\,du. $$

The equation is deterministic for each waveform, but the input waveform is random. That is the basic trick: filter every realization the same way, then ask what happens to means, correlations, and spectra.

Figure 1 · Sample-path filtering

input $x[n]$ impulse response $h[n]$ output $y[n]$

input process

filter

2. Mean and correlation propagation

Means and correlations propagate through the same impulse response. In discrete-time notation,

$$ \mu_y = \mu_x*h,\qquad R_{xy}=R_{xx}*\tilde h,\qquad R_{yy}=R_{xx}*h*\tilde h, $$

where $\tilde h[n]=h[-n]$. The output autocorrelation is the input autocorrelation blurred by the filter and its time reverse. This is the time-domain version of the PSD rule.

Figure 1b · Correlation propagation in the time domain

$R_{xx}$ $h * \tilde h$ $R_{yy}$

Drag the green impulse-response handles in the middle panel.

3. Time and frequency dual views

In frequency, the same statement is shorter:

$$ S_{yy}(\omega)=|H(\omega)|^2S_{xx}(\omega). $$

The figure below shows both versions at once: spectra on top, correlations on the bottom. The middle column is the system; the right column is what happens after the system.

Figure 2 · Same filter, two views

input system output

filter strength" valueId="fig2-strength-v" min="0.15" max="1.5" step="0.01" value="0.7" />

4. AR(1) as a leaky integrator

The AR(1) process

$$ X[n]=aX[n-1]+W[n],\qquad |a|\lt 1, $$

is white noise passed through the causal filter

$$ H(z)=\frac{1}{1-az^{-1}}. $$

Positive $a$ makes smooth persistence. Negative $a$ makes alternating signs. Near zero, the filter barely remembers the past.

Figure 3 · AR(1) sample path, autocorrelation, and PSD

$a$ 0.7

5. Stationarity and nonlinearities

An LTI filter preserves wide-sense stationarity: if $X$ is W.S.S., then $Y=X*h$ has constant mean and lag-only autocorrelation. A nonlinear transformation can still be stationary in special cases, but it no longer obeys the LTI propagation rules. The simple formulas above stop applying.

Does WSS suffice? An LTI filter only needs the input's first two moments to compute the output's first two moments, so W.S.S. in $\rightarrow$ W.S.S. out is automatic. Strict-sense stationarity is generally not preserved by nonlinear maps, but it is preserved by LTI filters when the input is Gaussian, because a filtered Gaussian process is still Gaussian and Gaussians are determined by their second-order structure. See the WSS vs SSS aside on the PSD page.

Figure 4 · LTI propagation vs. a memoryless nonlinearity

Figure 4b · Running summaries reveal when W.S.S. propagation stops applying

linear output running mean/variance nonlinear output running mean/variance

What to remember

Question	LTI answer	Where to look
What happens to a waveform?	Convolution: $y=x*h$.	Sample paths and impulse responses.
What happens to mean?	$\mu_y=\mu_x*h$.	DC gain and running averages.
What happens to correlation?	$R_{yy}=R_{xx}h\tilde h$.	Lag-domain smoothing by the filter autocorrelation.
What happens to PSD?	$S_{yy}=\|H\|^2S_{xx}$.	Frequency-domain spectrum shaping.

What next

The LTI page is the system view; the PSD page is the representation view.

Spectrum

PSD & Wiener-Khinchin

Back up to the Fourier-pair identity that makes the frequency-domain rule possible.

State space

Monte Carlo & Bayesian Sampling

See filtering used in the probabilistic inference sense: Kalman and particle filters.

Random variables

Entropy & Mutual Information

Switch from second-order structure to uncertainty and dependence.