Time Series Analysis

Best MSE (Mean Square Error) Predictor

对于所有可能的预测函数 \(f(X{n})\)，找到一个使 \(\mathbb{E}\big[\big(X{n} - f(X{n})\big)^{2} \big]\) 最小的 \(f\) 的 predictor。这样的 predictor 假设记为 \(m(X{n})\)， 称作 best MSE predictor，i.e.，

\[m(X{n}) = \mathop{\arg\min}\limits{f} \mathbb{E}\big[ \big( X{n+h} - f(X{n}) \big)^{2} \big] \]

我们知道：\(\mathop{\arg\min}\limits{f} \mathbb{E}\big[ \big( X{n+h} - f(X_{n}) \big)^{2} \big]\) 的解即为：

\[\mathbb{E}\big[ X{n+h} ~ \big| ~ X{n} \big] \]

---

证明：

基于 \(X{n}\) 求 \(\mathbb{E}\big[ \big( X{n+h} - f(X_{n}) \big)^{2} \big]\) 的最小值，实际上：

\[\mathop{\arg\min}\limits{f} \mathbb{E}\big[ \big( X{n+h} - f(X{n}) \big)^{2} \big] \iff \mathop{\arg\min}\limits{f} \mathbb{E}\big[ \big( X{n+h} - f(X{n}) \big)^{2} ~ \big| ~ X_{n} \big] \]

---

• 私以为更严谨的写法是 \(\mathop{\text{argmin}}\limits{f} ~ \mathbb{E}\Big[\Big(X{n+h} - f\big( X{n}\big)\Big)^{2} ~ | ~ \mathcal{F}{n}\Big]\)，其中 \(\left\{ \mathcal{F}{t}\right\}{t\geq 0}\) 为 \(\left\{ X{t} \right\}{t\geq 0}\) 相关的 natural filtration，but whatever。

---

等式右侧之部分：

\[\begin{align} \mathbb{E}\big[ \big( X{n+h} - f(X{n}) \big)^{2} ~ \big| ~ X{n} \big] & = \mathbb{E}[X{n+h}^{2} ~ | ~ X{n}] - 2f(X{n})\mathbb{E}[X{n+h} ~ | ~ X{n}] + f^{2}(X_{n}) \\ \end{align} \]

其中由于：

\[\begin{align} Var(X{n+h} ~ | ~ X{n}) & = \mathbb{E}\Big[ \big( X{n+h} - \mathbb{E}\big[ X{n+h}^{2} ~ | ~ X{n} \big] \big)^{2} ~ \Big| ~ X{n} \Big] \\ & = \mathbb{E}\big[ X{n+h}^{2} ~ \big| ~ X{n} \big] - 2\mathbb{E}^{2}\big[ X{n+h}^{2} ~ \big| ~ X{n} \big] + \mathbb{E}^{2}\big[ X{n+h}^{2} ~ \big| ~ X{n} \big] \\ & = \mathbb{E}\big[ X{n+h}^{2} ~ \big| ~ X{n} \big] - \mathbb{E}^{2}\big[ X{n+h}^{2} ~ \big| ~ X{n} \big] \end{align} \]

which gives that:

\[\implies Var(X{n+h} ~ | ~ X{n}) = \mathbb{E}\big[ X{n+h}^{2} ~ \big| ~ X{n} \big] - \mathbb{E}^{2}\big[ X{n+h} ~ \big| ~ X{n} \big] \]

因此，

\[\begin{align} \mathbb{E}\big[ \big( X{n+h} - f(X{n}) \big)^{2} ~ \big| ~ X{n} \big] & = Var(X{n+h} ~ | ~ X{n}) + \mathbb{E}^{2}\big[ X{n+h} ~ \big| ~ X{n}\big] - 2f(X{n})\mathbb{E}[X{n+h} ~ | ~ X{n}] + f^{2}(X{n}) \\ & = Var(X{n+h} ~ | ~ X{n}) + \Big( \mathbb{E}\big[ X{n+h} ~ \big| ~ X{n}\big] - f(X{n}) \Big)^{2} \end{align} \]

方差 \(Var(X{n+h} ~ | ~ X{n})\) 为定值，那么 optimal solution \(m(X_{n})\) 显而易见：

\[m(X{n}) = \mathbb{E}\big[ X{n+h} ~ \big| ~ X_{n} \big] \]

---

此时 \(\left\{ X_{t} \right\}\) 为一个 Stationary Gaussian Time Series, i.e.,

\[\begin{pmatrix} X{n+h}\\ X{n} \end{pmatrix} \sim N \begin{pmatrix} \begin{pmatrix} \mu \\ \mu \end{pmatrix}, ~ \begin{pmatrix} \gamma(0) & \gamma(h) \\ \gamma(h) & \gamma(0) \end{pmatrix} \end{pmatrix} \]

那么我们有：

\[X{n+h} ~ | ~ X{n} \sim N\Big( \mu + \rho(h)\big(X_{n} - \mu\big), ~ \gamma(0)\big(1 - \rho^{2}(h)\big) \Big) \]

其中 \(\rho(h)\) 为 \(\left\{ X_{t} \right\}\) 的 ACF，因此，

\[\mathbb{E}\big[ X{n+h} ~ \big| ~ X{n} \big] = m(X{n}) = \mu + \rho(h) \big( X{n} - \mu \big) \]

注意：

若 \(\left\{ X{t} \right\}\) 是一个 Gaussian time series，则一定能计算 best MSE predictor。而若 \(\left\{ X{t} \right\}\) 并非 Gaussian time series，则计算通常十分复杂。

因此，我们通常不找 best MSE predictor，而寻找 best linear predictor。

---

Best Linear Predictor (BLP)

在 BLP 假设下，我们寻找一个形如 \(f(X{n}) \propto aX{n} + b\) 的 predictor。

则目标为：

\[\text{minimize: } ~ S(a,b) = \mathbb{E} \big[ \big( X{n+h} - aX{n} -b \big)^{2} \big] \]

---

推导：

分别对 \(a, b\) 求偏微分：

\[\begin{align} \frac{\partial}{\partial b} S(a, b) & = \frac{\partial}{\partial b} \mathbb{E} \big[ \big( X{n+h} - aX{n} -b \big)^{2} \big] \\ & = -2 \mathbb{E} \big[ X{n+h} - aX{n} - b \big] \
\end{align} \]

令：

\[\frac{\partial}{\partial b} S(a, b) = 0 \]

则：

\[\begin{align} -2 \cdot & \mathbb{E} \big[ X{n+h} - aX{n} - b \big] = 0 \\ \implies & \qquad \mathbb{E}[X{n+h}] - a\mathbb{E}[X{n}] - b = 0\
\implies & \qquad \mu - a\mu - b = 0 \\ \implies & \qquad b^{\star} = (1 - a^{\star}) \mu \end{align} \]

回代并 take partial derivative on \(a\)：

\[\begin{align} \frac{\partial}{\partial a} S(a, b) & = \frac{\partial}{\partial a} \mathbb{E} \big[ \big( X{n+h} - aX{n} - (1 - a)\mu \big)^{2} \big] \\ & = \frac{\partial}{\partial a} \mathbb{E} \Big[ \Big( \big(X{n+h} - \mu \big) - \big( X{n} - \mu \big) a \Big)^{2} \Big] \
& = \mathbb{E} \Big[ - \big( X{n} - \mu \big) \Big( \big(X{n+h} - \mu \big) - \big( X_{n} - \mu \big) a \Big)\Big] \\ \end{align} \]

令：

\[\frac{\partial}{\partial a} S(a, b) = 0 \]

则：

\[\begin{align} & \mathbb{E} \Big[ - \big( X{n} - \mu \big) \Big( \big(X{n+h} - \mu \big) - \big( X{n} - \mu \big) a \Big)\Big] = 0 \
\implies & \qquad \mathbb{E} \Big[\big( X{n} - \mu \big) \Big( \big(X{n+h} - \mu \big) - \big( X{n} - \mu \big) a \Big)\Big] = 0 \\ \implies & \qquad \mathbb{E} \Big[\big( X{n} - \mu \big) \big(X{n+h} - \mu \big) - a \big( X{n} - \mu \big) \big( X{n} - \mu \big) \Big] = 0 \\ \implies & \qquad \mathbb{E} \Big[\big( X{n} - \mu \big) \big(X{n+h} - \mu \big) \Big] = a \cdot \mathbb{E} \Big[\big( X{n} - \mu \big) \big( X{n} - \mu \big) \Big] \\ \implies & \qquad \mathbb{E} \Big[\big( X{n} - \mathbb{E}[X{n}] \big) \big(X{n+h} - \mathbb{E}[X{n+h}] \big) \Big] = a \cdot \mathbb{E} \Big[\big( X{n} - \mathbb{E}[X{n}] \big)^{2} \Big] \\ \implies & \qquad \text{Cov}(X{n}, X{n+h}) = a \cdot \text{Var}(X_{n}) \\ \implies & \qquad a^{\star} = \frac{\gamma(h)}{\gamma(0)} = \rho(h) \end{align} \]

综上，time series \(\left\{ X_{n} \right\}\) 的 BLP 为：

\[f(X{n}) = l(X{n}) = \mu + \rho(h) \big( X_{n} - \mu \big) \]

且 BLP 相关的 MSE 为：

\[\begin{align} \text{MSE} & = \mathbb{E}\big[ \big( X{n+h} - l(X{n}) \big)^{2} \big] \\ & = \mathbb{E} \Big[ \Big( X{n+h} - \mu - \rho(h) \big( X{n} - \mu \big) \Big)^{2} \Big] \\ & = \rho(0) \cdot \big( 1 - \rho^{2}(h) \big) \end{align} \]