Browse Source

Add Carl's feedback

arxiv-v3
Markus Kaiser 3 years ago
parent
commit
7c511b8053
  1. BIN
      dynamic_dirichlet_deep_gp.pdf
  2. 12
      dynamic_dirichlet_deep_gp.tex

BIN
dynamic_dirichlet_deep_gp.pdf

Binary file not shown.

12
dynamic_dirichlet_deep_gp.tex

@ -528,10 +528,11 @@ The resulting data set $\D = \Set{\left( x, y_1 \right), \left( x, y_2 \right),
We use squared exponential kernels as priors for both the $f^{\pix{k}}$ and $\alpha^{\pix{k}}$ and $25$ inducing points in every GP.
\Cref{fig:semi_bimodal} shows the posterior of an DAGP with $K = 4$ modes applied to the data, which correctly identified the underlying functions.
Te figure shows the posterior belief about the assignments $\mat{A}$ and illustrates that DAGP recovered that it needs only three of the four available modes to explain the data.
The figure shows the posterior belief about the assignments $\mat{A}$ and illustrates that DAGP recovered that it needs only three of the four available modes to explain the data.
One of the modes is only assigned points in the interval $[0, 5]$ where the data is actually trimodal.
This separation is explicitly represented in the model via the assignment processes $\mat{\alpha}$.
This separation is explicitly represented in the model via the assignment processes $\mat{\alpha}$ (bottom panel in \cref{fig:semi_bimodal}).
\todo{Importantly we do not only cluster the data, we factorize the space.}
The model has disabled the mode $k = 2$ in the complete input space and has learned that the mode $k = 1$ is only relevant in the interval $[0, 5]$ where the three enabled modes each explain about a third of the data.
Outside this interval, the model has learned that one of the modes has about twice the assignment probability than the other one, thus correctly reconstructing the true generative process.
The DAGP is implicitly incentivized to explain the data using as few modes as possible through the likelihood term of the inferred $\mat{a_n}$ in \cref{eq:variational_bound}.
@ -574,8 +575,8 @@ At $x = -10$ the inferred modes and assignment processes start reverting to thei
% 10 & DAGP 5 & 0.535 \pm 0.004 & 0.506 \pm 0.005 & 0.851 \pm 0.003 & 0.851 \pm 0.003 & 0.662 \pm 0.009 & 0.581 \pm 0.012 \\
\addlinespace
10 & OMGP & -1.04 \pm 0.02 & -1.11 \pm 0.03 & 0.64 \pm 0.02 & 0.66 \pm 0.02 & -0.9 \pm 0.2 & -0.81 \pm 0.12 \\
10 & BNN+LV & 0.519 \pm 0.005 & \bfseries 0.524 \pm 0.005 & {\textemdash} & {\textemdash} & {\textemdash} & {\textemdash} \\
\midrule
10 & BNN+LV & 0.519 \pm 0.005 & \bfseries 0.524 \pm 0.005 & {\textemdash} & {\textemdash} & {\textemdash} & {\textemdash} \\
10 & GPR Mixed & 0.452 \pm 0.003 & 0.421 \pm 0.003 & {\textemdash} & {\textemdash} & {\textemdash} & {\textemdash} \\
10 & GPR Default & {\textemdash} & {\textemdash} & 0.873 \pm 0.001 & 0.867 \pm 0.001 & -7.01 \pm 0.11 & -7.54 \pm 0.14 \\
10 & GPR Short & {\textemdash} & {\textemdash} & -5.24 \pm 0.04 & -5.14 \pm 0.04 & 0.903 \pm 0.003 & 0.792 \pm 0.003 \\
@ -611,8 +612,9 @@ They are trained on the mixed data set, the default system and the short-pole sy
The GPR model predicts a unimodal posterior for the mixed data set which covers both systems.
Its mean prediction is approximately the mean of the two regimes and is physically implausible.
The DAGP and BNN+LV models yield informative multi-modal predictions with comparable performance.
\todo{How to say this nicely?}In our setup, OMGP could not successfully solve the data association problem and thus does not produce a good joint posterior.
A possible explanation is that OMGP's inference scheme is optimized for ordered one-dimensional training sets, which is not the case in the 4D cart-pole problem.
In our setup, OMGP could not successfully solve the data association problem and thus does not produce a useful joint posterior.
The OMGP's inference scheme is tailored to ordered one-dimensional problems.
It does not trivially translate to the 4D cart-pole problem.
As BNN+LV does not explicitly solve the data association problem, the model does not yield sub-models for the two different systems.
OMGP and DAGP yield such sub-models which can independently be used for predictions in the default or short-pole systems.

Loading…
Cancel
Save