Browse Source

Add Clemens' feedback

arxiv-v3
Markus Kaiser 3 years ago
parent
commit
ea27cb3906
  1. BIN
      dynamic_dirichlet_deep_gp.pdf
  2. 10
      dynamic_dirichlet_deep_gp.tex

BIN
dynamic_dirichlet_deep_gp.pdf

Binary file not shown.

10
dynamic_dirichlet_deep_gp.tex

@ -367,7 +367,7 @@ This extended bound thus has complexity $\Fun*{\Oh}{NM^2LK}$ to evaluate in the
% \setlength{\tabcolsep}{1pt}
\begin{tabularx}{\linewidth}{lYYYYYYYY}
\toprule
\resultrow{}{Bayesian}{Scalable Inference}{Approximate Joint}{Data Association}{Predictive Associations}{Multimodal Data}{Per-Mode Predictions}{Interpretable Priors}
\resultrow{}{Bayesian}{Scalable Inference}{Approximate Joint}{Data Association}{Predictive Associations}{Multimodal Data}{Separate Models}{Interpretable Priors}
\midrule
Experiment & & & & & \cref{tab:choicenet} & \cref{tab:cartpole} & \cref{fig:semi_bimodal} \\
\midrule
@ -389,14 +389,14 @@ In this section we investigate the behavior of the DAGP model.
We use an implementation of DAGP in TensorFlow~\parencite{tensorflow2015-whitepaper} based on GPflow~\parencite{matthews_gpflow_2017} and the implementation of DSVI~\parencite{salimbeni_doubly_2017}.
\Cref{tab:model_capabilities} compares qualitative properties of DAGP and related work.
Both standard Gaussian process regression (GPR) and multi-layer perceptrons (MLP) do not impose structure which enables the model to handle multi-modal data.
Mixture density networks (MDN)~\parencite{bishop_mixture_1994} the infinite mixtures of Gaussian processes (RGPR)~\parencite{rasmussen_infinite_2002} model are yield multi-modal posteriors through mixtures with many components but do not solve an association problem.
Mixture density networks (MDN)~\parencite{bishop_mixture_1994} and the infinite mixtures of Gaussian processes (RGPR)~\parencite{rasmussen_infinite_2002} model yield multi-modal posteriors through mixtures with many components but do not solve an association problem.
Similarly, Bayesian neural networks with added latent variables (BNN+LV)~\parencite{depeweg_learning_2016} represent such a mixture through a continuous latent variable.
Both the overlapping mixtures of Gaussian processes (OMGP)~\parencite{lazaro-gredilla_overlapping_2012} model and DAGP explicitly model the data association problem and yield independent models for the different generating processes.
However, OMGP assumes global relevance of the different modes.
In contrast, DAGP infers a spacial posterior of this relevance.
We evaluate or model on three problems to highlight advantages of the explicit structure of DAGP:
We evaluate our model on three problems to highlight the following advantages of the explicit structure of DAGP:
\emph{Interpretable priors give structure to ill-posed association problems.}
\emph{Interpretable priors give structure to ill-posed data association problems.}
In \cref{subsec:choicenet} we consider a noise separation problem, where a signal of interest is disturbed with uniform noise.
To solve this problem, assumptions about what constitutes a signal are needed.
The hierarchical structure of DAGP allows us to formulate independent and interpretable priors on the noise and signal processes.
@ -420,7 +420,7 @@ We show that the DAGP posterior contains two separate models for the two origina
\captionof{table}{
\label{tab:choicenet}
Results on the ChoiceNet data set.
The gray part of the table shows RMSE results for basline models from~\parencite{choi_choicenet_2018}.
The gray part of the table shows RMSE results for baseline models from~\parencite{choi_choicenet_2018}.
For our experiments using the same setup, we report RMSE comparable to the previous results together with MLL.
Both are calculated based on a test set of 1000 equally spaced samples of the noiseless underlying function.
}%

Loading…
Cancel
Save