Why Momentum Really Works
We often think of Momentum as a means of dampening oscillations and speeding up the iterations, leading to faster convergence. But it has other interesting behavior. It allows a larger range of step-sizes to be used, and creates its own oscillations. What is going on? Here’s a popular story about momentum [1, 2, 3]: gradient descent is a man walking down a hill. He follows the steepest path downwa
Bridging the Gap between Stochastic Gradient MCMC and Stochastic Optimization
Stochastic gradient Markov chain Monte Carlo (SG-MCMC) methods are Bayesian analogs to popular stochastic optimization methods; however, this connection is not well studied. We explore this relationship by applying simulated annealing to an SGMCMC algorithm. Furthermore, we extend recent SG-MCMC methods with two key components: i) adaptive preconditioners (as in ADAgrad or RMSprop), and ii) adapti
Edward – Home
A library for probabilistic modeling, inference, and criticism. Edward is a Python library for probabilistic modeling, inference, and criticism. It is a testbed for fast experimentation and research with probabilistic models, ranging from classical hierarchical models on small data sets to complex deep probabilistic models on large data sets. Edward fuses three fields: Bayesian statistics and mach
Trends and Highlights of ICML 2015 | Dustin Tran
There were a number of exciting things happening at ICML this past week, which took place in Lille, France. Deep learning remains the primary interest among a lot of research and excitement at ICML, where questions related to them would percolate even to the Bayesian nonparametrics and approximate inference sessions. It looks like a lot of the community has been paying more attention to introducin
PTGH：機械学習パラメータチューニングをDeep LearningとMCMCで高速に最適化するフレームワーク - 渋谷駅前で働くデータサイエンティストのブログ
苦節2年、とうとう完成しました。機械学習のパラメータチューニングに悩める皆さんのために、コーディングも数学も大の苦手な僕が頑張って作りました。それがPTGH (Parameter Tuning by God's Hand)フレームワークです。RでもPythonでも動きます。 中身としては、代表的な機械学習であるロジスティック回帰・SVM（線形orガウシアンカーネル）・ランダムフォレスト・Xgboost・Deep NN・Convolutional NNのそれぞれのパラメータチューニングを、arXivに上がっている論文に頻出のパターンに絞った上でそのパラメータ構成をRなら{mxnet}で、PythonならChainer / TensorFlowで回す際の記法に合わせてDeep Learningで学習させ、その学習済みモデルに基づいてMCMCで最適なパラメータの組み合わせを適応的に探し出して最適
A Complete Recipe for Stochastic Gradient MCMC
Authors Yi-An Ma, Tianqi Chen, Emily Fox Abstract Many recent Markov chain Monte Carlo (MCMC) samplers leverage continuous dynamics to define a transition kernel that efficiently explores a target distribution. In tandem, a focus has been on devising scalable variants that subsample the data and use stochastic gradients in place of full-data gradients in the dynamic simulations. However, such stoc
FACTORIE: Home
Inference The basic use of a graphical model is to perform inference: making predictions about the values of unobserved variables, conditioned on the values of observed variables and the parameters. FACTORIE has implementations of many common inference algorithms based on both belief propagation and MCMC. Tutorial » Learning Through a modular specification of inference, optimization, and learning
1
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
https://www.jstage.jst.go.jp/article/tjsai/31/6/31_AI30-C/_pdf
<b>Yizhe Zhang</b>
http://www.stat.harvard.edu/Faculty_Content/meng/JCGS01.pdf
Bayesian Neural Network in PyMC3
pymc3/GP-smoothing.ipynb at gp-smoothing-example · akuz/pymc3 · GitHub
Bayesian Sampling using Stochastic Gradient Thermostats
Yee Whye Teh