Adam: A Method for Stochastic Optimization
We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or paramet
Bayesian Reinforcement Learning: A Survey
Bayesian methods for machine learning have been widely investigated, yielding principled methods for incorporating prior information into inference algorithms. In this survey, we provide an in-depth review of the role of Bayesian methods for the reinforcement learning (RL) paradigm. The major incentives for incorporating Bayesian reasoning in RL are: 1) it provides an elegant approach to action-se
Sampling Generative Networks
We introduce several techniques for sampling and visualizing the latent spaces of generative models. Replacing linear interpolation with spherical linear interpolation prevents diverging from a model's prior distribution and produces sharper samples. J-Diagrams and MINE grids are introduced as visualizations of manifolds created by analogies and nearest neighbors. We demonstrate two new techniques
Formalizing Neurath's Ship: Approximate Algorithms for Online Causal Learning
Higher-level cognition depends on the ability to learn models of the world. We can characterize this at the computational level as a structure-learning problem with the goal of best identifying the prevailing causal relationships among a set of relata. However, the computational cost of performing exact Bayesian inference over causal models grows rapidly as the number of relata increases. This imp
Nonthermal Gravitino Production after Large Field Inflation
We revisit the nonthermal gravitino production at the (p)reheating stage after inflation. Particular attention is paid to large field inflation models with a $\mathbb{Z}_2$ symmetry, for which the previous perturbative analysis is inapplicable; and inflation models with a stabilizer superfield, which have not been studied non-perturbatively. It is found that in single-superfield inflation models (
Black hole formation from axion stars
The classical equations of motion for an axion with potential $V(\phi)=m_a^2f_a^2 [1-\cos (\phi/f_a)]$ possess quasi-stable, localized, oscillating solutions, which we refer to as "axion stars". We study, for the first time, collapse of axion stars numerically using the full non-linear Einstein equations of general relativity and the full non-perturbative cosine potential. We map regions on an "ax
Gray-box inference for structured Gaussian process models
We develop an automated variational inference method for Bayesian structured prediction problems with Gaussian process (GP) priors and linear-chain likelihoods. Our approach does not need to know the details of the structured likelihood model and can scale up to a large number of observations. Furthermore, we show that the required expected likelihood term and its gradients in the variational obje
The Microsoft 2016 Conversational Speech Recognition System
We describe Microsoft's conversational speech recognition system, in which we combine recent developments in neural-network-based acoustic and language modeling to advance the state of the art on the Switchboard recognition task. Inspired by machine learning ensemble techniques, the system uses a range of convolutional and recurrent neural networks. I-vector modeling and lattice-free MMI training
Reverse Bisimulations on Stable Configuration Structures
The relationships between various equivalences on configuration structures, including interleaving bisimulation (IB), step bisimulation (SB) and hereditary history-preserving (HH) bisimulation, have been investigated by van Glabbeek and Goltz (and later Fecher). Since HH bisimulation may be characterised by the use of reverse as well as forward transitions, it is of interest to investigate forms o
Practical Statistics for Particle Physicists
These three lectures provide an introduction to the main concepts of statistical data analysis useful for precision measurements and searches for new signals in High Energy Physics. The frequentist and Bayesian approaches to probability theory are introduced and, for both approaches, inference methods are presented. Hypothesis tests will be discussed, then significance and upper limit evaluation w
Efficient softmax approximation for GPUs
We propose an approximate strategy to efficiently train neural network based language models over very large vocabularies. Our approach, called adaptive softmax, circumvents the linear dependency on the vocabulary size by exploiting the unbalanced word distribution to form clusters that explicitly minimize the expectation of computation time. Our approach further reduces the computational time by
Very Simple Classifier: a Concept Binary Classifier toInvestigate Features Based on Subsampling and Localility
We propose Very Simple Classifier (VSC) a novel method designed to incorporate the concepts of subsampling and locality in the definition of features to be used as the input of a perceptron. The rationale is that locality theoretically guarantees a bound on the generalization error. Each feature in VSC is a max-margin classifier built on randomly-selected pairs of samples. The locality in VSC is a
Ask Me Anything:Dynamic Memory Networks for Natural Language Processing
Ask Me Anything: Dynamic Memory Networks for Natural Language Processing Ankit Kumar, Peter Ondruska, Mohit Iyyer, James Bradbury, Ishaan Gulrajani, Victor Zhong, Romain Paulus, Richard Socher firstname@metamind.io, MetaMind, Palo Alto, CA USA Abstract Most tasks in natural language processing can be cast into question answering (QA) problems over language input. We introduce the dynamic memory ne
Gravitational Waves as a New Probe of Bose-Einstein Condensate Dark Matter
There exists a class of ultralight Dark Matter (DM) models which could form a Bose-Einstein condensate (BEC) in the early universe and behave as a single coherent wave instead of individual particles in galaxies. We show that a generic BEC DM halo intervening along the line of sight of a gravitational wave (GW) signal could induce an observable change in the speed of GW, with the effective refract
Relativistic axions from collapsing Bose stars
The substructures of light bosonic (axion-like) dark matter may condense into compact Bose stars. We study collapses of the critical-mass stars caused by attractive self-interaction of the axion-like particles and find that these processes proceed in an unexpected universal way. First, nonlinear self-similar evolution (called "wave collapse" in condensed matter physics) forces the particles to fal
EgoNet-UIUC: A Dataset For Ego Network Research
In this report, we introduce the version one of EgoNet-UIUC, which is a dataset for ego network research. The dataset contains about 230 ego networks in Linkedin, which have about 33K users (with their attributes) and 283K relationships (with their relationship types) in total. We name this dataset as EgoNet-UIUC, which stands for Ego Network Dataset from University of Illinois at Urbana-Champaign
Random matrices meet machine learning: a large dimensional analysis of LS-SVM
This article proposes a performance analysis of kernel least squares support vector machines (LS-SVMs) based on a random matrix approach, in the regime where both the dimension of data $p$ and their number $n$ grow large at the same rate. Under a two-class Gaussian mixture model for the input data, we prove that the LS-SVM decision function is asymptotically normal with means and covariances shown
Predicting the future relevance of research institutions - The winning solution of the KDD Cup 2016
The world's collective knowledge is evolving through research and new scientific discoveries. It is becoming increasingly difficult to objectively rank the impact research institutes have on global advancements. However, since the funding, governmental support, staff and students quality all mirror the projected quality of the institution, it becomes essential to measure the affiliation's rating i
Discrete Variational Autoencoders
Probabilistic models with discrete latent variables naturally capture datasets composed of discrete classes. However, they are difficult to train efficiently, since backpropagation through discrete variables is generally not possible. We present a novel method to train a class of probabilistic models with discrete latent variables using the variational autoencoder framework, including backpropagat
CompAdaGrad: A Compressed, Complementary, Computationally-Efficient Adaptive Gradient Method
The adaptive gradient online learning method known as AdaGrad has seen widespread use in the machine learning community in stochastic and adversarial online learning problems and more recently in deep learning methods. The method's full-matrix incarnation offers much better theoretical guarantees and potentially better empirical performance than its diagonal version; however, this version is compu
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
