Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognition
We have recently shown that deep Long Short-Term Memory (LSTM) recurrent neural networks (RNNs) outperform feed forward deep neural networks (DNNs) as acoustic models for speech recognition. More recently, we have shown that the performance of sequence trained context dependent (CD) hidden Markov model (HMM) acoustic models using such LSTM RNNs can be equaled by sequence trained phone models initi
Optimizing and Contrasting Recurrent Neural Network Architectures
Recurrent Neural Networks (RNNs) have long been recognized for their potential to model complex time series. However, it remains to be determined what optimization techniques and recurrent architectures can be used to best realize this potential. The experiments presented take a deep look into Hessian free optimization, a powerful second order optimization method that has shown promising results,
Frontiers | RM-SORN: a reward-modulated self-organizing recurrent neural network
Institute of Cognitive Science, University of Osnabrück, Osnabrück, Germany Neural plasticity plays an important role in learning and memory. Reward-modulation of plasticity offers an explanation for the ability of the brain to adapt its neural activity to achieve a rewarded goal. Here, we define a neural network model that learns through the interaction of Intrinsic Plasticity (IP) and reward-mod
Improving performance of recurrent neural network with relu nonlinearity
In recent years significant progress has been made in successfully training recurrent neural networks (RNNs) on sequence learning problems involving long range temporal dependencies. The progress has been made on three fronts: (a) Algorithmic improvements involving sophisticated optimization techniques, (b) network design involving complex hidden layer nodes and specialized recurrent layer connect
BlackOut: Speeding up Recurrent Neural Network Language Models With Very Large Vocabularies
We propose BlackOut, an approximation algorithm to efficiently train massive recurrent neural network language models (RNNLMs) with million word vocabularies. BlackOut is motivated by using a discriminative loss, and we describe a new sampling strategy which significantly reduces computation while improving stability, sample efficiency, and rate of convergence. One way to understand BlackOut is to
A Convolutional Neural Network for Modelling Sentences
Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 655–665, Baltimore, Maryland, USA, June 23-25 2014. c 2014 Association for Computational Linguistics A Convolutional Neural Network for Modelling Sentences Nal Kalchbrenner Edward Grefenstette {nal.kalchbrenner, edward.grefenstette, phil.blunsom}@cs.ox.ac.uk Department of Computer Science University of
Modelling, Visualising and Summarising Documents with a Single Convolutional Neural Network
Capturing the compositional process which maps the meaning of words to that of documents is a central challenge for researchers in Natural Language Processing and Information Retrieval. We introduce a model that is able to represent the meaning of documents by embedding them in a low dimensional vector space, while preserving distinctions of word and sentence order crucial for capturing nuanced se
Encoding Source Language with Convolutional Neural Network for Machine Translation
The recently proposed neural network joint model (NNJM) (Devlin et al., 2014) augments the n-gram target language model with a heuristically chosen source context window, achieving state-of-the-art performance in SMT. In this paper, we give a more systematic treatment by summarizing the relevant source information through a convolutional architecture guided by the target information. With differen
Enhanced Image Classification With a Fast-Learning Shallow Convolutional Neural Network
We present a neural network architecture and training method designed to enable very rapid training and low implementation complexity. Due to its training speed and very few tunable parameters, the method has strong potential for applications requiring frequent retraining or online training. The approach is characterized by (a) convolutional filters based on biologically inspired visual processing
Error-Driven Incremental Learning in Deep Convolutional Neural Network for Large-Scale Image Classification - Microsoft Research
Supervised learning using deep convolutional neural network has shown its promise in large-scale image classification task. As a building block, it is now well positioned to be part of a larger system that tackles real-life multimedia tasks. An unresolved issue is that such model is trained on a static snapshot of data. Instead, this paper positions the training as a continuous learning process as
Probabilistic Binary-Mask Cocktail-Party Source Separation in a Convolutional Deep Neural Network
Separation of competing speech is a key challenge in signal processing and a feat routinely performed by the human auditory brain. A long standing benchmark of the spectrogram approach to source separation is known as the ideal binary mask. Here, we train a convolutional deep neural network, on a two-speaker cocktail party problem, to make probabilistic predictions about binary masks. Our results
SuperCNN: A Superpixelwise Convolutional Neural Network for Salient Object Detection - Shengfeng He
SuperCNN: A Superpixelwise Convolutional Neural Network for Salient Object Detection Abstract Existing computational models for salient object detection primarily rely on hand-crafted features, which are only able to capture low-level contrast information. In this paper, we learn the hierarchical contrast features by formulating salient object detection as a binary labeling problem using deep lear
Fashion Apparel Detection: The Role of Deep Convolutional Neural Network and Pose-dependent Priors
In this work, we propose and address a new computer vision task, which we call fashion item detection, where the aim is to detect various fashion items a person in the image is wearing or carrying. The types of fashion items we consider in this work include hat, glasses, bag, pants, shoes and so on. The detection of fashion items can be an important first step of various e-commerce applications fo
Fuego 1.1 vs Convolutional Neural Network Sample Games
Fuego 1.1 vs Convolutional Neural Network - Sample Games Background: In December 2014, Christopher Clark and Amos Storkey of the University of Edinburgh published their paper "Teaching Deep Convolutional Neural Networks to Play Go" on arXiv. It led to an intense discussion on the computer-go mailing list. The network reached 87% wins against Gnu Go, and 14% wins against Fuego 1.1 with 10 seconds p
Deep Karaoke_Extracting Vocal Sounds from Musical Mixtures using a Convolutional Deep Neural Network_arxiv
Deep Karaoke: Extracting Vocals from Musical Mixtures Using a Convolutional Deep Neural Network Andrew J.R. Simpson #1 , Gerard Roma#2 , Mark D. Plumbley#3 # Centre for Vision, Speech and Signal Processing, University of Surrey Guildford, UK 1 andrew.simpson@surrey.ac.uk 2 g.roma@surrey.ac.uk 3 m.plumbley@surrey.ac.uk Abstract—Identification and extraction of singing voice from within musical mixt
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
How does LSTM help prevent the vanishing (and exploding) gradient problem in a recurrent neural network?
http://www.vision.cs.chubu.ac.jp/FLABResearchArchive/Bachelor/B13/Paper/fukui.pdf
Pedestrian Detection by Deep Convolutional Neural Network
Pedestrian Detection by Deep Convolutional Neural Network
http://www.vision.cs.chubu.ac.jp/flabresearcharchive/bachelor/B13/Paper/fukui.pdf