Language GANs Falling Short

Generating high-quality text with sufficient diversity is essential for a wide range of Natural Language Generation (NLG) tasks. Maximum-Likelihood (MLE) models trained with teacher forcing have consistently been reported as weak baselines, where poor performance is attributed to exposure bias (Bengio et al., 2015; Ranzato et al., 2015); at inference time, the model is fed its own prediction inste

[showyou]

>テキスト生成において品質と多様性という2面で評価を行う方法と、その評価において基本的な最尤推定のモデル(MLE)がGANベースよりも優位であることを示した研究。softmaxのtemperatureは高いと確率が等しい=多様性に寄与