[DLHacks]XLNet を動かして可視化してみた

10. XLNetの出力ベクトルをPCAやT-SNEなどで可視化したい Final layer output 2. Code: やりたいこと 10 Embeddings Transformer Layer 0 Transformer Layer 1 Transformer Layer 22 Transformer Layer 23 [CLS] This is a pen . [SEP] [PAD] … … All output tensor shape is (seq_len, n_dim) = (200, 1024) 200 tokens * 32 sentences ̶> 6,400 points in 1024 dimensions Vector Token sentence_id (0 - 31) This 0 is 0 [SEP] 0 [PAD] 0 This 0 That 1