Introduction In the last few years, reinforcement learning (RL) has made remarkable progress, including beating world-champion Go players, controlling robotic hands, and even painting pictures. One of the key sub-problems of RL is value estimation – learning the long-term consequences of being in a state. This can be tricky because future returns are generally noisy, affected by many things other
![The Paths Perspective on Value Learning](https://cdn-ak-scissors.b.st-hatena.com/image/square/933b638808329aed9528543cb56499b2f0cc35c3/height=288;version=1;width=512/https%3A%2F%2Fdistill.pub%2F2019%2Fpaths-perspective-on-value-learning%2Fthumbnail.jpg)