#StackBounty: #neural-networks #online #attention How to use/treat a hidden layer as the new target to predict in a neural network?

Bounty: 50

Let’s suppose I have a neural network with one hidden layer. During training, for a given pair of (input, target), I want to perform several iterations, such that the first iteration would be trying to predict the target, and the second iteration would be to somehow use my prediction (or other information learned from the first iteration) as my new target.

My initial thinking to solve this would be to go through a full epoch using the initial true targets and then at the second epoch, I would be able to use the predictions as the new targets. However, this seems like it could all be integrated in one network, end to end.

Is there a possible way to do this without having some information leakage?

Just for people interested in something like this, I found the following that is similar: https://wayve.ai/blog/dreaming-about-driving-imagination-rl


Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.