Staying in the Learning Loop