What Happens When You Stop Training?