Value generalisation: value correction
I firmly believe that value generalisation[1]is the key to AI Alignment. That, indeed, it is necessary and almost sufficient for alignment.But I won't be arguing that grand point today; instead, I'll focus on a specific RL example of an agent that displays value correction: it realises its current reward function is (probably) incorrect, and acts to correct it.Thus there are:The initial situation, in distribution, where the human displays how to maximise the true reward.The out of distribution s
By Stuart_Armstrong