The Continual Constitutional Convention: How language models revise the documents that govern them
Key takeaway
Executive SummaryHuman societies exhibit broad shifts in values and norms over time, often driven by moral disagreements.One way of understanding moral progress is the resolution of these disagreements through a process of reflection by rational…
Why it's on PDOOM
PDOOM selected this story as relevant to alignment & control.
From Alignment Forum
Executive SummaryHuman societies exhibit broad shifts in values and norms over time, often driven by moral disagreements.One way of understanding moral progress is the resolution of these disagreements through a process of reflection by rational agents.Frontier AI companies use documents such as constitutions, model specs, and system prompts to guide the moral orientation of AI systems.In this experiment, we simulate value reflection in LLMs by asking them to edit these documents. We provide the
By peterkirgis