Meaning of alignment tax | Babel Free

Definitions

A cost to the capabilities of an artificial intelligence resulting from the effects of aligning it with human ethics and morality.

Examples

“I like⟳ this notion of an "alignment tax" […] the reason⟳ I might compromise is if there's some tension, between having the AI that's robustly trying to do what I want⟳, and having the AI that is competent or intelligent, and the alignment tax is intended to capture⟳ that gap—that cost⟳ that I incur if I insist on alignment.”

“The fact that larger models are less subject to forgetting may be related to the fact that larger models do not incur significant alignment taxes.”

“We want⟳ an alignment procedure that avoids an alignment tax, because it incentivizes the use⟳ of models that are unaligned but more capable on these tasks.”

“We note⟳ that instead of an alignment tax our proposal entails a safety dividend – the more rational the system the more capable and the safer it will be.”

CEFR level