Meaning of alignment tax | Babel Free

Definitions

A cost to the capabilities of an artificial intelligence resulting from the effects of aligning it with human ethics and morality.

Examples

“I like this notion of an "alignment tax" […] the reason I might compromise is if there's some tension, between having the AI that's robustly trying to do what I want, and having the AI that is competent or intelligent, and the alignment tax is intended to capture that gap—that cost that I incur if I insist on alignment.”

“The fact that larger models are less subject to forgetting may be related to the fact that larger models do not incur significant alignment taxes.”

“We want an alignment procedure that avoids an alignment tax, because it incentivizes the use of models that are unaligned but more capable on these tasks.”

“We note that instead of an alignment tax our proposal entails a safety dividend – the more rational the system the more capable and the safer it will be.”

CEFR level