What AI Thinks It Knows About You

It calls to mind a maxim about why it is so hard to understand ourselves: “If the human brain were so simple that we could understand it, we would be so simple that we couldn’t.” If models were simple enough for us to grasp what’s going on inside when they run, they’d produce answers so dull that there might not be much payoff to understanding how they came about.

If there’s no way to know how these models work, it makes accepting the full spectrum of their behaviors (at least after humans’ efforts at “fine-tuning” them) a sort of all-or-nothing proposition. Those kinds of choices have been presented before. Did we want aspirin even though for 100 years we couldn’t explain how it made headaches go away? There, both regulators and the public said yes.