The Audit Imperative — Why Systems That Govern You Must Be Readable by You
Open a terminal and type a command you’ve never run before. You can read its source, trace its output, pipe it into another command, undo whatever it did. Now try to find out why your loan application was denied, why your insurance premium went up, why a hiring algorithm ranked you below a stranger. You can’t. Not because the reasoning is too complex to explain — because no one who built the system intended for you to see it. Put the two side by side and a question opens up that has nothing to do with computers specifically: what obligation does a system owe the people it governs, and why do some systems honor it while others treat opacity as a feature? …