Technology

Open a terminal and type a command you’ve never run before. You can read its source, trace its output, pipe it into another command, undo whatever it did. Now try to find out why your loan application was denied, why your insurance premium went up, why a hiring algorithm ranked you below a stranger. You can’t. Not because the reasoning is too complex to explain — because no one who built the system intended for you to see it. Put the two side by side and a question opens up that has nothing to do with computers specifically: what obligation does a system owe the people it governs, and why do some systems honor it while others treat opacity as a feature? …

An earlier post closed on a question I said I wasn’t qualified to answer: if music is a key and the listener a changing lock, can a machine learn the shape of your lock well enough to cut one on demand — an endless supply of the thing that fits, addictive precisely because it fits? I said I suspected yes. That’s not an answer, it’s a hunch, and a hunch doesn’t tell you where the line is. The honest version of the question isn’t whether personalization can become manipulation — of course it can — but where the threshold sits, and how you’d know if a specific platform, the one open in your other tab right now, has crossed it. …

Two point six million years passed between the first deliberately chipped stone edge and the first controlled use of fire. Twenty-seven years passed between the public internet and a machine that drafts a legal brief on request. Somewhere in that arithmetic is the whole history of human invention, and it is not primarily a story about tools getting better. It is a story about the gap between tools shrinking, relentlessly, until the gap itself became the news. AI is not a rupture with that history. It is the newest point on a curve that has bent the same direction since a hominin first swung a rock with an edge on it — and the open question is whether a species built for millennium-scale change can now survive a decade-scale one. …

Luca Pacioli did not invent double-entry bookkeeping — Venetian merchants had been using it for at least a century before his 1494 treatise wrote the method down — but he did name the discipline’s real trick. Every transaction gets entered twice, once as a debit and once as a credit, and if the two columns ever fail to match, an error exists somewhere in the world the ledger describes. The genius of the system is not that it records money. It is that it manufactures a built-in alarm for reality not matching the record. Historians of science have mostly ignored this as a source of method, filing it under commerce rather than epistemology. But run the same trick on nature instead of a merchant’s warehouse, and a surprising amount of the history of discovery turns out to be exactly this: someone auditing a ledger, finding it would not close, and naming whatever was missing so it would. …

In 1926, fifty years after Bell’s patent, the Knights of Columbus Adult Education Committee circulated a set of discussion questions for its study groups. Two of them, verbatim: Does the telephone make men more active or more lazy? And: Does the telephone break up home life and the old practice of visiting friends? Read them again and swap the noun. Half a century into the technology — not in its novelty phase, not in the first flush of alarm, but two generations deep, when every household that could afford one had one — serious adults were still convening to ask whether the device was making them lazy and dissolving the family. It is this decade’s discourse with the serial numbers filed off. The only thing that has changed in a hundred years is the object on the table. …

Every introduction to neural networks explains what weights and biases do. A weight multiplies an input to make it stronger or weaker. A bias shifts the activation threshold left or right. Together they determine whether a neuron fires. But almost nobody explains why they are called that. The names are treated as arbitrary labels, as if the early researchers could have called them “twiddles” and “knobs” and it would have been the same. It would not have been the same. The names carry the history — and the physics — that the math obscures. …

Imagine a stadium. Not with a crowd, but with the field itself filled by tightrope walkers, arranged in rows, each on a wire, each holding a long pole. You stand at one end and shout a word. The walkers in the first row feel it—each differently, depending on where they stand—and they wobble, find their balance, and their lamps come on at different brightnesses. That pattern of light falls on the second row. They balance. Their lamps light the third. And so on, through hundreds of rows, until the last row’s lights spell out a single thing: the next word. Then you add that word to what you shouted and do it all again. And again, until you have a sentence, a paragraph, an answer. …

In 1958, Frank Rosenblatt built a machine that could learn. Not be programmed—learn. The Mark I Perceptron was a room of wires and motorized potentiometers wired to a grid of four hundred photocells, and when you showed it images, it adjusted itself until it could tell them apart. The New York Times reported that the Navy expected it to “walk, talk, see, write, reproduce itself and be conscious of its existence.” It could do none of these things. What it could do was draw a line. …

The reassurance has become a reflex: “Don’t worry about AI replacing your job. Learn a trade. Become a plumber. You’ll always be needed.” It’s not false. But it’s not entirely true either—probably about 75% correct, which is the most dangerous place for an argument to land. I. Three Hidden Variables The advice works until you account for three things that never make it into the conversation: saturation, incompetence, and structural obsolescence. …

In One Hundred Years of Solitude, when the gypsies bring a flying carpet to Macondo, José Arcadio Buendía stands unmoved. “Let them dream,” he says. “We’ll do better flying than they are doing, and with more scientific resources than a miserable bedspread.” He is the rationalist in a village of magic—the one man insisting on understanding how things actually work rather than being dazzled by how they appear. Then he ties himself to a chestnut tree and never recovers. …