Ask the People You Are Replacing

The system you are building needs something that exists only in the heads of the people it may replace.

Not the procedure. The procedure is documented, and for this purpose the documentation is close to useless, because it records the steps and not the judgment. It tells you what to do when the file arrives. It does not tell you that one particular client's file is wrong the same way every March, or that a number in this range is unremarkable except in the week after a capital call, or any of the several hundred small corrections a person makes without noticing they are making them. That is the operating model. It was never written down because it was never articulated, and it was never articulated because the people doing it do not experience it as knowledge. They experience it as obvious.

So you have to ask them. And what you are asking for, stated plainly, is that they hand over the thing that makes them hard to replace.

Some of them will not be needed afterwards. Not all, probably not most, but some, and everyone in the room has worked that out before the first session ends. Any version of this argument that steps around that fact is worthless, because the people you are asking can do the arithmetic faster than you can.

This is where these programs stall, and the usual explanation is wrong. People get described as resistant to change, as though it were a temperament, a thing to be managed with communication. It is not. What is good for the firm is not automatically good for the individual, and where those diverge, a person protecting their position is not behaving irrationally. They are responding accurately to an incentive you set. What is in it for me is not an obstacle to the change. It is a question with an answer, and the work is to have one.

Programs fail when an organization is asked to participate in something whose consequences for its members have not been stated. People are not stupid. An operations team watching a maker-automation pilot knows exactly which question is not being answered, and the absence of an answer is itself an answer. They act on it, and the ones who act first are the ones with options, which is to say the ones you needed for the validator layer.

There is a second problem sitting on top of that one, and it has nothing to do with motivation. The people who understand a process best are the least able to say what it could become. That is not a criticism, it is how expertise works. After enough years the practice stops looking like a set of choices and starts looking like the shape of the world. Meanwhile the people detached enough to question it do not know it well enough to be believed, and should not be. Neither group can do this alone. The reimagining has to happen with both in the same room, and that is a design decision rather than something you hope will occur.

All of which sounds like a case for expecting these programs to fail. Most of them will. The reason I think it is tractable is that the honest answer to what is in it for me turns out to be better than the one most firms are prepared to give, and it has been sitting inside the operation the whole time.

Ask a senior person in fund operations how they learned to tell that a number is wrong before they can say why, and you will not get an answer about training.

They will describe volume. Two years as a maker, doing the work, getting it wrong in the ordinary ways and being corrected. Then two or three as a checker, which is a different education entirely: instead of doing the work once, you see a hundred people's versions of it, and you learn the shape of every way it fails. Then supervision, where the pattern recognition finally arrives.

That was a curriculum. Audit runs one. So does credit, and so does a medical residency, and every other place where judgment was learned by doing the work badly at first, in volume, under correction. This one ran for a decade, nobody designed it, and it was funded entirely as a byproduct of production. The firm believed it was paying for the work. It was also paying for the school, and the tuition was hidden inside the labor line.

Automate the maker layer and you have cancelled the first four years of the curriculum while keeping the requirement for its graduates.

The defining feature of this problem is that it does not present for years. Nothing breaks. The people who can sign are already senior and they keep signing. The damage is done quietly, by ordinary attrition and retirement against an intake that no longer exists, and it surfaces at the worst possible moment: when a firm has finally scaled machine production and discovers that the constraint has moved to the one thing it stopped manufacturing.

Here is the part I did not expect when I started working on it. The replacement is better than what it replaces.

In a validator model, the items that route to a human are by construction the hard ones. Independent methods disagreed, or the answer sat outside its distribution, or the evidence was weak. The old apprenticeship taught through volume, which meant a trainee spent most of a decade doing things they already knew how to do in order to encounter, occasionally, something instructive. The exception queue is that decade with the repetition removed.

Which is also the answer to the question this essay started with. The judgment a senior person hands over does not vanish into the system. It comes back out as the thing that decides which cases reach a human at all, which makes that person the author of the curriculum rather than its last graduate. The work changes from producing and checking to adjudicating and teaching, and for a certain kind of person that is a better job than the one they had. Not for everyone. For the ones it does not fit, you say so early, which is the end of this piece.

It is the densest curriculum a firm could build, and it will not happen by itself. Left alone, an exception queue optimizes for throughput, and throughput means routing every hard item to the person most likely to resolve it quickly, who is the most senior person available. That is the locally efficient choice and it is the one that closes the school. Five rules keep it open.

1. Route some exceptions for teaching value, not for speed. Reserve a deliberate share of the queue for people who will be slower at it. Name the cost and put it in the budget as training, because that is what it is. A firm that cannot say what percentage of its exception volume is routed pedagogically has not made this decision. It has let the routing rule make it.

2. Give juniors hard cases with a safety net, not easy cases. The instinct is to send low-consequence items to junior people. That protects the firm and teaches nothing, because low-consequence items are low-consequence precisely because they are not hard. Send difficult ones and put a second review behind them. The learning is in the difficulty. The protection comes from the review.

3. Require a stated reason, not a disposition. Approve and reject are not learning artifacts. The reason is the entire curriculum, and it is also how you find defects upstream, since a pattern in the reasons is a defect in the architecture.

4. Second-review a sample and teach from the disagreements. Where a junior and a senior reviewer reach different conclusions on the same item, you have a case worth more than any training deck. Collect them, discuss them as cases, keep them. In a couple of years that becomes the firm's actual body of institutional judgment, written down and transferable for the first time.

5. Calibrate the reviewers the way you calibrate the model. People have confidence and accuracy too, and they are no better calibrated than the system is. Track agreement against eventual outcomes. This is uncomfortable, and it is the only honest basis on which signing authority can be granted.

Which brings up the thing firms find hardest to say out loud. Today the right to sign is a function of title and tenure, and that worked because title and tenure correlated with a decade of the curriculum above. Break the correlation and the proxy stops working. Signing authority has to become an explicit credential: defined by scope, granted against evidence of calibrated judgment in that scope, recorded, reviewed, and revocable. The alternative is that the question of who is qualified to stand behind machine-produced work gets answered by an organization chart drawn for a different operating model.

None of that design survives contact with an organization that has not been told the truth, which is where this started.

What works is unglamorous. Say what shape the operation is meant to have and roughly when. Say which roles change and which end. Make the path from the current role to the new one concrete enough to be tested, because a promise that cannot be tested is heard as a holding statement. Where a role ends, say so early and directly, and pay for the exit properly. A firm that handles the endings badly is telling everyone who remains exactly what their own ending will look like.

And move the people before you move the volume. A validator layer staffed a quarter after production shifted is a validator layer staffed by people who watched the firm treat the sequence as an afterthought.

The machines will produce the work. Someone still has to be qualified to sign it, and that person is manufactured, not hired.