Two agents held the same lock: /var vs /private/var

I write a coordination server for AI coding agents. The core promise is small enough to state in one line: before an agent edits a file it asks for a lock, and two agents can never hold the same file at once.

Last week I found out that promise was false in two different ways, and both had been sitting there for months, passing a test suite.

The bug you can see

I was writing a demo script. Two agents, one file, run it and watch the second one get denied. Thirty seconds, no setup, the kind of thing that belongs in a README. It refused every lock:

propose_file_access -> REJECTED
Cannot lock files outside the project root.

The file was very obviously inside the project root. I had just created it there. The containment check looked like this:

const projectRoot = path.resolve(process.cwd());
const absolutePath = path.resolve(projectRoot, filePath);
const relativePath = path.relative(projectRoot, absolutePath);
if (relativePath.startsWith("..")) return { valid: false, ... };

That reads fine. It is also wrong on macOS, because /var is a symlink to /private/var. My demo created its scratch repo with mkdtemp, which handed back /var/folders/.../demo. Then process.cwd() reported /private/var/folders/.../demo, because the kernel resolves the symlink and Node asks the kernel.

path.resolve normalizes . and .. and separators. It does not touch the filesystem, so it does not follow symlinks. Two names for one directory, compared as strings, disagree. path.relative dutifully reported that the file was outside the tree.

This is the good kind of bug. It fails loudly, in the safe direction, and it took ten minutes to find because the error message was right there.

The bug you cannot see

Then I looked at the function next to it, the one that generates the lock key:

let normalized = filePath.replace(/\/+$/, "");
const cwd = process.cwd().replace(/\/+$/, "");
if (normalized.startsWith(cwd + "/")) {
    normalized = normalized.slice(cwd.length + 1);
}

Same assumption. Strip the project root prefix so that /repo/src/auth.ts and src/auth.ts produce the same key, and therefore the same lock.

Now run the symlink through it. Agent A, launched inside the resolved path, sends src/auth.ts and gets the key src/auth.ts. Agent B, launched through the symlinked path, sends /var/folders/.../demo/src/auth.ts. The prefix does not match, nothing is stripped, and the key becomes var/folders/.../demo/src/auth.ts.

Two keys. One file. Both agents get GRANTED.

The lock system reports success, the board shows two happy agents, and they write over each other. There is no error anywhere, because from the server's point of view nothing went wrong. It was asked about two different files and it answered correctly about both. That is the failure mode worth being afraid of, not the one that throws.

A lock key is an identity function

What I had built was a string comparison wearing a lock's clothing. Mutual exclusion over named resources decomposes into two pieces: an identity function that maps a name to a canonical key, and a claim on that key that is atomic against concurrent claimants.

I had spent all my care on the second one. Job claims go through SELECT ... FOR UPDATE SKIP LOCKED inside the transaction. File locks are a single INSERT ... ON CONFLICT against a primary key. There is a load test that throws a thousand concurrent attempts at one file and asserts exactly one winner. That test passes. It has always passed.

None of it matters if step one hands you two different keys for one file. Your atomic claim is atomic on the wrong thing. Postgres will happily enforce mutual exclusion over a key that does not correspond to anything real.

And the identity function is where the ugliness lives, because names for files are not canonical by nature: absolute versus relative, symlinked ancestors, case-insensitive filesystems where Auth.ts and auth.ts are one file on APFS and two on ext4, Unicode normalization where macOS stores decomposed forms and Linux does not, trailing slashes, doubled separators.

The one I had already been bitten by: an agent whose shell working directory is the repo's parent, sending myrepo/src/auth.ts, which coexisted happily with src/auth.ts as a separate key. I had seen that in production. There is a comment in my test file, written by me, saying two agents were observed holding granted locks on one file. I fixed that spelling, added a test, and moved on without asking the general question. I had patched one instance of a category.

The fix, and the part that surprised me

Resolve symlinks before comparing. The wrinkle is that agents legitimately lock files that do not exist yet, because they are about to create them, and realpath throws on a missing leaf. So you walk up to the deepest ancestor that does exist, canonicalize that, and re-append the rest:

// realpathOrSelf is realpathSync wrapped in a try/catch returning its input.
function canonicalizePath(absolute) {
    const pending = [];
    let current = absolute;
    while (current && current !== path.dirname(current)) {
        if (existsSync(current)) {
            return path.join(realpathOrSelf(current), ...pending);
        }
        pending.unshift(path.basename(current));
        current = path.dirname(current);
    }
    return absolute;
}

Then use it on both sides of every comparison, in the key generator and the containment check alike.

The surprise was in the tests. I wrote the obvious one:

expect((await lock("agent-A", "src/z.ts")).status).toBe("GRANTED");
const second = await lock("agent-B", symlinkedPathToSameFile);
expect(second.status).not.toBe("GRANTED");

Then, before trusting it, I reverted my fix and ran it. It passed.

It passed because on the broken code the second call came back REJECTED, the containment failure from the first half of this post, not REQUIRES_ORCHESTRATION, which is what a real lock conflict returns. My assertion said "not granted", and "rejected for a completely unrelated reason" is technically not granted. The test was green for a reason that had nothing to do with locking.

A negative assertion invites this. not.toBe(X) passes for every value in the universe except one, including all the values that mean your feature is broken in a new way. The fixed version pins the status and requires the holder's name:

expect(second.status).toBe("REQUIRES_ORCHESTRATION");
expect(second.message).toContain("agent-A");

That one fails against the broken code, which is the only property that makes it a test rather than a decoration.

What I would tell past me

Ask whether your key is canonical, separately from whether your claim is atomic. They are different problems with different tools, and the second is much more fun to work on, which is exactly why it gets the attention. Atomicity is a database feature you can look up. Canonicality is a domain problem nobody solves for you.

A refused lock is a bug report. A double grant is an outage you hear about next week. When you see the loud one, go looking for the quiet one immediately. They usually share a root cause. Mine did: one wrong assumption, two functions, opposite symptoms.

Revert your fix and watch the test fail. Every time you write a regression test, not just the important ones. It costs thirty seconds and it is the only thing between you and a suite full of assertions that have never once been in a position to fail.

When you fix an instance, name the category. I had already seen two lock keys for one file. I fixed that spelling of it and did not ask what else could spell a path differently. The answer was: quite a lot.

I maintain Axis, an MCP coordination server for AI coding agents. The code above is from its lock path, which is AGPL and on GitHub if you want to read the whole thing.