Programming Surface Studies
Programs on a FrogNet do not talk to each other. They share what they know.
That is the claim this page argues, and every other claim here is downstream of it. The Living Network earns the abstraction; memory states it. This page is about what memory does to the surface you program against, and what stops being application code when it changes.
Tuples and REST — the tables, api.php, the backgammon case. Linda: Gelernter and Carriero, Yale, 1985; the three-coordinate key is FrogNet’s convention, not Linda’s.
Tuples and REST, part 2.
Is message choreography the assembly language of distributed systems?
The last five bugs you chased.
- A message that arrived twice.
- A timeout tuned for a network that behaved differently on Tuesday.
- A service that came back up and rejoined with stale state.
- A retry that succeeded on the third attempt and left two half-finished things behind it.
- Something that only happened under load, in the environment you cannot attach a debugger to.
None of that was your actual job.
And the reflex that produces those bugs survives being told about it: the rule was broken six times in one day by the person who wrote it down. All six mistakes, and how you spot each →
Before
- Version an endpoint, coordinate a deploy order, update mocks in four suites.
- The integration environment is a distributed system of its own, and it is down.
- Three in the morning, a retry storm.
After
- Put the value in FrogNet Memory; whoever needs it reads it.
- The simulator runs the real code against a modelled topology, on a laptop.
- Nothing to retry — nobody is delivering anything.
Four terms recur, and they are not synonyms.
Each names a different layer. Getting them straight now saves a great deal of confusion later — and it is the difference between a claim about a network and a claim about a programming surface.
| Name | What it is |
|---|---|
| FrogNet | The distributed network operating environment — discovery, routing, transport, semantic compression, security, split and merge, elections. The living network itself. |
| FrogNet Memory | The distributed shared-memory architecture, implemented by api.php. Values written by any machine and read by any other. This is what you write to. |
| UnREST | The programming model and API for working with FrogNet Memory. The programming surface — not a protocol. |
| Tuple space | The computational model underneath it all: David Gelernter and Nicholas Carriero's Linda, 1985. The idea is not ours and we say so more than once. |
Everything on this page is a claim about the third row. The first row is plumbing, and it is not what makes the argument.
Every decade we raised the surface. Distributed systems sat that one out.
Each of these was sold as new capability. Each was really recurring complexity moved into the platform, so applications could get back to the actual problem.
| We stopped | We started | What stopped existing |
|---|---|---|
| machine instructions | languages | hand register allocation |
| disk sectors | files | block management code |
| scheduling by hand | threads | your own run loop |
| provisioning servers | VMs, containers | machine-specific everything |
| describing conversations | sharing state | endpoints, retries, schemas, queue topology |
Applications still program communication. That is the row that has not happened yet, and the first four are the reason to expect it will.
Check it against your own codebase. It takes about thirty seconds.
If your code depends on knowing where the data came from, you cannot properly mock or simulate it.
This is also what makes the simulator something other than a demo. FrogNet runs its real discovery, routing and election code against a modelled topology — because none of that code asks where anything came from. The simulator is evidence that the dependency is gone, not a mock-up of a network.
Which puts the division of labour where it belongs: applications test domain correctness; the platform tests communication correctness.
Derived from Magnum Croakus, Part XI — The Programming Surface. The books are free →
Comparing programming surfaces by measurement rather than by argument.
The claim this site makes is that describing conversations is a surface, and that a higher one exists. That is exactly the kind of claim people are right to be sceptical of, because it is easy to assert and hard to check. The simulator is what makes it checkable — and it is the same simulator, executing the same real discovery, routing, election, convergence and recovery code. No second laboratory is built for the purpose, because a laboratory built to prove a point is not a laboratory.
The framing is not that anybody did it wrong. It is: here is an extraordinary distributed system, built by excellent engineers; here is what the programming surface forced them to build; here is what changes if the surface changes. Showing that brilliant people had to write an enormous amount of communication machinery because the surface gave them no choice makes the point without insulting anyone — and the machinery is right there in their own documentation.
The method · Magnum Croakus ch. 41a
Three arms, one variable
Take a real distributed system whose source is public. Model it three ways, changing exactly one thing at a time.
Arm 1 · the existing architecture, as it is
Services and components, endpoints and message flows, ownership and authority, shared and replicated state, synchronisation, retry and recovery, test infrastructure, failure handling. Modelled to understand how it works today — not to criticise it.
Arm 2 · the same architecture, over FrogNet
The application is untouched; only the communications substrate is replaced. This isolates what topology, routing, transport and recovery contribute, with the programming model held constant.
Arm 3 · the same system, re-expressed through UnREST
Messages become shared operational state. Authority replaces message ownership. Applications maintain state rather than conversations. Only the programming surface changes.
Why the middle arm is the one that matters
Identical topology, identical workload, identical failures, identical correctness checks, across all three. Anyone can inspect the model, change the assumptions, rerun it, and disagree with the result — which is the only property that makes any of it worth publishing.
The middle arm is the one that keeps the exercise honest, and it is the one most likely to come back flat. "Not much changed" is a real possible outcome, and publishing it is what earns any belief in the third. A body of work that only ever reports the flattering arm is not a body of work.
What is being measured
Not throughput. Engineering cost. How much code exists only to support communication. How much testing validates communication rather than correctness. How much debugging reconstructs conversations instead of inspecting state. How much architecture exists solely because applications must describe who talks to whom.
Countable things: communication code, synchronisation logic, retries, message types, endpoints, ownership transfer, integration complexity, mock infrastructure, failure surface, convergence behaviour.
Two disciplines that make those numbers mean anything
Publish the classification rule before the counts. Deciding what is "communication code" is a judgement about every file, and making it case by case is grading your own homework. Write the rule once, apply it mechanically, and report how many files were genuinely ambiguous.
Name the asymmetry yourself. The original was written under deadlines, staffing constraints and changing requirements, by people who did not know the answer. The re-expression is written afterwards, by advocates who do. That does not invalidate the comparison, and pretending it is symmetric will be noticed.
The rule about sources
Only systems whose source is public, built by their own documented process. Repository, commit hash, build date, toolchain versions and the machine it ran on — recorded in the study rather than in a footnote, because a study against "the current version" is unreproducible six months later, and reproducibility is the entire claim.
If a project does not build clean from a fresh checkout, say what was needed. Not as a criticism: the next person reproducing the work needs the same steps, and "three things had to be patched" is itself a fact about the surface.
What cannot be measured this way
Some of the most instructive systems are commercial, and there is nothing to check out. Those still have something to offer — but they are predictions, not studies, and the difference has to be visible on the page. No counts, no percentages, nothing that looks like a result. State the belief, state that it is unproven, and state the method by which it could be shown wrong.
A prediction with a stated test is a claim someone can hold you to. A prediction without one is an opinion.
Prediction 01 · not a study
Photon — a shared world expressed as conversations
An authoritative-node architecture for real-time multiplayer, in production at very large scale, and good at the job it was built for. Photon is commercial and there is no source to check out, so under the rule above it does not qualify as a study. It is published here as a prediction: a belief, stated as unproven, with the test that would settle it.
What the system does
Distributes a shared game world across authoritative nodes, each owning a set of entities. Ownership changes hands as players move and load shifts. Interest management limits what each client is told about. It is a mature, heavily exercised design, and the reason it looks the way it does is that the programming surface is conversations between nodes.
What the surface forced
The parts of the architecture that exist only because applications describe conversations: message types and their versions, the authority handoff protocol, the ordering and idempotency rules that keep a handoff from being applied twice, reconnection and state catch-up for a client that dropped, and the test doubles that stand in for all of it. Entities do not physically move between machines — but the protocol that describes them moving has to exist, and be correct, and be tested.
What I believe changes
Expressed as shared operational state, authority still changes hands — but entities do not conceptually move, because there is no destination to move them to. Developers debug state rather than conversations: what is the value, and how stale is it, instead of which message was lost and in what order the two survivors arrived.
Equally specifically, what does not change: the tick loop, the physics, the interest management, and the game. None of those were a communication problem, and none of them get easier.
Why this is a prediction, and how to show it wrong
No counts appear above, and none will. A prediction from someone who has built that class of system is real evidence of a particular kind — it says: here is what the surface forced me to build, more than once, and here is what I believe changes. What it must not do is wear the clothes of a measurement.
The stated test. Take any open-source authoritative-node multiplayer server with a public repository. Model it in the three arms above at a named commit. If the communication machinery — message types, authority handoff, ordering and idempotency rules, catch-up, and the test doubles for all of it — does not substantially shrink in arm three, the prediction is wrong, and it will be published as wrong.
We have not reimplemented Photon. This describes what the model implies, from public material, and the honest thing is to say so. If someone who works on it finds a sentence wrong, we will fix it publicly and say who found it.
Four studies with real numbers is a reference; one is an anecdote. The prediction above is published to invite the arguments, and the first real study gets written better for having had them. The method in full is chapter 41a →
Three more, in the order they are being written.
All three have public source, so all three are studies rather than predictions — three arms each, selection rule and classification rule published before any counts.
Kubernetes operators
Reconciliation loops are already state-convergent, which makes this the most interesting case rather than the easiest one. The question is what is left of an operator when the state it converges on is shared rather than fetched.
ROS 2 robotics
Planners, controllers and simulators reading one shared state — while the hard real-time loops stay local and untouched, because those were never a distributed problem.
Enterprise microservices
The orchestration layer that existed to sequence calls has nothing left to sequence.
No diagram required.
A noticeboard in a shared kitchen does not send messages. Somebody writes "milk is finished" and it stays written until it stops being true. Nobody retries it. Nobody worries whether you were in the room when it was written. If you want to know, you look.
Almost every distributed program written today works the other way: it phones each person individually, hopes they picked up, calls back if they did not, and keeps a list of who has been told. That is a great deal of machinery to replace a noticeboard.
Assembly is fine. People still write it where it belongs.
Boost was not wrong. It was excellent, widely used, and it worked. What happened was that enough people did the same difficult thing often enough that the language absorbed it, and afterwards nobody wrote it by hand again. The transition happened because the recurring part became well enough understood to move down a layer.
So the honest claim is not that FrogNet is the new surface. It is that there is one to be had, and here is a working demonstration that the idea holds together. A video call that holds 720p at 850 kbps and is still up at 450 — picture thinned, audio intact — is a real measurement of a real link on a real day. It does not need help.
Is message choreography the assembly language of distributed systems?
Most of what you know has a premise.
Hand-tune your inner loops. Know your register pressure. Count cycles. Every one of those was correct, and every one was quietly conditional on something a compiler later removed. The people who worked out which wisdom had expired wrote the next thirty years of software.
A great deal of what this profession knows about distributed systems has the same shape. Real-time media rides UDP — because a late frame is worse than a lost one, which is true when you have no better way to shed load. Exactly-once delivery is impossible — for events, which compound when duplicated. Cache invalidation is hard — when a cache is a copy held at a distance from the authority. APIs must be versioned — when the producer owes a shape contract to named consumers. You need a service registry — when services have locations.
None of those is wrong. Each is downstream of a premise, and the premise is message passing.
Name a law in your own field. Recover the assumption that made it true. Check whether this substrate still supplies it.
That audit is open across every field and protocol, and almost none of it has been done. It needs no licence, no install, and nothing from us. And publishing "this one still holds, and here is why" is worth as much as finding one that has expired — arguably more, because it tells everyone else where not to dig.
Not every law expires. Reads scale by replication, and one elected host is a serialization point, which is a ceiling — the coherence win and the scaling cost are the same fact. Anyone doing this honestly will find laws that are still laws. That is what makes it research rather than advocacy.
A guild, not a shortlist.
The FrogNet Guild is a professional body with working groups, an open method, and no gate at the door. Membership is not granted — you join, you read the method, and you start arguing with it.
The subsystems are working groups rather than vacancies, and joining one costs nothing and asks nobody's permission. Most contributions are small: a topology the allocator gets wrong, a test case that fails, a paragraph of the method that could be gamed, a study arm rerun with different assumptions and a different answer. Publishing a flat result is as much a contribution as publishing a favourable one — a body of work that only reports what flatters it is not a body of work.
Taking responsibility for an area is a step further, and it is a step you take rather than one you are offered. Somebody eventually says "I'll own the specification for the wire format," or "I'll maintain the classification rule," and that person is whoever put in the work — not whoever asked first. There is no cap on how many people do that, and there is no seniority ladder to climb before you can.
Standards and governance are working groups too, with no code in them at all: the open specification the protocol is headed for, and the Foundation's formation underneath it. Neither needs engine access, a licence, or export screening.
A messiah has followers; a journeyman has a guild. Better gets demonstrated against the simulator rather than argued, so the work is judged on merit rather than on who made it — mine included.