Atproto for Distributed Systems Engineers
The AT Protocol decentralizes a social network by turning a standard stream-processing backend inside out — making each internal service a public API that anyone can run — rather than relying on federation or blockchain consensus.
ActivityPub federation works well until a popular account on one instance sends a post to followers on ten thousand others. ATProto’s architects looked at that failure mode and decided the right mental model for an open social network was not federation — it was Kafka with public topics.
Source
Atproto for distributed systems engineers
By: Bluesky / atproto.com
Published: 2026
Type: Engineering article
The post earns its place in this collection by being one of the clearest written explanations of why the architectural choices in ATProto are what they are. It walks from “one SQL database” to stream processing before introducing the protocol, making the design decisions arrive with their motivations rather than as unexplained choices.
Turning the backend inside out
The standard path for scaling a social network is well understood by now. You start with a SQL database behind an app server, add caches when reads get expensive, move to a NoSQL cluster when strong consistency becomes the bottleneck, and then discover that without SQL you’ve lost joins and aggregation queries. The solution to that last problem is precomputed views — programs that listen to a stream of writes and maintain fast, denormalized read models. You need a durable event log so those view servers can replay missed updates after crashes. When those pieces are in place, you have a stream processing architecture, and it scales to hundreds of millions of users because you’ve traded strong consistency for eventual consistency in exchange for the ability to add capacity horizontally at every layer.
ATProto starts from that architecture and asks: what would it take to open every internal component to the public?
The NoSQL cluster becomes Personal Data Servers. Each user has a data repository — a structured collection of signed JSON documents organized into named collections. Records inside those collections are addressed by AT-URIs that encode the host, the user’s DID, the collection name, and a record key. Signing each record cryptographically means any server that receives a copy of that record can verify its authenticity without trusting the server that sent it. The data travels across operators; the signing key stays with the user’s original server.
The event log becomes a public Relay — a firehose that any downstream consumer can subscribe to. This is the same pattern as Kafka topics in a centralized architecture, except the topic is not private infrastructure. It broadcasts every write that any Personal Data Server emits, across the entire network. Downstream consumers subscribe once and receive the stream; they don’t need to know about individual user servers or poll them separately.
The view servers become Appviews. An Appview subscribes to the Relay, processes the event stream, and maintains the materialized views that a specific application needs. The Bluesky app’s Appview, for example, builds follower graphs, like counts, and feed indexes from the same raw event stream that any other application could use. If you want to build a different social application on ATProto — a photography community, a news aggregator, a code-sharing platform — you write an Appview that consumes the same stream and builds the views your application requires.
This creates a circular data flow that is worth tracing through explicitly. A user logs in with OAuth, which tells your Appview which Personal Data Server holds their repository and grants permission to read and write it. A write from your application commits a record to the user’s repository, which fires an event into the Relay, which fans out to every Appview subscribed to that relay, including your own. You listen to the event stream even for writes you initiated, because you are not the only source of writes for users you care about. Other applications write to the same repositories; the relay delivers all of it.
The “no steps backwards” constraint — which the post mentions was established before Bluesky was started — explains why ATProto chose this model over federation. In ActivityPub federation, a popular post on one instance might trigger direct push notifications to followers spread across thousands of other instances. Each receiving instance processes the delivery independently and must fan out further to its own followers. The network topology becomes a directed graph of push connections, and under load the push cost scales with the number of cross-instance follower relationships, not with the number of posts. ATProto’s model inverts this: upstream components emit events once into the relay; downstream components pull from it. The relay bears the fan-out cost, and because the relay is a purpose-built streaming component rather than a general-purpose ActivityPub inbox, it can be scaled independently from the application layer.
The practical consequence is that ATProto’s architecture remains scalable in the same way that centralized social networks are scalable, while the canonical data lives on servers that users choose and control. The scaling techniques used in any large-scale stream-processing deployment — partitioned topics, consumer groups, incremental view maintenance — apply directly. The open-ness is in the protocol, not in the choice of a lower-throughput consistency model.
The authority and moderation questions are where the architecture is most unresolved. Any Appview can build any view; there’s no authoritative global ranking or search. Labeling services can attach moderation signals to records, but the labeling layer is still young. The protocol is explicit that this is an area of active development rather than a solved problem.
My take
The reframe worth carrying forward is that “open” and “scalable” are not in tension if you choose the right abstraction boundary. ATProto draws the boundary at the data repository and the event stream, making those public while leaving the query infrastructure — the Appview — free to be as purpose-built and high-performance as a centralized application’s read layer. The openness is structural: your data is addressable, portable, and signed regardless of which Appview is querying it. The performance is also structural: the query layer doesn’t have to be open because the underlying data is.
Where this is useful
- Designing protocols for data portability where you want application ecosystems to share user data without all agreeing on query semantics
- Understanding why federated push architectures hit throughput limits and what the pull-based alternative looks like at the protocol level
- When NOT applicable: small-scale applications where the operational overhead of running Personal Data Servers, Relays, and Appviews as separate components is not justified by the scale or openness requirements
Apply this: When designing a multi-application data sharing protocol, ask whether downstream consumers should push updates to each other or subscribe to a shared event stream — the pull model scales horizontally in ways that cross-instance push delivery cannot.
What I want to remember
Remember: ATProto decentralizes a social network by making the canonical data layer (signed user repositories) and the event propagation layer (the relay) public APIs, while leaving each application’s query layer free to be as specialized as needed — openness at the data level does not require sacrificing performance at the read level.