Comment by nightpool 3 days ago
Note that all of this reflects design decisions on Bluesky's closed-source "AppView" server—any federated servers interacting with Bluesky would need to construct their own timelines, and do not get the benefit of the work described here.
This is great to hear—my current understanding of the most recent state of the art on the topic is https://alice.bsky.sh/post/3laega7icmi2q which mentions that the self-hosted appview is not yet open source. So I'm glad to hear the situation has changed in the past 3 months.
The statement "any federated servers interacting with Bluesky" is ambiguous, because Bluesky's federated model means there's many different types of servers, and one user's view of what a "federated server" could be vastly different from another.
Federated PDS-s (which is probably the closest to what people mean when they say they want to federate on bluesky) would not need to reconstruct timelines if their users use the bsky.app appview.
Thanks, that's a fair point that I was overlooking. When I say a "federated server", I don't just mean a self-hosted PDS, I mean a third party app that potentially has its own lexicon and design decisions. Creating a robust third-party app that can meaningfully interact with the Bluesky network is still a very difficult engineering challenge, which I think this article does a good job demonstrating—that was the tension I was trying to underscore in my comment. Bluesky may be solving those engineering challenges for those clients who are satisfied with Bluesky's frontend and AppView, but every single other app built on top of ATProto will have to resolve those same challenges. This is directly downstream from Bluesky's "global firehose" topology and various design decisions that stem from that.
What reason does Bluesky give for not opening up their AppView code?
Another notable component that is closed source is the discovery feed generator, where at least there is some reason.
I asked this and got
> We did a backend rewrite from postgres to scylla and it has a bunch of deployment specific stuff, but is functionally identical to the open source postgres version. Its not really a "v2" in terms of new features, we just made it make use of our hardware really well[1]
Thanks, so are both the Postgres and Scylla versions maintained in terms of new features?
I wasn't aware that AppView v1 was open source, and the most recent info I'm aware of on the topic is https://alice.bsky.sh/post/3laega7icmi2q, https://github.com/bluesky-social/atproto/discussions/2961 and https://docs.bsky.app/docs/advanced-guides/federation-archit..., and everything I've heard about Bluesky was that open source appview is "still coming".
It's not coming, it never went away… As I understand it, the "business layer" with all the logic is above the data later, shared by the Postgres and Scylla versions, and the data layer just makes queries to the database. I think they are using the Postgres version locally for development.
The App View frontend is open source: https://github.com/bluesky-social/social-app
Much of the backend is open source as well: https://github.com/bluesky-social/atproto/tree/main/packages
What is not are the extra services they run to provide a better and faster UX. Even if it was open source, it likely costs 10s of thousands to run per month (they have moved largely to "onprem" hardware instead of the cloud aiui)
the backend (the AppView) can be found here:
https://github.com/bluesky-social/atproto/tree/main/packages...
there are various supporting services written in Go as well
App View is a bit fuzzy of a term. To me it seems like a combination of frontend, backend, custom lexicon, and supporting services. There isn't really another place in the spec or design where clients or browsers fit in, which do in fact provide a view of the network via an app.
"UI" is part of the definition they give in the glossary
when I read the spec it seemed like the operator of an AppView & Relay would be most in need of compensation for their hosting costs due to the amount of demand on those components so I believe the spec allows an operator to implement their own AppView & monetize it as that operator sees fit, so that they can afford to operate the service and maybe even make money off of it so that they can make it their full time jobs.
It seems this way to me as well. ATProto fundamentally changes how monetization works in social media by removing lockin. It's going to be interesting to see what emerges from this design decision.
Another interesting way to view ATProto is that it could be a collection of headless features and network browsers that leverage those feature providers.
what else? profit by means of doing work that benefits first and foremost the private proprietors of the closed source
if they gave it away (which used to be unfeasible until the digital era) they feel they’re loosing their valuable effort which they’re wont on concentrating, not diluting.
My thinking has evolved on this topic significantly as of late. My current thinking is we should create a secure gossip network on top of the Bluesky API, and forgot about all the DAG-CBOR stuff that gets stripped from the Jetstream. Hash the posts on the gossip layer and if posts change then diff them. This is all prep for when X billionaire buys out Bluesky then we just pop some signing key crypto on top of this gossip layer and wow! It's distributed!
As others have noted, the appview is open source. The dataplane has two implementations, one in postgres and another in scylla. The scylla dataplane is closed, the postgres one is open.
The interesting next stage for the postgres implementation is to create a sync engine for partial syncs of the network, so that an appview can run affordably. We ran some benches on the current state of the postgres implementation and found we could index 300k users on a $100/mo vps. I think with a couple of weeks of optimization that could reach 1mm users.