Code and Let Live
(fly.io)428 points by usrme 2 days ago
428 points by usrme 2 days ago
I know you know this, as you posted it, but readers might want to look at this related thread:
Fly's Sprites.dev addresses dev environment sandboxes and API sandboxes together - https://news.ycombinator.com/item?id=46561089 - Jan 2026 (10 comments)
I have found container-use to be super useful for this.
https://container-use.com/quickstart
BTW Simon, I was super happy when I heard on Theo's podcast that he will be encouraging you to monetise your work more. I'm super appreciative of your work and I'm pretty convinced that the more you profit from it, the better the universe will be!!!
> BTW Simon, I was super happy when I heard on Theo's podcast that he will be encouraging you to monetise your work more.
There goes the neighborhood.
I really want to love this, but my experience in the first 20 seconds is unfortunately like some of my other experiences coding against Fly APIs, they're broken.
https://sprites.dev/api has this command:
$ curl -X POST "https://api.sprites.dev/v1/sprites" \ -H "Authorization: Bearer $SPRITES_TOKEN" \ -d '{"name": "my-sprite"}'
which responds with
{"error":"name is required"}
if you use the request body in the full "Create Sprite" documentation at https://sprites.dev/api/sprites#create then it does work.
can I live with some rough edges for some personal workflows that only impact me when things break? sure. however, I was thinking about playing with some CI/CD stuff using sprites that would impact our whole team if things broke and I'm really on the fence because of this experience in the first 20 seconds.
Fly team - please put some black box probes or just better testing on the example you give in the quick start. if you document it, test it.
Believe it or not, that's the only example that's not autogenerated from tests (yet).
https://github.com/superfly/sprites-js/tree/main/examples https://github.com/superfly/sprites-go/tree/main/examples https://github.com/superfly/sprites-py/tree/main/examples https://github.com/superfly/sprites-ex/tree/main/examples
yep that would fix it. just needs a little docs change.
a "quick start" really should just work when you copy paste them.
If it turns out that someone is willing to pay for some other protocol then they just have to hook it up to that MIME type and start serving. It might be possible that they can derive an implementation of that protocol from their data schemas.
If they hardcode JSON such a change would be breaking for their previous users.
Can this issue be reported?
I wish more companies had open issue trackers (some proprietary software have issues on Github for example, but, it doesn't need to be Github, just let people discuss issues in the open)
On the documentation, you install the NPM package via Anthropic: https://sprites.dev/api (on this page, if you select Node)
npm install @anthropic-ai/sprites
Is there some relationship between Anthropic and Fly.io that I didn't hear about?
I know it's one me for thinking this -- since the domain is fly.io -- but I was really hoping this is some local solution. Not self-hosted, but just local. A thin command line wrapper to something (docker? bubblewrap?) that gave me sort of a containerized "VM" experience for my local machine using CoW.
If you are on mac, you can use Coderunner[1]. It will run locally on your and execute any AI generated code in an apple container.
1. Coderunner - https://github.com/instavm/coderunner
Check out LXC and the wider Incus set of projects: https://linuxcontainers.org/incus/.
Running IncusOS on some local hardware with ZFS underneath is a phenomenally powerful sandbox.
Yeah I can make an lxc container called "ai" that has an ssh read key and then a few pre cloned projects. When I want to work I can clone and start it then get the same effect on my own hardware and for free. Just need a small little wrapper to make this a bit more streamlined
I might have missed this in the docs, but is there a way to fork/clone a sprite, or restore a checkpoint into a new one?
Use cases: set up my preferred env in one sprite and use that as a template for others; or fire off a few independent sprites with claude code exploring alternative solutions, then choose a winner and reap the rest.
Do you expect that to replace git worktree for getting Claude to work on multiple things in parallel? That was something I was curious about watching the demo video.
Can’t edit, but adding I noticed that there’s a limit of 3 sprites running concurrently for pay as you go, so that’s probably not a realistic day-to-day workflow.
> It's coming, and it'll make sense how and why next week when I run the "how this shit works" post.
Thanks! Also looking forward to reading the post :)
> the idea is just to get more real-world usage first
My particular wish notwithstanding, I agree with this.
This is seriously cool - it's exactly the DX and API I've been waiting for from sandboxed execution providers.
I'd love to be able to configure the base image/VM in a way that doesn't bundle coding tools or anything else I don't need, and comes with some other binaries installed (I'm more interested in using this as an API for a sandbox use-case I have). Is there a way to do this at the moment / is this on the roadmap?
Another option would be configuring the sprite via checkpoint and then cloning the checkpoint from a base sprite, but I don't see this option anywhere either.
This is on the roadmap. The open question right now is if we can just do "fork from checkpoint" for customized template environments, or if we need all the docker infrastructure.
If the fat bundled environment harmful for you, or just extra stuff you don't care about?
Not harmful for now - "fork from checkpoint" would be perfectly fine for me at the moment. The main issue (as flagged in the post) is that setting up additional tooling can take a while!
In the longer term, docker is nice from a reproducibility + CI perspective, and a docker build is already something can easily work with and track in my system.
One thing I've heard but not verified with other sandboxed execution providers is that startup times for custom images can be quite slow, so it could be a potential differentiator given Fly's existing infra.
Yes! It would be kinda cool to have the ability to docker-deploy (think the fly method even -- just to get your sprite on its feet the way YOU want it) a base sprite image and then just go from there in the normal sprite way from then on.
> Claude is a hyper-productive five-year-old savant. It’s uncannily smart, wants to stick its finger in every available electrical socket, and works best when you find a way to let it zap itself.
This alone was worth the upvote!
Philosophically, I like Fly and have been a customer since very early on.
That said, I dread having to do anything CLI related, which for hobby projects is like once every few weeks.
Glancing at the docs for Sprite, I worry that this will be another CLI where a good 95% of the time that I go to invoke a command, my workflow is interrupted by an auto-updater that takes longer than whatever interaction I'm trying to do and derails my train of thought.
How does the sprite ecosystem know when a sprite needs additional resources? If you start maxing out cores or ram usage gets too high you just automatically get allocated more cores/ram? (Assuming live/dynamically correct?)
I've been having so much fun working on sprites (and working with sprites) the last the several months. There's some neat parts of the Elixir side of this we're going to open source soon.
Also check out the 5 min demo we put out where I walk thru some sprite basics: https://www.youtube.com/watch?v=7BfTLlwO4hw
One of the coolest things about this is that Claude in his environment --- without him asking to --- knows how to drive Sprites. If you ask it to run a server, it will register it as a local service so it survives reboots. Without you asking to, it'll checkpoint when it makes big changes. I think this is kind of freaky.
I can't say enough how, if you're using this like Kurt and Chris have been, you have like, a dozen sleeping Sprites in your Sprite list. If you're not doing anything with them, they're not really costing you anything. When you want to do something new, there's no point figuring out which of your existing Sprites to do it on. Just make a new one.
Always having a sane place to run anything I happen to be doing, without making any decisions, it's a weird feeling.
That’s a great demo! For curious mere mortals, are all those custom instructions that make Claude know how to use it public? I’d like to learn how to drive it myself too, just out of curiosity!
Check out the skills that are installed on the box by default
Does anyone know of similar solutions that can be self-hosted? (without a 12 service stack like Daytona [1])
I'd really love a locally hostable limited version of this, so I can do some quick messing around before switching to the cloud version for long term usage. It cannot be too hard to spin up an API compatible version that just uses the local device + 20GB or something right?
Wow, this looks absolutely fantastic. Can't wait to take it for a spin. I'm actually surprised it isn't seeing more traction here!
In particular, I'm really excited about the extremely fast start up time and checkpointing. I'm curious if anyone knows any alternatives in this space?
Okay this is super interesting!
As I was reading this I was a bit confused by the issues they mention, but at work I use Claude SSHed to a persistent dev server and I’d be annoyed if I didn’t have eg my git repos there all the time or any part of that workflow was ephemeral. I’m not really aware of what everyone else is doing with sandboxes etc.
But the bit at the end with the MDM server made it click for me. I’ve started generating tiny iOS apps for personal software stuff, because they solve data storage better than the web (at least on iOS). A database on some other server seems like a bad fit/overkill for this stuff, client side storage is too flaky because Safari. But iOS apps are limiting in their own annoying ways compared to web apps.
This looks like a really interesting solution, I can just store the data on a sprite with SQLite or whatever. Visit its URL to use my app, then does it go away on its own after a short time? I could have done that before with a server with storage, but this seems easier/probably cheaper.
If this works well/the way I’m hoping it might be the sweet spot for simple personal software that needs persistent data and you want to run anywhere.
One feature that would make this really nice is if it could have something like Vercel preview environments, where I need to auth my fly account to view the URL. That'd solve the public URL without me needing to do my own auth thing in every app.
You can deploy from XCode to your iPhone, and it seems to behave like any other app when you do that. I do have a paid Apple developer account, and I think I read that if you don't then you have to re-sign the app every 7 days. If you wanted a small number of users then I don't think this would work. I think you could use TestFlight, which is Apple's method for distributing an unreleased version of an app, but I'm not sure what the review process would look like for that. Android would be much easier as long as you can still sideload APKs, you could just build the APK and send it to everyone to install. I read that there were some changes to sideloading APKs but I don't know the details.
In terms of actually making the app, I don't know Swift or iOS at all so it's all generated. Usual caveats, and I'm only running them on my own phone. I ask Claude (not code) to help me with the spec, I give it some bullet points and it asks a bunch of clarifying questions then gives me a spec. I put that in a new directory, fire up Claude and use the ralph-loop plugin (https://github.com/anthropics/claude-code/tree/main/plugins/...):
> /ralph-loop:ralph-loop "Implement the iOS app described in app-spec.md. You have access to xcode CLI tools. You should write tests and use them to verify your work. The task will be complete when the app is fully implemented, with all tests passing. Output <promise>COMPLETE</promise> when finished." --max-iterations 50 --completion-promise "COMPLETE"
Once it's done you can open the app in XCode, test it in a simulator, play with it and iterate a bit and then send it to your phone!
Editing to add because I can't edit the original post: I think the limiting factor here might be the concurrent sprites limit. It seems like if you're on pay-as-you-go then you can only have 3 running concurrently, and have to subscribe to get 10.
fly.io is doing really good work. I've super enjoyed building our product on their platform. I love fly-replay combined with super fast start-up.
I've been thinking a lot about how to run agents (and skills) securely while giving them a lot of powerful capabilities.
I recently used their macaroons library to turn arbitrary API keys (e.g. for stripe's API) into macaroons. I route requests for an upstream host (like stripe) through Envoy as a mitm proxy which injects the real creds after verifying the macaroon.
It is such a powerful pattern. I'm always worried about leaking sensitive keys through prompt injection attacks (or just sending them to anthropic), but in this model you can attenuate the keys (both capabilities & validity window) client side. The Envoy proxy lives inside my flycast network so it can't be accessed externally.
It would be so cool if fly built something like this into sprites.dev (though I can see how it would be spooky to have fly install their own certs for stripe, etc...)
If you read Ben Toews work on the tokenizer you have a good sense of where I want Sprites to go with key leaks and prompt injection:
Awesome stuff! Thanks for the reply.
Tokenizer is an explicit proxy though right?
My use case is very similar, but I wanted a transparent proxy so I could run unmodified scripts. It is a tricky design decision though.
I also mount a little fuse filesystem that mints macaroon on read (with a shorter lifetime, probably inspired by y'all but i forget from where).
I work on realtime collaboration of markdown files (currently in Obsidian), which has become a shared-context substrate for agents, skills, etc.. Our own company workspace has skills that have scoped access to fly, stripe, gmail, etc. We're definitely drinking the file-over-app personal-software-for-teams Kool-Aid, so the problem space for us includes access control and auditing.
Love your work :)
We have enough control over the execution environment in a Sprite (unlike a Fly Machine, where the implied Linux contract we have with our users gets in the way) that we can trivially hide explicit proxies.
We can also attach Macaroons to Fly Machines and Sprites for configurable ambient privileges, something I've wanted us to expose as a feature for a very long time.
Did you write up anything about this? Is this off the shelf behavior for Envoy or did you create this API yourself?
AFAIK fly.io run firecracker and cloud-hypervisor VMs. This seems to have a copy-on-write filesystem underneath.
Given their principled take on only trusting full-VM boundaries, I doubt they moved any of the storage stack into the untrusted VM.
So maybe a virtio-block device passing through discard to some underlying CoW storage stack, or maybe virtio-fs if it's running on ch instead of fc? Would be interesting to hear more about the underlying design choices and trade-offs.
Edit: from their website, "Since it's just ext4, you won't run into weird edge cases like you might with NFS or FUSE mounts. You can happily use shared memory files, for example, so you can run SQLite in all its modes." So it's a virtio block device supporting discard that's exposed to the VM. Interesting; fc doesn't support virtio discard passthrough, and support for ch is still in progress...
My impression is that you (very reasonably) treat anything inside the VM as untrusted. If you want trusted rollback, presumably that implies that the VM can't have any ability to tamper with the snapshot?
But maybe you have parts of the stack that don't need to be trusted inside the VM somehow? Looking forward to the article.
I want something like this, but running on my own box. I now have a Linux box with plenty of RAM and storage under my desk. (It happens to be an NVIDIA DGX Spark, but I'm not really interested in passing the GPU through to these sandboxed VMs; I know that's not practical anyway.) Maybe I'll see if I can hack together a local solution like this using Firecracker.
What about `docker run`? It'll be the same isolated container that keeps state. You can also mount some local directory
This looks great, i've been wanting a dev sandbox that doesn't run the risk of costing a lot if I forget to turn it off.
I had a few issues
1. manpath: can't set the locale; make sure $LC_* and $LANG are correct
suspect this is due to it inheriting locale from my local machine? easy to get around with some updates to .bashrc
2. the $SHELL environment in my sprite is `/opt/homebrew/bin/fish` I use fish on my local (mac + homebrew) machine and it seems to have inherited from my local machine, its nice to be using fish in the sprite, but seems weird that $SHELL in the sprite points to non-existent path. Slightly concerning that a local env var is being transferred to a remote machine without my explicit permission, I have some sensitive env vars locally.
Good point about quietly grabbing env vars, we can warn about that on first run. All it's getting are these:
var envVars []string
shellEnvVars := []string{
"BASH_VERSION",
"ZSH_VERSION",
"FISH_VERSION",
"KSH_VERSION",
"tcsh",
"SHELL",
}
If you want to skip this, running `sprite exec -tty /bin/bash --login` or similar avoids the magic.
> When you start a feature branch on your own, do you create an entirely new development environment to do it?
… yes? We have a few wrapper scripts around worktree operations that copy some docker volumes (pg data, bundle cache, etc.) from the base and spins up an entirely new stack on different ports with a host alias. We don’t have to install any deps beyond that because we copied over the ruby gems bundle cache and we’re using Yarn PnP + “zero installs” for client-side deps.
Wait - you have a repository with a dev environment, and now that you want a new feature branch, you’re creating an entirely new dev environment?
Maybe I’ve been isolated from The World for too long, but this sounds … unhealthy.
This seems cool, but beware that Fly's other products are not exactly models of stability and polish.
API downtime is a semi-frequent occurrence, as are transient API errors and slowness.
I've also had a ticket open with support for weeks due to rampant billing issues. For instance, a destroyed instance still shows up in my usage report as actively accruing billed time, and at a rate faster than is even possible (something like 2 hours for every 1 actual hour that has passed.)
They've released two new products in the AI space, this and Phoenix.new, and my worry is that they are focused on new products over making what they have good and reliable.
yeah nobody should use this based on reliability and support alone
On one hand it sounds cool. On the other, I feel like I missed it.
Is this just a fancy VPS like digital ocean with, https endpoint, snapshot and restore?
(Same thing goes for exe.dev)
Yes, plus:
* Near-instant creation
* Automatic spin-down scale-to-zero, so you're not paying for it when it's not in use.
If you're using these like we are internally, you've got like 2 dozen of them sitting around in the background sleeping. They're BIC disposable computers. "When in doubt just make another one."
Will you have higher tier pricing plans in the future? I don't see a way to sleep them (if you mean other than idle), and the max plan has 10 running concurrently
I see.
Also "containers" always had the option to attach durable storage via bind mounts.
I still get confused by the "this isn't containers" but it's kind of similar.
Maybe I am just too caught up in semantics.
A VPS that is instant to boot, super simple automatic routing and https proxy, with snapshot and durable is a win regardless.
"Containers" are that, and fast, in part because they share kernels, so there's no serious rebooting happening. But the consequence of that design is you share a kernel with untrusted cotenants.
And then there's just the idea of being able to pull these out of the sky literally whenever you want one. If you want to try something new out real quick, it makes no sense to figure out which of your existing Sprites to use. Just make a new one. If you're a little OCD, like I am, every once in awhile you can go prune, if you really care.
something that isn’t clear to me: what’s the billing when i’m not actively using a sprite? does that go to zero as well, or am i still being billed for storage?
If it's similar to cloudflare, then it should be usage based. That is you only pay for what is active. (ie: if you are running a task that is waiting on network for 1 hour, you don't pay for cpu but your app is loaded and you are paying for memory). So if your app is dormant (not using cpu or memory), you only pay for the storage you are using.
That's roughly what Cloudflare containers are right? (with migrations being the checkpoints?). Cloudflare containers are also nearly instant and have scale-to-zero pricing. The only difference here is the CLI?
Your pricing looks competitive on compute but roughly 4-5 times more expensive on memory and double on storage.
Basically endgame VPS. Instant creation, snapshotting, restore. Actually quite impressive even if you don't buy the whole Claude spiel.
If you have an LXD setup working for your own workloads that's working well for you, that's awesome. Why would we want to talk you out of that? Fundamentally you're getting at the difference between "elastic" cloud services and personal infrastructure. Personal infra is great!
If it helps: Jerome has been working for a couple months on a local, open-source Rust version of Sprites, so you can use the same DX with your own infrastructure. We just think this is the right "shape" for modern sandboxes, wherever you actually run them.
Glad to hear that the coming local version of Sprites will be open-source. I hope there will be some way to financially reward that work, aside from buying Fly services that I likely wouldn't use.
I like Partners In Health, myself. https://www.pih.org/
Playing around with this for a small amount of time, it is very neat but also there are a bunch of things that are unclear / undocumented (I assume the documentation is coming so I'm not faulting them for it not being there yet).
Some things that are unclear:
- How should I auth to github? sprite console doesn't use ssh (afaik) so I guess not agent forwarding?
- What on machine api's are available? Can I use the fly oidc provider[1]? There's a /.sprite/api.sock but curl'ing /v1/tokens/oidc gets a 404.
- How much is it going to cost me? I know there is pricing but its hard to figure out what actual usage would be like. Also I don't see any usage info in the webui right now.
To follow up on this a bit, something that I really want is a way to build and launch apps from an llm really easily. I am imagining and environment with a database, object storage, and a publicly reachable webserver. I think this could be that with OIDC auth to an s3 bucket and litestream.
I was previously thinking about doing the same thing on my homeserver with tailscale to expose the web interface publicly and tailscale oidc auth to an s3 bucket for object storage.
I saw this headline, saw the tweets and missed what this was about.
Then read Simon Willison's breakdown and got the 'Aha!'.
I like what they've done, played with it and immediately started to plan how I'd try to implement it myself.
I guess this will be the way to go, for development setups instead of using a dedicated machine. Especially when mobile clients are created for Sprites.
I liked this idea so much I signed up and linked my personal cc (to my job email) to try it out. Unfortunately, it keeps saying "You must add a credit card to use Sprites with this organization" - even though I just linked a card. No way to continue from there: it's a loop that shows my account with an "activate" button, clicking it shows the error and my account again. Fly.io says I have an account now and it's "in good standing".. :(
Like it, a lot. I think the future of software is going to be unimaginably dynamic. Maybe apps will not have statically defined feature sets, they will adjust themselves around what the user wants and the data it has access to. I’m not entirely sure what that looks like yet, but things like this are a step in that direction.
> I think the future of software is going to be unimaginably dynamic.
>...I’m not entirely sure what that looks like yet, but things like this are a step in that direction.
This made me stop and think for a moment as to what this would look like as well. I'm having trouble finding it, but I think there was a post by Joe Armstrong (of Erlang) that talked about globally (as in across system boundaries, not global as in global variable) addressable functions?
Not sure if I've read such an article, but it would be a reasonable next step from the globally addressable processes of the BEAM VM.
As I understand it Unison tries to do something like that but that might be wrong.
Treating agents like full computers instead of ephemeral sandboxes makes a lot of sense—durable state and checkpoints solve real pain points that stateless containers force you to work around. Curious how this approach scales when you need dozens or hundreds of Sprites at once.
i dont think i really get what this gives me over docker. everything i read is how i work for years
seems not to justify submitting to a proprietary single vendor solution where users are locked into opaque checkpoints they forgot how to migrate away from. this is not something made for users lets be clear. there are tens or hundreds of vm layers for defense in depth for docker so thats a non argument, no one says docker has to provide security its for tooling and common practices that allow vendor independence and moving to self hosted stacks as needed!
sprites.dev looks very interesting to me. Is there a way to set up a limit to how much scaling a sprite can get, or to set a spending limit? I wouldn't want to spin something up, and then be surprised by an unexpectedly high bill.
The sprite installer got stuck after "Installed to ..." for me. After waiting a few minutes I just ctrl+ced and looked at what it does after and manually ran "sprite auth setup --token <token>" and that seems to just hang for me.
I spun one up, started a server on port 8080, ran `sprite url`, it gave me a URL, that URL just has `{ "error": "unauthorized" }`. How am I supposed to access it?
Do we handle our own certs or do you have a proxy in front of the sprites that can do auto ssl stuff?
Oh, thanks, that works. ([edit] rewrote this whole post) I guess I need to install my own tunneling into the VM to do web development on it, but that's not so bad. The lack of regional support is crippling, because whatever region you put me in is ~200ms from me and the typing lag is terrible.
I'd love to adopt this for all my development (which I currently do using rented cloud instances, so I'm pretty comfortable with the remote development paradigm). I'm especially excited about the snapshot/clone pattern, and have (this past week) been researching solutions for exactly this problem.
Hope you launch multiple regions for this ASAP. Will be watching.
Without getting into Kurt's galaxy-brained take on the declining importance of "production" in a post-AI world, I'd say: yeah, run prod apps on Fly Machines, for more predictable performance, scaling, and pricing. Do exploratory computing --- "figuring out what you'd run on a Fly Machine" --- in Sprites.
It'd cool to create a MCP for this so you can have your agents run persistent code/other agents.
This is a large pain point today if you aren't technical, most of the chat interfaces just let you create frontend only apps.
You can do this now without an MCP, by auth'ing the `sprite` command inside of a Sprite and telling Claude to go document it for you. You can do things like "make me three versions of this feature on three different Sprites so I can compare them". It is spooky how easy it is to teach agents this stuff.
I thought fly.io snapshots weren't guaranteed to stick around? Although I can can't find the docs mentioning it, but i checked within the last few months... maybe they changed it?
it seems like when you snapshot, you snapshot memory AND the filesystem (immutable ftw), that's pretty awesome
i am dying to know: firecracker still? I know you have an upcoming post abt it, but i'm incredibly impatient when it comes to fool new infra
Alright nerd-snipe snooping research post happning now!
Seems like they are using JuiceFS under the hood, with an overlay root for your CoW semantics. JuiceFS gives them instant clone (because they're not cloning the whole rootfs), while the chnages to the overlay are done as an overlayfs and probably synced back to S3 via a custom block device they have mounted into firecracker.
You can also see they are using juicefs it for the "policy" directly (which I'm assuming is the network policy functionality). iirc juicefs has support for block devices too, so maybe they are using that to back the rootfs overlay.
One concerning thing is the `/var/lib/docker` mount - i ran this in an ubuntu container, did they... attach it? Maybe that's a coincidence, but docker is not installed on the sprite by default. (the terminal is also super busted when used through an ubuntu container)
https://pastebin.com/raw/kt6q9fuA (edit: moved terminal output to pastebin because it was so ugly here)
I played with a similar stack recently, my guess is they are: 1. making some base vm, snapshotting it 2. when you create a vm, they just restore a copy and push metadata to it (probably via one of the mounts) 3. any changes that you make to the rootfs are stored on the juicefs block device (the overlay), which is relatively minimal compared to the base os. JucieFS also supports snapshotting, so that's probably how they support memory + filesystem snapshot and restore so quick
interestingly, seems they provision maybe a max disk size of 100GB for total checkpoints?
```
NAME TYPE SIZE FSTYPE MOUNTPOINTS
loop0 loop 100G /.sprite/checkpoints/active
```
fuse is definitely being used within the VMM, i can see a fuse mount and id being assigned. They're probably using juicefs directly for the policy mount because that doesn't need to be local nvme-cached, just consistent. The local-nvme -> s3 write-through runs on the hypervisor through a custom block device they attach to the firecracker vmm. This might just be the --cache-dir + --writeback cache option in juicefs. Wild guess is just 1 file per block.
guessing the "s3" here is tigris, since fly.io seems to have a relatoinship with them, and that probably keeps latency down for the filesystem
i think firecracker, just snooping around a sprite i see a lot of virtio-mmio, which afaik CHV would be using PCI in those instances
This has been in the works for quite awhile here. We put a long bet on "slow create fast start/stop" --- which is a really interesting and useful shape for execution environments --- but it didn't make sense to sandboxers, so "fast create" has been the White Whale at Fly.io for over a year.
Not really. One of the primary features of sprites.dev that I don't see anywhere on exe.dev is a fast way to create and restore checkpoints, like a git repo for your entire VM.
This is needed for sandboxes if you don't want to throw them away and start over when something goes wrong.
With sprites.dev you can create an additional checkpoint and then turn Claude Code (or your preferred agent) loose to do anything. Even if it burns down the sandbox you can just restore a checkpoint in about a second.
[exe.dev co-founder here] If you are curious, we have a `clone` command coming soon for sub-section creation of a new VM out of an existing VM. This is our first pass at checkpointing, rather than introducing an independent `snapshot` noun, you can keep a VM around as the snapshot.
We realize that is not going to cover all the business cases we have been discussing with customers and plan to introduce a snapshot concept (in particular for rewinding the state of a VM to an automatic backup), but we have a lot of FS work underway before we can launch it. There are some other things we want out of our VMs that we cannot do using conventional cloud techniques, so we have code to write.
Yes that’s certainly a great feature and they don’t have it currently. For what it’s worth, they do have a teaser about “Persistent disks with some really interesting work coming soon.”
something simpler I've did, in the same spirit: LXC containers (using Incus) in a VM. LXC containers look and feel like VMs, but are very lightweight. And the VM they all run in provide the hard sandbox.
and when I spin up a new LXC container cloud-init sets it up with the agents and my repos inside
Yeah that's about right.
It's a fast starting and fast pausing persistent VM, with a ton of built in developer tools (including a preconfigured Claude Code) and an extra JSON API for executing commands within it so you can treat it as a sandbox.
You may find my writeup here useful: https://simonwillison.net/2026/Jan/9/sprites-dev/
How exactly can code agents make use of this? You install claude code inside a Sprite and run it there? Do you also need to put all your codebase in this sprite?
Will you guys get mad if I try to do something like transcription with a tiny model on a sprite?
I'm not really sure I get the value of these being remotely hosted. We're writing code on super powerful machines with hypervisors built in.
My libvirt setup does this right now, I have a little dumb cli I wrote that lets me create, start, stop, save, restore, and destroy preconfigured machines. I use it for testing provisioning scripts and playbooks. You get the full cloud experience by including a cloud-init ISO so you can ssh to it the moment it boots with my key. Didn't realize I was at the frontier of computing paradigms.
Don't get me wrong the interface fly has is super nice but it feels like the endgame isn't remote hosted computers but a nice user-friendly interface (i.e. what docker did) but it's for persistent local VMs.
Sure, but plenty of users don't want to have to do/configure all that locally, sorta like I want shared hosting vs my own VPS as a sort of analogy.
Thanks for the writeup on the libvirt setup. At some point I used local docker containers for this.
So this is neat and useful and I think will/should get traction.
So let's say sprite is my building/dev ground floor. I get my thing/app to where I want it, but at the end of the day I think my thing/app is so awesome that it should be a production app for the whole world, and, I want to actually deploy it on fly, say.
Have you guys thought about that workflow, and what it might take to push button/migrate a sprite app over to fly?
Also, any plans for GPU sprites?
It depends on which Fly person you talk to. If you talk to Kurt he'll try to sell you on his crazy dream of how all software is going to be malleable and "prod" doesn't mean anything anymore. If you ask me: tell Claude to make a Dockerfile of the current state of your Sprite, and then deploy it as a Fly Machine. It's a good question, and we're working out how the transition from Sprite to Fly Machine works, but that's how I'd do it today.
I don't think we're going to do anything new with GPUs any time soon.
Fly.io is so cool.
The way they force at least one "fuck" into all of their promo is just so cool.
Erm, I mean so fuck cool!
I'm really excited about https://sprites.dev/ - it hits two of my favourite problems at once:
1. Developer environment sandboxes. This is a cheap and convenient way to run Claude Code / Codex CLI / etc in YOLO mode in a persistent sandboxed VM with a restricted blast radius if something goes wrong.
2. Sandbox API. Fly now have a product that lets me make a simple JSON API call to run untrusted code in a new sandbox. There's even snapshotting support so I can roll back to a known state after running that code.
I wrote more a bunch more about this here: https://simonwillison.net/2026/Jan/9/sprites-dev/