Comment by __MatrixMan__
Comment by __MatrixMan__ 9 hours ago
Are we talking about the same https://github.com/tweag/jupyenv ?
It encapsulates the kernel, which encapsulates pretty much everything for the notebook, right? I haven't worked with Slurm or OpenPBS, but I think if you let nix build the images that your tasks are running in then I think you're covered for pretty much everything except things that only exist at runtime like database connections. Not a perfect black box, but close.
> It encapsulates the kernel, which encapsulates pretty much everything for the notebook, right?
Not even close to everything. In real world the environment of a notebook consists of a bunch of things provided by whoever set up the lab, i.e. storage and tools.
Typical examples include setting up Lustre or Ceph in a way that it will be accessible from a notebook (but that would also involve authentication, potentially).
And, in terms of tools: a workload manager that's perhaps integrated with Jupyter to schedule notebook execution on available nodes, but also to simply run workloads. Also, just a bunch of stuff written by this or another research group. Just this week I had to install and configure some tool for arterial spin labeling, but that would be the case with any kind of research: there's plenty of stuff that researchers will rely on on, that is central to their research, but isn't directly related to Jupyter.
By and large, Jupyter is just a front-end to any particular system where research happens, it's not the system itself.