Comment by refulgentis

Comment by refulgentis 2 days ago

I'm embarrassed to ask: can someone elaborate on, say, what we have now that we didn't have before the repo existed?

I have studiously avoided making models, though I've been adjacent to their output for years now... I think the root of my confusion is I kinda assumed there was already PyTorch based scripts for inference / training. (I assumed _at least_ inference scripts were released with models, and kinda figured fine-tuning / training ones were too)

So then I'm not sure if I'm just looking at a clean room / dirty room rewrite of those. Or maybe everyone is using "PyTorch" but it's usually calling into CUDA/C/some proprietary thingy that is much harder to grok than a pure PyTorch impl?

Anyways, these arent great guesses, so I'll stop myself here. :)

_tqr3 2 days ago

Stability AI, creators of Stable Diffusion models release their products under own Stability AI Community License which is not "free" like MIT license. You are not allowed to modify the weights in certain ways.

This package is basically running the model (inference) and maybe fine tuning it using existing AI weights. A great way to learn but still could run into same licensing issue.

Reply View 3 replies

refulgentis 2 days ago

You can't finetune SD 3.5!?
I thought the community license stuff was about keeping people from using it in prod and charging for it without Stability getting at least a small taste.
This sucks.
I haven't been keeping up with gooner squad on Civit, but I did have some understanding SD was less popular, but I thought it was just because 3.5 came far too long after Flux with too little, if any, quality increase to be worth building new scaffolding for.

Reply View | 2 replies
- fc417fc802 2 days ago
  
  > You can't finetune SD 3.5!?
  They don't want you finetuning it in specific ways that might make them look bad by association.
  
  Reply View | 0 replies
- djhn 12 hours ago
  
  So, out of interest, what are good TLDR sources for following the gooner scene? Like some highlights newsletter, subreddit, podcast, youtube channel or something? I’m interested in keeping up with their methods, not their results and output.
  
  Reply View | 0 replies

rockemsockem 2 days ago

I believe this is the main piece

> with minimal dependencies

I haven't tried running SD 3.5 specifically, but it's built on hugging face libraries which I personally always find to be a mess of dependencies that make it really hard to setup without the exact configuration the original developers used (which is often not provided in enough detail to actually work). This makes it pretty hard to run certain models especially if it's a few months/years after the original release.

For example this appears to be the requirements for the stability AI reference implementation for SD3.5 and there are no versions specified and it includes "transformers" which is just an enormous library.

https://github.com/Stability-AI/sd3.5/blob/main/requirements...

Reply View 1 reply

refulgentis 2 days ago

Ah, tyvm, that maps well onto my knowledge set, I have a ONNX inference wrapper written in Dart. However, I have never been able to leverage transformers.js ONNX demo code, i.e. have a reference to port to Dart.
IIRC it is written in an abstraction layer that supports a transformers-like API surface. This also makes it opaque to figure out what you're actually passing to the model, adding a Python dep mess on top of that...woo boy.

Reply View | 0 replies