The prompt was just an example and usually my prompts get quite a bit longer than that. But in 1.5 models I manage to get what I want to see eventually. I also find that throwing in qualifiers like “mesmerizing” does do something to the image, although in can be subtle.

However, what I wanted to say here was that in SDXL my prompting seems to go to nowhere and I feel I’m not able to get out the kind of image I have in my head. Keeping the prompt example, in SD1.5 using a custom model like Deliberate 2.0 I’m able to end up with an image of a hat wearing cat surround by surreal looking candy pops. (however the final prompt for this reads). In SDXL my images “break” (i.e start looking flat, unrefined or even bizarre) at some point long before I can direct them towards my imagined result. All my usual approaches like reducing CFG, re-ordering prompts, using a variety of qualifierts don’t seem to work like I’m used to.

And tbh, I think this has to be expected. These are new models, so we need new toools (prompts) to work with them. I just haven’t learned how to do it yet and I’m asking how others do it :)

Thank you for the input! I recently upgraded my PC to be able to handle Stable Diffusion, and I got 12GB of VRAM to work with at the moment. I also have recently started to self-host some applications on a VPS, so some basics are there.

As for what I’d like to do with Stable Diffusion: One of my hobbies is storytelling and worldbuilding. I would like to (one day) be able to work on a story with a LLM and then prompt it: “now give me a drawing of the character we just introduced to the story” and the LLM would automagically rope in Stable Diffusion and produce a workable drawing with it. I think that this is probably beyond the capability of the current tools, but this is what I would like to achieve. I will definitely look into langchain to see what I can do with it.

That’s also where the questions about context length and cross thread referencing come from. I did some work with ChatGPT and am amazed at how good a tool it is to “brainstorm with myself” in developing stories. However, it does not remember the story bits I’ve been working on 2 hours ago, which kinda bummed me out … :)

I look them up at lemmyverse.net

I go there about once a week to see if there are new communities I might be interested in. I’m on a selfhosted single-user instance, so my “all” is identical to my “subscribed” and this is how I populate my feed.

Yeah, reducing CFG can help a lot. It sometimes feels to me, that getting a good image is knowing at what point to let loose …