Hi there, I’m curious to know other people’s approach in working with Stable Diffusion. I’m just a hobbyist myself and work on creating images to illustrate the fictional worlds I’m building for fun.

However, I find that getting very specific images (that are still visually pleasing) is really difficult.

So, how do you approach it? Are you trying to “force” your imagined picture out by making use of control net, inpainting and img2img? I find that this approach usually leeds to the exact image composition I’m after but will yield completely ugly pictures. Even after hours of inpainting the best I can get to is “sorta ok’ish”, surely far away from “stunning”. I played around with control net for dozens of hours already, experimenting with multi-control, weighting, control net only in parts of the image, different starting and ending steps, … but it’s only kinda getting there.

Now, opposed to that, a few prompts can generate really stunning images, but they will usually only vaguely resemble what I had in mind (if it’s anything else than a person in a generic pose). Composing an image by only prompts is by no means easier/faster than the more direct approach mentioned above. And I seem to always arrive at a point where the “prompt breaks”. Don’t know how to describe this, but in my experience when I’m getting too specific in prompting, the resulting image will suddenly become ugly (like architecture that is too closely described in the prompt having all wrong angles suddenly).

So, how to you approach image generation? Do you give a few prompts and see what SD can spit out with that? Taking delight in the unexpected results and explore visual styles more than specific image compositions? Or are you trying to be stubborn like me and want to use it as a tool for illustrating imagination - which at the latter it doesn’t seem nearly as good at as at the former.

  • DrakeRichards@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    1 year ago

    I’ve found that AI generation is good if you have a vague idea of what you want, but it can be frustrating if have something specific in mind. I try to approach generation with that in mind: I’ll plan out the large points of what I want but keep an open mind on the finer details.

    If I wanted to generate a more specific image, I would first try to do a sketch in another program and then feed that into ControlNet. I haven’t actually done this though since I’m usually able to get something close enough that I can work with.