Note: Use at your own risk. Be ethical about usage. Further, I am not a professional programmer/developer.
I am using Automatic1111 because with some modification you can also use it with nearly every GPU. For other stable diffusion UIs you often need NVIDIA GPU.
Lord of the Rings
First I generated images of Lord of the Ring characters based on the book description. I chose the following setting: Model: Juggernaut XL (https://civitai.com/models/133005/juggernaut-xl); CFG: 5-7; 512x704; Sampler: DPM++ 2M Karras; Steps: 30; Upsacler1: 4x_NMKD-Siax_200k; Upsscaler2: ESRGAN_4x (visibility: 0.5).
I tried to stay as close as possible to the books. In order to still follow a systematic approach, I used the following prompt template:
Portrait of a [race] [gender] in their [age], [shape of face], [description of eyes], [description of hairstyle], [description of facial features], [description of outfit], [description of a random background].
Highlights
Here are four highlights I chose. I chose those images because the generated outcome surprised me quite a bit!
Quite grown up Legolas!
Super creppy Gollum
Gimli with a cool helmet but weird mustache
Goblin style Treebeard
All generation of LotR
Here you can enjoy the rest of the image generations in a slider.
Star Wars (mainly inspired by the Mandalorian)
I then decided to extent my work to Star Wars which I am a big fan of. Here I tried to apply a different model (rMadArt; https://civitai.com/models/18208?modelVersionId=248717 ) and different Loras (baby_yodah; difConsistency_detail; difConsistency_photo; can all be found on https://civitai.com/) to create funny/wild Star Wars character combinations (mainly inspired by the Mandalorian). The results are quite fun. Here are my personal favourites:
Mando and Batman Fusion.
Nice Skywalker Baywatch collab!
Grogu enjoying a cold pint.
My personal favourite: I told SD to generate an image of the Mandalorian as Spartan soldier, holding Captain Americas shield and Thors hammer. Everything worked out fine but the hammer. SD just made Thor himself the hammer. Apparently SD does have difficulties generating hammers. Still love the output!
For more creations, check the slider and enjoy!
Hope you had as much fun with this post than I did generating those images. I know there are still some issues - especially with hands, fingers, weird eyes and hammers - but Stable Diffusion is a lot about trying out. So happy trying and generating.