ControlNet is equipped with numerous features making it challenging to determine which ones to utilize, isn’t it? To assist those individuals in resolving this predicament, we will be presenting practical cases of ControlNet in this article. This edition will concentrate on ControlNet Soft Edge. By perusing through this article, you will comprehend the following inquiries:
- What is ControlNet Soft Edge?
- What are some precise practical instances of ControlNet Soft Edge?
- How does ControlNet Soft Edge distinguish itself from comparable techniques such as ControlNet Lineart, Scribble, and Canny?
What is ControlNet Soft Edge?
ControlNet Soft Edge is a variant of ControlNet that specializes in extracting the edges of an image, essentially creating a sketch-like representation. While it shares similarities with Scribble, Soft Edge is capable of capturing even finer lines. What sets Soft Edge apart is its ability to extract line drawings with various patterns, offering multiple options for customization. Its selective extraction process allows for the preservation of intricate details, unaffected by noise.
Distinctions between “Prompted Image Generation” and “ControlNet Soft Edge”
When generating an image based on a prompt, the process relies on text input to create the desired output.
In contrast, ControlNet Soft Edge works by extracting the contours and crucial features from the original image and then proceeds to colorize it to generate the final image. This approach ensures that the resulting image retains the detailed information from the original contours. As a consequence, this technique provides precise control over composition, shape, and details, eliminating the need for repetitive image regeneration and offering convenience.
Here are some of the capabilities of ControlNet Soft Edge:
- Altering the texture, color, and appearance of materials
- Converting real-life images into illustrations, or vice versa
- Adding color to line drawings
Altering the texture, color, and appearance of materials
The image on the left represents the reference image, while the image on the right showcases the modified texture achieved using Soft Edge. In this example, the prompt was to tan the skin.
Converting real-life images into illustrations, or vice versa
The reference image is displayed on the left, while the converted illustration using Soft Edge is shown on the right.
The reference image is presented on the left, while the image transformed into a real-life representation using Scribble is exhibited on the right.
Adding color to line drawings
The left image displays the reference image, and the right image showcases the application of colors using Soft Edge.
Using ControlNet Soft Edge
Preparing for ControlNet Soft Edge
ControlNet Soft Edge is a feature of ControlNet, an extension of the Stable Diffusion Web UI. Therefore, in order to use ControlNet Soft Edge, ControlNet must be installed. If you haven’t installed it yet, please follow the instructions in the article below to install ControlNet.
Installing ControlNet Soft Edge
To use ControlNet Soft Edge, you need to have the ControlNet Model. Download the following two files from the link below and place them in stable-diffusion-webui/models/ControlNet.
- control_v11p_sd15_softedge.pth
- control_v11p_sd15_softedge.yaml
lllyasviel/ControlNet-v1-1 at main (huggingface.co)
Using ControlNet Soft Edge
Follow these steps to configure the ControlNet menu:
- Enter the prompt for the image you want to generate.
- Click the “▼” to open the ControlNet menu.
- Set the reference image in the ControlNet menu screen.
- Check “Enable” to activate ControlNet.
- Select “SoftEdge” for Control Type. This will set the Preprocessor and ControlNet Model.
- Click the feature extraction button “💥” to perform feature extraction. The results of the feature extraction will be displayed after the preprocessing is applied.
- ControlNet Soft Edge will now be applied. Click “Generate” to create the image.
Examples of Using ControlNet Soft Edge
Now, let’s go through the specific steps for the examples mentioned in “What Can You Do with ControlNet Soft Edge?”
Modifying Texture, Color, and Tone
ControlNet Soft Edge allows you to make adjustments to textures, colors, and tones. For instance, you can change the texture of someone’s skin in an image. If the skin appears blemished, you can enhance it to make it look flawless, alter the skin color, or even add sweat.
Let’s try giving the skin a tanned look.
Follow these settings in the ControlNet menu:
- Drag and drop the image into the ControlNet menu.
- Check the “Enable” box.
- Select “SoftEdge” for Control Type.
- Click the feature extraction button “💥”.
Next, input the prompt for the generated image.
Prompt: (brown skin, tanned skin: 2), 1girl, a 20 years old pretty Japanese girl in a classroom school uniform, blackboard
If you have any other specific settings for image generation, such as image size, feel free to adjust them.
The resulting image, with a tanned skin tone, is displayed below.
Converting Photographic Images into Illustrations, and Illustrations into Photographs
Converting Photographic Images into Illustrations:
Next, let’s convert a photographic image into an illustration. The settings in ControlNet remain the same as before.
- Change the model to an illustration-based model.
- Input the prompt for the generated image.
- Generate the image.
Model: AnythingV5Ink_ink.safetensors [a1535d0a42]
Prompt: 1girl, a 20 years old pretty Japanese girl in a classroom school uniform, blackboard
If you have any other specific settings for image generation, such as image size, feel free to adjust them.
The resulting image is shown below.
Converting Illustrations into Photographic Images:
Next, let’s convert an illustration into a photographic image. Please note that the settings in ControlNet are different from the previous ones. Follow the steps below to configure and extract features using ControlNet.
- Drag and drop the image into the ControlNet menu.
- Check the “Enable” box.
- Select “SoftEdge” for Control Type.
- Click the feature extraction button “💥”.
- Select “My Prompt is more important” in Control Mode. This is done to prioritize the prompt during image generation, as faithfully converting a line drawing can sometimes result in elements like large eyes.
Next, adjust the image generation settings.
- Since this is a photographic image, switch the model to a photographic image-based model.
- Input the prompt for the generated image.
- Generate the image.
Model: beautifulRealistic_brav5.safetensors [ac68270450]
Prompt: 1girl, a 20 years old pretty Japanese girl in a classroom school uniform, blackboard
If you have any other specific settings for image generation, such as image size, feel free to adjust them.
The resulting images are displayed below.
Coloring Line Drawings
Now, let’s discuss coloring line drawings. The settings for texture, color, and tone are the same as before.
- Drag and drop the image into the ControlNet menu.
- Check the “Enable” box.
- Select “SoftEdge” for Control Type.
- Click the feature extraction button “💥”.
Next, adjust the image generation settings.
- Switch the model to an illustration-based model.
- Input the prompt for the generated image.
- Generate the image.
Model: AnythingV5Ink_ink.safetensors [a1535d0a42]
Prompt: 1girl, suits, question
If you have any other specific settings for image generation, such as image size, feel free to adjust them.
The results are shown below.
ControlNet Soft Edge Settings
Preprocessor Options
- Soft Edge HED: This option applies a soft edge effect to represent edges with varying widths and brightness. It produces smoother lines that retain more details and content compared to fixed-width edges like Canny.
- Soft Edge HEDSafe: Similar to Soft Edge HED, this option includes additional safety measures to prevent the generation of inappropriate content.
- Soft Edge PIDINet: This option utilizes deep neural networks to extract contours from images.
- Soft Edge PIDISafe: Similar to Soft Edge PIDINet, this option also includes additional safety measures to prevent the generation of inappropriate content.
コメント