There are instances in Stable Diffusion where attempting complex poses may not yield the desired outcome, causing your body to contort into peculiar shapes. This occurs because Stable Diffusion can only identify poses that it has previously learned.
However, this does not mean that generating complex poses using Stable Diffusion is impossible. Rest assured, there is a solution: ControlNet OpenPose.
By utilizing ControlNet OpenPose, you can extract poses from images showcasing stick figures or ideal poses and generate images based on those same poses. This article will delve into the features, usage, and step-by-step process of ControlNet OpenPose, providing a comprehensive explanation.
I recommend checking out “Stable Diffusion Pose Prompt with 40 Examples” on prompt as well.
What is ControlNet OpenPose?
ControlNet OpenPose is a technology that extracts the body structure from an image and generates an image with the same pose. It has the ability to create stick figures and generate images based on them. OpenPose is the underlying technology that extracts the human body structure from photos, and ControlNet utilizes this technology to generate images.
Difference from “Specifying poses from prompts”
While it is possible to specify poses using prompts, ControlNet offers a significant advantage in terms of control over the generated pose. This makes it much easier to create desired poses, especially those that are complex. When relying solely on prompts, it can be challenging to accurately convey the pose or achieve the desired result, often leading to trial and error and wasting valuable time.
By leveraging OpenPose and practicing with ControlNet, we can save time and improve our skills in generating accurate and ideal images.
What can be achieved with ControlNet OpenPose?
ControlNet OpenPose offers two main functionalities:
- Pose extraction from images and generation of images with the same pose.
- Creation and editing of stick figures, followed by image generation with the same pose.
To delve into the details, let’s first examine the outcomes. The subsequent section will provide a thorough explanation of the steps involved.
Pose Extraction from Images and Image Generation with the Same Pose
The left side showcases a reference image of a cheerleader obtained from pakutaso, while the right side presents the product of pose extraction through ControlNet’s OpenPose and the resulting image based on that pose.
Stick Figure Creation and Image Generation with the Same Pose
The left side presents a stick figure crafted using the OpenPose Editor, as elaborated later on. On the right side, you can see the image generated from the stick figure. It is also feasible to extract the stick figure from an existing image and proceed with modifications.
How to Use ControlNet OpenPose
Setting Up ControlNet OpenPose
ControlNet OpenPose is a feature of the Stable Diffusion Web UI’s extension, ControlNet. To use ControlNet OpenPose, you must have ControlNet installed. If you have not installed it yet, please refer to the following article for instructions on how to install ControlNet:
Downloading ControlNet OpenPose Model
To use ControlNet OpenPose, you need to download the ControlNet’s OpenPose Model. Please download the following two files from the link below and place them in the stable-diffusion-webui/models/ControlNet directory:
- control_v11e_sd15_OpenPose.pth
- control_v11e_sd15_OpenPose.yaml
lllyasviel/ControlNet-v1-1 at main (huggingface.co)
How to Use ControlNet OpenPose
- Input the prompt to generate images.
- Click the “▼” button to access the ControlNet menu.
- Set the reference image in the ControlNet menu.
- Check the “Enable” checkbox in the ControlNet menu.
- Select “OpenPose” as the Control Type. This will set the Preprocessor and ControlNet Model.
- Click the “💥” button for feature extraction. Preprocessing will be applied, and the result of feature extraction will be displayed.
- Click “Generate” to produce an image. This will create an image with the ControlNet Openpose applied.
Afterwards, provide prompts or other inputs to generate the image. The image will be created with stick figures in the same pose.
Create/Edit Stick Figures and Generate Images with the Same Poses
Here, we will introduce a method to create stick figures and generate images with the same pose.
To create stick figures, install the OpenPose Editor. Once installed, restart the Stable Diffusion Web UI. It is recommended to specify the GitHub repository directly, as there are multiple plugins with the same name.
- Navigate to the “Extensions” tab.
- Select the “Install from URL” tab.
- Input the following URL: “https://github.com/fkunn1326/OpenPose-editor.git“.
- Click “Install”.
GitHub – fkunn1326/openpose-editor: Openpose Editor for AUTOMATIC1111’s stable-diffusion-webui
Creating Stick Figures with ControlNet OpenPose Editor
After the installation is completed, a tab named “OpenPose Editor” will be added. Follow these steps to create stick figures and send them to txt2img.
- Switch to the OpenPose Editor tab.
- Adjust the image size.
- Edit the pose.
- Send to the txt2img screen.
Go back to the txt2img screen and configure the ControlNet menu as follows:
- Check “Enable”.
- Select “OpenPose” as the Control Type.
- Choose “none” as the Preprocessor. This is because stick figures already have features.
Then, provide prompts or other inputs to generate the image. The image will be created with stick figures in the same pose.
Extracting Poses from Images and Editing Stick Figures with ControlNet OpenPose Editor
We have explained the steps for creating stick figures, but you can also extract poses from images and edit stick figures with ControlNet OpenPose Editor. The steps are as follows:
- Select an image and detect the pose.
- Edit the stick figure.
- Send it to txt2img.
Explanation of Setting Values
ControlNet OpenPose provides several options for preprocessing:
- OpenPose: This option allows for simple full-body detection.
- dw_OpenPose_full: This is a newer model that can detect face, hands, and full body.
- OpenPose_full: This model can detect face, hands, and full body.
- OpenPose_face: This model can detect face and full body.
- OpenPose_faceonly: This model specifically detects the face.
- OpenPose_hand: This model can detect hands and full body.
Let’s take a look at the results of each extraction:
FAQ
How can ControlNet OpenPose detect multiple people?
There is no specific action required to detect multiple people. If you input images with multiple persons into OpenPose, it will automatically extract the poses for each person.
How can I generate images with multiple people using ControlNet OpenPose?
To generate images with multiple people, you can extract stick figures from images that already have multiple individuals.
How do I save the extracted poses automatically?
To save the extracted poses, please refer to the provided article for step-by-step instructions. By setting this up, you can conveniently reuse your favorite poses.
ControlNet configuration: Multiple combinations, automatic saving of features, etc (kindanai.com)
ControlNet is not being applied
If ControlNet is not being applied, please ensure that the “Enable” box is checked.
ControlNet OpenPose is not recognizing
Occasionally, when the “Generate” button is clicked, ControlNet OpenPose may generate an image with completely different poses, resulting in a failure to recognize it.
OpenPose is a highly accurate model, so even if some body parts are not recognized, it is unlikely that the whole pose will be missed as long as people exist in the image. If recognition fails, please check the following:
- Verify that “none” is not selected in the Preprocessor.
- Ensure that the Model is not empty. If you are unable to select a model, it is likely that you have not downloaded it, so please try downloading the model.
- Don’t forget to click the feature extraction button (💥). This is a step that beginners often overlook, so please be mindful of it.
For cases of missing body parts, it is recommended to use the most accurate Preprocessor, dw_OpenPose_full.
コメント