ControlNet 1.1’s new feature: Lineart, coloring, and high-quality image generation from line drawings.

In this article, we aim to discuss the latest features introduced in ControlNet 1.1, namely Lineart and Anime Lineart. By going through this article, you will gain insights into the following queries:

• How can Lineart and Anime Lineart be effectively utilized?
• How can Lineart and Anime Lineart be distinguished from each other?
• What are the notable performance attributes of Lineart and Anime Lineart?

ControlNet Lineart technology provides a versatile solution for making various modifications to images.

With ControlNet LineArt, you can achieve the following:

• Alter the texture and appearance of objects.
• Illustrate real-life images and convert illustrations into photographs.
• Colorize images.

ControlNet Lineart has the capability to modify surface textures and appearances of buildings, wallpapers, and human skin.

Moreover, ControlNet Lineart can extract lineart from real-life images and produce illustrations.

On the other hand, ControlNet Anime Lineart can generate lifelike images from illustrations and sketches. It can extract lineart from illustrations to create realistic images or generate photographs from sketches.

Additionally, ControlNet Anime Lineart enables the coloring of lineart, including anime-style coloring, galge-style coloring, and thick coloring. By switching models, it allows for greater flexibility in the coloring process.

In summary, ControlNet Lineart technology offers a wide array of possibilities for modifying and enhancing images.

How to Use ControlNet Lineart&Anime Lineart

Installing ControlNet

ControlNet Reference is a feature of ControlNet, an extension of the Stable Diffusion Web UI. To use ControlNet Reference, you must have ControlNet installed. If you haven’t installed it yet, please follow the instructions in the following article to install ControlNet.

What is ControlNet? What Can It Do? A Comprehensive Guide to Installing ControlNet on Stable Diffusion Web UI – (kindanai.com)

Downloading ControlNet Lineart&Anime Lineart Models

To use ControlNet Lineart&Anime Lineart, you need to download the ControlNet models for each one. Please download the following two files from the links provided and place them in stable-diffusion-webui/models/ControlNet.

For ControlNet Lineart:

• control_v11p_sd15_lineart.pth
• control_v11p_sd15_lineart.yaml

For Anime Lineart:

• control_v11p_sd15s2_lineart_anime.pth
• control_v11p_sd15s2_lineart_anime.yaml

lllyasviel/ControlNet-v1-1 at main (huggingface.co)

How to Use ControlNet Lineart in the Web UI

First, open the ControlNet menu.

Follow these steps to configure the ControlNet menu:

1. Drag and drop the image onto the ControlNet menu screen.
2. Check the “Enable” box.
3. Select “Lineart” under Control Type.
4. Click the feature extraction button “💥”.

Next, enter the prompts for generating the image. If you want to change other settings such as image size, make the necessary modifications. Click “Generate” to generate an image with lineart.

How to Use ControlNet Anime Lineart in the Web UI

First, open the ControlNet menu.

Follow these steps to configure the ControlNet menu:

1. Set the image.
2. Check the “Enable” box.
3. Select “Lineart” for Control Type.
4. Select “lineart_anime_denoise” for Preprocessor.
5. Switch the Model to “control_v11_p_sd15s_lineart_anime”.
6. Extract the features.
7. Generate the image.

Next, enter the prompts for generating the image. If you want to change other settings such as image size, make the necessary modifications.

Which is the superior option: ControlNet Lineart or Lineart anime?

If your goal is to capture thin and intricate lines, I would suggest opting for Lineart anime.

In general, it is common practice to use Lineart for reference images that are in live-action format, whereas illustrations or line drawings tend to utilize Lineart anime. While this approach is not incorrect, personally, I would recommend choosing Lineart anime when discerning fine lines is your priority, and sticking with Lineart otherwise.

Changing Texture and Texture Change

Experimenting with Skin Tanning

Prompt: (brown skin, tanned skin: 2), 1 girl, a 20-year-old attractive Japanese girl wearing a school uniform in a classroom with a blackboard. Set up the ControlNet using the following steps:

1. Set the image.
2. Enable the “Enable” option.
3. Select Lineart.
4. Perform feature extraction.

Transforming into Autumn Leaves

Prompt: Enter “forest, autumn leaves” and set up the image. Perform feature extraction using the same settings as in the previous example of “Experimenting with Skin Tanning.” Ensure that the “Enable” option is checked.

After generating the image, you will notice the transformation from vibrant green leaves to autumn leaves.

Converting Live Images to Illustrations, Illustrating Photos

Converting Live Images to Illustrations

To convert a live image into an illustration, follow these steps:

1. Set the image to be transformed.
2. Enable the transformation.
3. Select the Lineart option.
4. Perform feature extraction.

Next, switch the model to an anime-style model. For this, use the AnythingV5Ink_ink.safetensors model.

Now, the live image has been successfully converted into an illustration.

Generating Live Images from Illustrations

To generate live images from illustrations, follow these steps:

1. Set the illustration image.
2. Enable the transformation.
3. Select the Lineart option.
4. Switch the model to “control_v11p_sd15s2_lineart_anime”. This selects the anime lineart model as the reference image.
5. Click the feature extraction button “💥”.
6. Select “My Prompt is more important” to avoid image distortion caused by anime lineart enlarging the eyes. By giving more importance to the prompt, we prevent such distortion.

Before generating the image, switch to a realistic model and generate the image. For this, use the Beautiful Realistic Asians v5 model.

The illustration has now been successfully converted into a live image.

Generating Live Images from Lineart

To generate live images from lineart, follow these steps:

1. Set the image as a shopping cart.
2. Enable the transformation.
3. Select the Lineart option.
4. Switch the Preprocessor to “lineart_anime_denoise”. This selects the anime lineart Preprocessor as the reference image.
5. Switch the Model to “control_v11p_sd15s2_lineart_anime”. This selects the anime lineart model as the reference image.
6. Click the feature extraction button “💥”.

Before generating the image, switch to a realistic model and generate the image. For this, use the Beautiful Realistic Asians v5 model.

The lineart has now been successfully converted into a live image.

We have tested this process with different images, and the lineart consistently produces high-quality images.

Coloring Lineart

1. Upload the lineart image.
2. Enable the “Enable” option.
3. Choose the Lineart category.
4. Select the “control_v11p_sd15s2_lineart_anime” model, as it is specifically designed for anime lineart.
5. Click on the feature extraction button “💥” to extract the features.

Next, switch to an anime-style model. In this case, switch to the AnythingV5Ink_ink.safetensors model.

Enter “1boy, jacket, suits, board” in the prompt to generate an image that matches the lineart.

Introduction to Lineart Preprocessors

The Lineart tool offers a range of preprocessors that can be utilized to enhance your artwork:

• Lineart:
• The “lineart_coarse” preprocessor generates bold and rugged lines for a distinct artistic effect.
• The “lineart_realistic” preprocessor produces delicate and thin lines for a more realistic appearance.
• The “lineart_standard” preprocessor provides an intermediate option between the coarse and realistic styles.
• Anime Lineart:
• The “lineart_anime” preprocessor creates slightly thicker lines, ideal for anime-style illustrations.
• The “lineart_anime_denoise” preprocessor is designed to remove any noise from the lineart_anime output, resulting in cleaner lines.