Stable Diffusion - Dreambooth Training Finetuning Setup Part 1/2

I wanted to train a stable diffusion network that can generate images of the PFM-1 anti-personnel mine. This mine is of Russian design and injures mostly civilians in many countries of the world. Since the mine itself is difficult to detect, I had the idea of using stable diffusion to generate images that could be used to test a neural network specialized in detecting this mine. Since stable diffusion will certainly not reproduce the exact geometry of the mine, such images could still be used for testing false positive detections.

So I set up Dreambooth under Automatic1111 and trained my first own mesh on this mine type. You can see such an image that I was able to create with Stable Deffusion in the following picture.

PFM-1 mine stable diffusion generated

The result looked quite good with the small number of images I had from the Internet. The next section is about setting up Dreambooth under Automatic1111.

Table of Contents

Dreambooth installation

Basically, the installation of Dreambooth under Automatic1111 is done very quickly. First of all, start Automatic1111 and open the web interface. Then switch to the “Extensions” tab at the top right of the menu.

Automatic1111 Stable Diffusion – installation finished

After you have clicked on the tab “Extensions” you will see the following menu. Here you click on the tab “Available” to get an overview which extensions are available.

Automatic1111 Stable Diffusion – Dreambooth installation tab extensions

The menu you see now shows a big button with the name “Load from”. Press this button to load or refresh the list with all extensions once. After that you should get a list like the following.

Automatic1111 Stable Diffusion – Dreambooth installation available extensions

In this list, simply look for the entry “Dreambooth” and click the “Install” button to the right to install this extension.

Automatic1111 Stable Diffusion – Dreambooth installation install extensions

The installation is now fully automatic and you do not have to do anything except wait until it is complete. The progress of the installation is a bit hard to see but it is displayed on the top right of the list.

Automatic1111 Stable Diffusion – Dreambooth installation install progress

After the installation is finished, restart Automatic1111. Now you will see a Dreambooth tab on the web interface of Automatic1111. Click on this tab and then the menu should look something like the following screenshot.

Automatic1111 Stable Diffusion – Dreambooth tab

This interface is now used to configure the training and, of course, the training itself.

Xformers extension missing

If you get the message that Xformers is not installed when you start Automatic1111, then hopefully these instructions on the Dreamtbooth page will help you. I followed these instructions and was able to install Xformers without any warning or error message. Now I start Automatic1111 the message does not appear anymore. However, on the web interface at the very end at the bottom the message appears that Xformers is not available. Give me feedback if you could solve the problem then I will add your solution here.

URL: Building xformers on Linux (from anonymous user)

Summary

It is really very impressive how easy everything is to handle. I would not have thought that I could get started so quickly as a user with a neural network that was customized for me. Of course you need the necessary hardware to make the whole thing fun and you don’t have to wait days for the result. But with an appropriate NVIDIA graphics card from the family of the 30XX or newer the training is really fast. Of course, you have to gain experience how to configure the training, but after a few attempts it worked quite well for me.

Click here for part 2/2