Recent Posts21
General Chat / Fine Tuning Stable Diffusion with Sauerbraten screenshots
« Last post by Salatiel on November 27, 2022, 01:49:02 AM »Hello!
Just another update on the "put Sauer into stuff" quest, a few months ago I posted an experiment manipulating some screenshots with an AI model: https://sauerworld.org/forum/index.php?topic=652.0
Now access to Dall-e 2 is public (albeit limited), and freer/open source alternatives like Stable Diffusion have emerged and have since surpassed it.
Right now you can "fine tune" Stable Diffusion for free using DreamBooth, a utility that helps you customize an already trained model, that is, you can add new elements (like your face or captain cannon's) to an existing model without having to train everything again. That way, instead of manipulating an existing image, you can create a completely new one
With about 50 screenshots featuring Captain Cannon these are the results I got so far:
Sorry to make you see a captain cannon face reveal out of the blue like that.
I'll update the post as soon as I have the results of the other playermodels and maps, but if you want to go ahead and try it out:
Fine Tuning Stable Diffusion with Dreambooth in Google Colab
1: Create a HuggingFace account, you will need it to get a token: https://huggingface.co/settings/tokens.
2: Open the Google Colab notebook (a fork): https://gist.github.com/SalatielSauer/957f240032a56dfc19edca4c7e11db18
3: Click on File and save a copy to your Google Drive:
4: Before running any cell, create a folder called "screenshots" next to the existing sample_data, that's where you'll put all your .jpg screenshots named img*.jpg, where * is a number in ascending order.
5: In the "Settings for teaching a new concept" cell, find and change the range of the for loop from 100-150 to the amount of screenshots you have uploaded, if your images start from img0.jpg and go up to img50.jpg, put 0 and 50 as the range.
Right in the next cell you can change the prompt that will be associated with your new element.
6: With all that done you can go back to the first cell and run them one by one, making sure to wait for the previous one to finish before moving on to the next.
The last three cells are where the magic happens, it may take a few minutes to finish executing them, but if everything goes well, you will be able to customize the prompt in the last cell and generate the image.
try playing around with the guidance_scale and num_inference_steps values too!
On this notebook I disabled the nsfw filter ('safety_checker') as it was giving some false positives, keep that in mind
The original Google Colab notebook:
https://github.com/PsorTheDoctor/artificial-intelligence/blob/master/modern_approach/text_to_image/dreambooth.ipynb
Search engine for prompts, also generates images: https://lexica.art/
Just another update on the "put Sauer into stuff" quest, a few months ago I posted an experiment manipulating some screenshots with an AI model: https://sauerworld.org/forum/index.php?topic=652.0
Now access to Dall-e 2 is public (albeit limited), and freer/open source alternatives like Stable Diffusion have emerged and have since surpassed it.
Right now you can "fine tune" Stable Diffusion for free using DreamBooth, a utility that helps you customize an already trained model, that is, you can add new elements (like your face or captain cannon's) to an existing model without having to train everything again. That way, instead of manipulating an existing image, you can create a completely new one
With about 50 screenshots featuring Captain Cannon these are the results I got so far:
Sorry to make you see a captain cannon face reveal out of the blue like that.
I'll update the post as soon as I have the results of the other playermodels and maps, but if you want to go ahead and try it out:
Fine Tuning Stable Diffusion with Dreambooth in Google Colab
1: Create a HuggingFace account, you will need it to get a token: https://huggingface.co/settings/tokens.
2: Open the Google Colab notebook (a fork): https://gist.github.com/SalatielSauer/957f240032a56dfc19edca4c7e11db18
3: Click on File and save a copy to your Google Drive:
4: Before running any cell, create a folder called "screenshots" next to the existing sample_data, that's where you'll put all your .jpg screenshots named img*.jpg, where * is a number in ascending order.
5: In the "Settings for teaching a new concept" cell, find and change the range of the for loop from 100-150 to the amount of screenshots you have uploaded, if your images start from img0.jpg and go up to img50.jpg, put 0 and 50 as the range.
Right in the next cell you can change the prompt that will be associated with your new element.
6: With all that done you can go back to the first cell and run them one by one, making sure to wait for the previous one to finish before moving on to the next.
The last three cells are where the magic happens, it may take a few minutes to finish executing them, but if everything goes well, you will be able to customize the prompt in the last cell and generate the image.
try playing around with the guidance_scale and num_inference_steps values too!
On this notebook I disabled the nsfw filter ('safety_checker') as it was giving some false positives, keep that in mind
The original Google Colab notebook:
https://github.com/PsorTheDoctor/artificial-intelligence/blob/master/modern_approach/text_to_image/dreambooth.ipynb
Search engine for prompts, also generates images: https://lexica.art/
