Automated art, illustrating Outpainting with DALL-E2

 kw: experiments, simulated intelligence, art, generated art, images, prompts, photo essays

A charity, prospecting for contributions, included some Christmas cards with their appeal. This one really appealed to me. I decided to scan it and upload it to Dall-E2 so I could extend it by Outpainting. In earlier posts on automated art I wrote of Expanding, but Outpainting is the main term used by OpenAI. The original painting is "Holiday Social", copyright Geno Peoples 2017. The artist's website shows this and many other paintings in this "luminous cottage" style and other similar styles (note that the website is not secure).

I tried scanning at various resolutions with descreening, both automated and manual. What worked best was a cell phone photo, manually descreened and reduced in pixel resolution. I suppose I could have downloaded it from the website, where an image 1100x717 is available. If you want one of his paintings big enough to print on canvas you can buy it there. The image above is 1024x763, and is a bit narrower than the original.

When you log into Dall-E2, you can either start with a prompt to generate an image or upload something you want to edit. I prepared this image, and a slightly larger rendition (1450x1080), to experiment with uploading and outpainting. Eventually I ran three outpainting sessions. Grammatical note: In a lengthy discussion like this, I tire of circuitous locutions using the pronoun "one", so I will instead use "you" language.

When you open a Dall-E2 session and upload an image, you have the option to crop it to a square or leave it uncropped. In all cases I skipped cropping. If you crop the image, it is squared up to a size of 1024x1024 pixels or smaller. Every Generate action is done within a Generation Frame of size 1024x1024.

First Session 

I first used the 1450x1080 version, which was loaded at full size when I clicked "Skip cropping". It is necessary to add a prompt before you can click Generate. I began by adding material at all four corners. The image below shows the Edit screen after the first two additions. The buttons at the bottom are, from left to right, Select (for moving the Generation Frame), Scroll (for moving the entire image, Frame and all), Erase, Add Frame (for initiating a Generation Frame), and Upload (which I haven't used. I suspect it lets you add an object to the image but I don't know). The Generate button is above this part of the screen, on the right side. There is a download button next to it, and I use it frequently, keeping in mind the admonition in the gray box at lower left (you may need to click on the image to see it large enough to read that text). Downloaded images are PNG files, about 3-4 times the size of JPG files, but with no compression losses. 

The prompt at this point reads "A holiday visit on a snowy day at sunset in the style of Geno Peoples"; I changed it later.

When you click Generate at any point, you have four choices to choose among. If none is acceptable, you can click Cancel and Generate again (consuming a credit...sigh). After outpainting in all four directions I had this result:

The two extensions in the sky added versions of the sun near sunset, which I didn't like. I combined more outpainting with editing. The next image shows the upper right corner during this process.

The white circle at center right is the eraser. I got rid of the bright bit of sky and an unusual looking window in the tall building. I also removed the sun image at left (not shown in this crop) during the following Generate event.

After getting the sky to my liking, I added material below the two extensions I had made, which yielded the image below, which is now 2560x1728 (the image here is smaller). I can crop a 16:9 portion for my wallpaper folder, any size from 1920x1080 to 2560x1440.

It's notable that the extension of the village to the right in particular adds buildings that are not quite the same style as the original ones. Producing this image consumed 8 credits.

Second Session

I started a new session, uploaded the 1024x763 version and skipped cropping. Here I am about to add material at upper right. You can see that the uploaded image is smaller than a Generation Frame. I was hoping to produce a larger village, relative to the initial painting.

I wonder if I should have outpainted in smaller increments, such as having the Frame begin at the base of the large house. I did some Cancels this time. Buildings being added were often in a clashing style. Midway through the process I appended the words "finely detailed" to the prompt. It made little difference. Here we see the result after four corners were extended:

This image needs some work. As the next image shows, I did a lot of erasing while further extending. I ignored the added Moon; I plan to crop it out of a 16:9 image later.

Erasing while outpainting produced an image I like a bit better, shown next, but I like it less than the final image from the first session. 

This image (here it is reduced) is 2304 x 1792. I can crop out anything from 1920x1080 to 2304x1296 if I decide to use it as wallpaper. Producing it consumed 10 credits. Not as much bang for the buck as usual.

Third Session

I decided to try again with the 1024x763 version. Before outpainting I entered a different prompt: An old-time village in winter just after sunset with meltwater on snowy roads, in luminous cottage style

This time I extended downward first, to fill a 1024x1024 square, then I extended to the right and the left, with one Cancel, to get this:

This is 2240x1024. The added material is a bit cruder than the original painting, so I won't outpaint further. Rather, to produce a wallpaper image I'll increase the resolution (I use IrfanView, but Photoshop and Gimp are also good) to 2362x1080. I may use Unsharp Masking to boost the apparent sharpness a little. Then I can crop out a 1920x1080 piece, probably more to the left to include less of the reddish buildings. In the prompt I didn't refer to Geno Peoples, but used a more generic style note. I am not sure to what extent it helped. This image consumed 4 credits. I like it better than the one from Session 2.

Dall-E2 is an enjoyable collaborator, very useful but sometimes recalcitrant. It keeps things interesting.