openOutpaint/README.md

140 lines
13 KiB
Markdown
Raw Normal View History

2022-11-19 04:12:11 +00:00
# hello there 🐠
2022-11-16 00:48:41 +00:00
![openOutpaint creating some undersea wildlife](docs/01-demo.gif)
2022-11-16 06:09:43 +00:00
this is a completely vanilla javascript and html canvas outpainting convenience doodad built for the API optionally exposed by [AUTOMATIC1111's stable diffusion webUI](https://github.com/AUTOMATIC1111/stable-diffusion-webui), operating similarly to a few others which certainly have superior functionality. this simply offers an alternative for my following vain desires:
2022-11-16 00:59:30 +00:00
- avoiding the overhead of an additional virtual python evnironment or impacting a pre-existing one
- operates against the API exposed by A1111's webUI
2022-11-16 00:48:41 +00:00
- no external dependencies, extremely boring vanilla
- no external connectivity, self-hosted and offline
- unobfuscated (cough cough)
- <a name="terrible"></a>i am terrible at javascript and should probably correct that
- i have never used html canvas for anything before and should try it out
2022-11-18 03:21:48 +00:00
## features
- a big ol' 2560x1440 canvas for you to paint all over _(infinite canvas area planned, in //todo already)_
- inpainting/touchup blob
- easily change samplers/steps/CFG/etc options for each dream summoned from the latent void
2022-11-18 03:21:48 +00:00
- optional grid snapping for precision
- optional overmasking for better seams between outpaints (suggested by @lifeh2o ([a](https://www.reddit.com/r/StableDiffusion/comments/ywf8np/i_made_a_completely_local_offline_opensource/iwl6s06/),[b](https://www.reddit.com/r/StableDiffusion/comments/ys9lhq/kollai_an_infinite_multiuser_canvas_running_on/ivzygwk/?context=3)) and i think it's a slick idea)
- optional hi-res fix for blank/txt2img dreams which, if enabled, uses image width/height / 2 as firstpass size
2022-11-19 00:23:58 +00:00
- import arbitrary images and superimpose on the canvas wherever you'd like ([extra fun with transparent .pngs!](#arbitrary_transparent))
2022-11-18 03:21:48 +00:00
- "temporary" monitors at the bottom to see exactly what mask/image you're feeding img2img, no i'm certainly not using them as actual imagedata sources or anything
- saves your preferences to browser localstorage for maximum convenience
2022-11-18 03:21:48 +00:00
2022-11-16 00:48:41 +00:00
## operation
### prerequisities
you'll obviously need A1111's webUI installed before you can use this, thus you're presumed to have an operational python install up and running to boot.
2022-11-16 00:48:41 +00:00
### notes
2022-11-17 02:47:21 +00:00
- technically you can run it directly in browser as a bare `file://` protocol webpage but that's _really_ not recommended as you'll have to add `null` as an accepted domain to your `--cors-allow-origins` option which just seems like it's a visibly poor decision
- a deliciously simple launch script (thanks [@jasonmhead](https://github.com/jasonmhead)! (https://github.com/zero01101/openOutpaint/pull/1)) is included to pop up a teensy tiny python-based local webserver, however you may have to manually `chmod +x openOutpaint.sh` on mac/linux
- the address http://127.0.0.1:3456 will be used as the host address for openOutpaint in the below quickstart; your local setup may use a different IP address or port. you can of course modify the included launch script to point at a different port than 3456 if desired, as well
- if your scale factor is > 8 (generating an image larger than 512x512), try the "auto txt2img HR fix" option
### quickstart speedrun
1. edit your `cors-allow-origins` to include https://zero01101.github.io and run webUI
2. go to https://zero01101.github.io/openOutpaint/ and fill in the host value with your webUI API address
3. click things and do stuff
### quickstart normal edition
1. clone this repo to your homelab's webserver (i mean who doesn't have a couple of those lying around?) or somewhere on your local pc
2. configure your local webhost in your homelab to serve the newly cloned repo like the technological bastion you are, or simply run the included `openOutpaint.bat` on windows or `openOutpaint.sh` on mac/linux.
3. modify your `webui-user.sh` or `webui-user.bat`'s `COMMANDLINE_ARGS` variable to contain ` --api --cors-allow-origins=http://127.0.0.1:3456`
2022-11-16 00:48:41 +00:00
4. execute your webui-user script and wait for it to be ready
5. **APPLY THE FOLLOWING SETTINGS IN A1111 WEBUI ONCE IT IS READY:**
- select an inpainting checkpoint/model - ([runwayml/stable-diffusion-inpainting](https://huggingface.co/runwayml/stable-diffusion-inpainting) [3e16efc8] is recommended)
- set your `Inpainting conditioning mask strength` to `1`
- disable the `Apply color correction to img2img results to match original colors.` option (the last 2 options are found under the stable diffusion category in the settings tab by default unless you've already moved it to your quicksettings list, and if so, you know where to set them already)
2022-11-16 23:19:29 +00:00
6. open your locally-hosted web server at http://127.0.0.1:3456 (or wherever, i'm not your boss)
7. update the host field if necessary to point at your stable diffusion API address, change my stupid prompts with whatever you want, click somewhere in the canvas, and wait _OR_ you can load an existing image from your computer using the "open img" button
2022-11-16 03:38:54 +00:00
8. once an image appears*, click the `<` and `>` buttons at the bottom-left corner of the image to cycle through the others in the batch if you requested multiple (it defaults to 2 batch size, 2 batch count) - click `y` to choose one you like, or `n` to cancel that image generation batch outright and possibly try again
2022-11-16 00:48:41 +00:00
9. now that you've got a starter, click somewhere near it to outpaint - try and include as much of the "context" as possible in the reticle for the best result convergence
2022-11-17 02:47:21 +00:00
10. enable the mask mode to prepare previously rendered imagery for touchups/inpainting, then paint over the objectionable region; once your masked region is drawn, disable mask mode and change your prompt if necessary, then click over the canvas containing the mask you just painted to request the refined image(s)
2022-11-17 02:37:09 +00:00
11. play around with the available options!
2022-11-19 00:23:58 +00:00
- snap to grid, uh, snaps to the grid
- mask mode is practically identical to A1111's inpainting mask, except that...
- scale factor affects the size of both the painting reticle __and__ mask blob
2022-11-19 00:27:32 +00:00
- overmask and related px value expands the area masked in img2img outpaint requests, so as to attempt to minimize the seam between images; it's _partially_ functional currently and correcting the flaws are already in the //todo and issue tracker
2022-11-17 02:37:09 +00:00
- ...everything else is pretty much just a regular stable diffusion option so i presume you know how you use those
12. click "dl canvas" to save the cropped region of outpainted canvas (thanks [@Kalekki](https://github.com/Kalekki)! (https://github.com/zero01101/openOutpaint/pull/2))
13. click "new image" to blank the canvas and start all over only to discover that it's like 2 AM and you have to go to sleep because you have work in about 4 hours
2022-11-16 00:48:41 +00:00
2022-11-16 03:38:54 +00:00
*if it _doesn't_ create an image, check your console output to see if you've got CORS errors
2022-11-16 00:48:41 +00:00
2022-11-16 04:50:59 +00:00
## //todo
### in order of "priority"/likelihood of me doing it
2022-11-17 02:20:04 +00:00
- [ ] lots and lots of readme updates (ongoing)
- [ ] comment basically everything that isn't self documenting (ongoing)
- [ ] CHORE: refactor all the duplicated JS code (ongoing, guaranteed to get worse before it gets better)
- [x] overmask seam of img2img - BUG: it kinda sucks currently, just moves the seam instead of fixing it, i want to try to gradient-fade the edge but filter = 'blur(Ypx)' is _awful_ for this and my remedial per-pixel loops crash the browser because i am the embodiment of inefficiency
2022-11-16 23:23:13 +00:00
- [x] split out CSS to its own file (remedial cleanup task)
- [x] ability to blank/new canvas without making the user refresh the page because that's pretty janky
2022-11-16 04:50:59 +00:00
- [ ] add error handling for async/XHR POST in case of, yknow, errors
2022-11-17 02:37:09 +00:00
- [ ] image erase region in case you decide later that you're not too happy with earlier results (technically i guess you could just mask over the entire region you dislike but that's... bad)
- [ ] controls for the rest of API-available options (e.g. ~~hires fix~~, inpaint fill modes, etc)
- [ ] ~~save user-set option values to browser localstorage to persist your preferred, uh, preferences~~ (thanks again [@Kalekki](https://github.com/Kalekki)! (https://github.com/zero01101/openOutpaint/pull/5))
2022-11-16 05:59:29 +00:00
- [ ] render progress spinner/bar
2022-11-16 23:21:03 +00:00
- [ ] ~~smart crop downloaded image~~
- [x] import external image and ~~scale/~~superimpose at will on canvas for in/outpainting
- [ ] scaling of imported arbitrary image before superimposition
- [ ] "numpad" selector for determining how reticle is anchored against actual mouse cursor (currently works like a "5" [center] on the "numpad" paradigm)
2022-11-16 03:26:37 +00:00
- [ ] discrete size control for mask and target reticle, discrete x/y axes for reticle
2022-11-16 03:15:52 +00:00
- [ ] floating/togglable menu leftnav bar with categorized/sensibly laid-out options
- [ ] infinite canvas
2022-11-16 00:48:41 +00:00
- [ ] global undo/redo
- [ ] inpainting sketch tools
2022-11-18 03:21:48 +00:00
- [ ] split out JS to separation-of-concerns individual files (oh no)
2022-11-16 23:25:04 +00:00
- [ ] something actually similar to a "user interface", preferably visually pleasant and would make my mom say "well that makes sense" if she looked at it
2022-11-16 00:48:41 +00:00
- [ ] eventually delete the generated mask display canvases at the bottom of the page, but they're useful for debugging canvas pixel offsets sometimes
2022-11-16 04:50:59 +00:00
- [ ] see if i can use fewer canvases overall; seems wasteful, canvas isn't free yknow
- [ ] upscaling output canvas??? sure let's make 16k wallpapers that'll be neat
2022-11-16 00:48:41 +00:00
- [ ] honestly probably refactor literally everything
## pull requests
**PLEASE SEND PULL REQUESTS**
2022-11-16 03:26:37 +00:00
i am begging you, yes you personally reading this, please fix my horrible code and feel free to insult it, but i absolutely refuse to budge on no 3rd party libraries or dependencies, not even jquery, nothing. vanilla is a very complex and layered flavor if you give it a chance.
2022-11-17 02:20:04 +00:00
## bug reports
please do! kindly indicate your OS, browser, versions of both, any errors in devtools/console output, what you were trying to do, what you expected, what happened unexpectedly or incorrectly, if something caught fire (please call the fire department first), the usual
## known bugs :(
- generated images display +1px on x/y during approve/reject state, doesn't affect output, just annoying
- erase mask is like entirely broken
- odd-numbered scale factors don't snap correctly
- arbitrary "pasted" images require clicking twice to place them and i _don't know why_ [(yes i do)](#terrible), just getting them to be arbitrarily placable was a giant pain because i'm not got the smarts
- selecting an aribtrary image by double-clicking it in the file picker can sometimes trigger a dream request that errors out if your file picker is "above" the canvas; i tried to alleviate that by temporarily removing the mouse(move/down/up) handlers for the canvas context on selection of a file, but i'm POSITIVE it's an improper solution and not quite sure if it's even fully effective
2022-11-17 02:20:04 +00:00
## warranty
[lmao](https://github.com/moyix/fauxpilot#support-and-warranty)
2022-11-19 00:23:58 +00:00
## samples
2022-11-17 02:20:04 +00:00
generated using 100% openOutpaint UI defaults except for switching to/from mask mode and changing scale factor to adjust the size of the mask blob, there's some neat stuff down there even if it disregarded the `people, humans, divers` negative prompt but in its defense there is only one singular person, human, diver in there, so according to the no homers club treatise of 1995 it's technically correct
_(see https://github.com/zero01101/openOutpaint/commit/92ab9d231542ea5f7a3c85563acf5cd3cb16a928 for attempted counterattack)_
2022-11-16 05:59:29 +00:00
![fishies n stuff](docs/02-sample.png)
2022-11-19 00:23:58 +00:00
<a name="arbitrary_transparent"></a>simple transparent PNGs slapped on the canvas twice and default UI settings used on the southern half
![imported images that have been _changed_](docs/03-arbimg.png)
2022-11-19 02:22:48 +00:00
imported a transparent clip of a [relatively famous happy lil kitty](https://commons.wikimedia.org/wiki/File:So_happy_smiling_cat.jpg), used default openOutpaint settings except changing the prompt to "a cat on a space station", eventually received this magnificent vision
![many cats on a space station](docs/04-catsonaspacestation.png)
## version history
- 0.0.1 - txt2img proof of concept
- 0.0.2 - img2img outpainting proof of concept
- 0.0.3 - image masking/img2img inpainting proof of concept
- 0.0.4 - batch size/batch count, approve/reject system implementations, snap-to-grid, other people are now allowed to see this thing [01f8c6a](https://github.com/zero01101/openOutpaint/commit/01f8c6ab3f49739439a0990d6f5f0967a9a0bf12)
- 0.0.4.1 - extremely minor revisions [02cb01a](https://github.com/zero01101/openOutpaint/commit/02cb01ac062ef93878ff4161eabcedfa8e125be6)
- 0.0.4.2 - pull requests (&lt;3), downloaded images now have a timestamped name, css breakout because hopefully this will become halfway attractive enough to benefit from non-inline stylesheets [70ad4fe](https://github.com/zero01101/openOutpaint/commit/70ad4fe081bdbd507afc5af3cc2a4435924b66e3)
- 0.0.4.3 - overmasking, settings saved to localstorage [fca2e01](https://github.com/zero01101/openOutpaint/commit/fca2e01b8a4ecfe3d062c4090d5886e1033e8f38)
2022-11-19 00:05:10 +00:00
- 0.0.5 - import arbitrary image from user's machine, "auto" txt2img hires fix, Very Important "new image" button [3b7f4e3](https://github.com/zero01101/openOutpaint/commit/3b7f4e3759d0d1f3b38eba7249e5b58bc8162c75)
2022-11-19 04:12:11 +00:00
## why the fish?
deep aquatic life is _fascinating_ so i went with something underwater for a default prompt which led to making a cute little orange fish favicon which led to "fish is mascot"
2022-11-19 04:15:30 +00:00
the end 🐠