935ae153e1
Cleanup.
6 months ago
38c22e631a
Fix case where model was not properly unloaded in merging workflows.
6 months ago
5d8bbb7281
Cleanup.
6 months ago
2c1d2375d6
Fix.
6 months ago
64ccb3c7e3
Rework IPEX check for future inclusion of XPU into Pytorch upstream and do a bit more optimization of ipex.optimize(). ( #4562 )
6 months ago
7c6bb84016
Code cleanups.
6 months ago
c54d3ed5e6
Fix issue with models staying loaded in memory.
6 months ago
7b70b266d8
Generalize MacOS version check for force-upcast-attention ( #4548 )
...
This code automatically forces upcasting attention for MacOS versions 14.5 and 14.6. My computer returns the string "14.6.1" for `platform.mac_ver()[0]`, so this generalizes the comparison to catch more versions.
I am running MacOS Sonoma 14.6.1 (latest version) and was seeing black image generation on previously functional workflows after recent software updates. This PR solved the issue for me.
See comfyanonymous/ComfyUI#3521
6 months ago
843a7ff70c
fp16 is actually faster than fp32 on a GTX 1080.
6 months ago
a60620dcea
Fix slow performance on 10 series Nvidia GPUs.
6 months ago
03ec517afb
Remove useless line, adjust windows default reserved vram.
6 months ago
9953f22fce
Add --fast argument to enable experimental optimizations.
...
Optimizations that might break things/lower quality will be put behind
this flag first and might be enabled by default in the future.
Currently the only optimization is float8_e4m3fn matrix multiplication on
4000/ADA series Nvidia cards or later. If you have one of these cards you
will see a speed boost when using fp8_e4m3fn flux for example.
6 months ago
1b3eee672c
Fix potential issue with multi devices.
6 months ago
045377ea89
Add a --reserve-vram argument if you don't want comfy to use all of it.
...
--reserve-vram 1.0 for example will make ComfyUI try to keep 1GB vram free.
This can also be useful if workflows are failing because of OOM errors but
in that case please report it if --reserve-vram improves your situation.
6 months ago
be0726c1ed
Remove duplication.
6 months ago
39fb74c5bd
Fix bug when model cannot be partially unloaded.
6 months ago
74e124f4d7
Fix some issues with TE being in lowvram mode.
6 months ago
b8ffb2937f
Memory tweaks.
6 months ago
ad76574cb8
Fix some potential issues with the previous commits.
6 months ago
5c69cde037
Load TE model straight to vram if certain conditions are met.
6 months ago
1de69fe4d5
Fix some issues with inference slowing down.
7 months ago
55ad9d5f8c
Fix regression.
7 months ago
037c38eb0f
Try to improve inference speed on some machines.
7 months ago
66d4233210
Fix.
7 months ago
08f92d55e9
Partial model shift support.
7 months ago
6969fc9ba4
Make supported_dtypes a priority list.
7 months ago
b334605a66
Fix OOMs happening in some cases.
...
A cloned model patcher sometimes reported a model was loaded on a device
when it wasn't.
7 months ago
c14ac98fed
Unload models and load them back in lowvram mode no free vram.
7 months ago
8edbcf5209
Improve performance on some lowend GPUs.
7 months ago
f7a5107784
Fix crash.
7 months ago
91be9c2867
Tweak lowvram memory formula.
7 months ago
03c5018c98
Lower lowvram memory to 1/3 of free memory.
7 months ago
2ba5cc8b86
Fix some issues.
7 months ago
1e68002b87
Cap lowvram to half of free memory.
7 months ago
ba9095e5bd
Automatically use fp8 for diffusion model weights if:
...
Checkpoint contains weights in fp8.
There isn't enough memory to load the diffusion model in GPU vram.
7 months ago
d965474aaa
Make ComfyUI split batches a higher priority than weight offload.
7 months ago
a6decf1e62
Fix bfloat16 potentially not being enabled on mps.
7 months ago
1aa9cf3292
Make lowvram more aggressive on low memory machines.
7 months ago
5f98de7697
Load flux t5 in fp8 if weights are in fp8.
7 months ago
7ad574bffd
Mac supports bf16 just make sure you are using the latest pytorch.
7 months ago
e2382b6adb
Make lowvram less aggressive when there are large amounts of free memory.
7 months ago
6425252c4f
Use fp16 as the default vae dtype for the audio VAE.
8 months ago
0ec513d877
Add a --force-channels-last to inference models in channel last mode.
8 months ago
5eb98f0092
Exempt IPEX from non_blocking previews fixing segmentation faults. ( #3708 )
8 months ago
0e49211a11
Load the SD3 T5xxl model in the same dtype stored in the checkpoint.
9 months ago
104fcea0c8
Add function to get the list of currently loaded models.
9 months ago
b1fd26fe9e
pytorch xpu should be flash or mem efficient attention?
9 months ago
b249862080
Add an annoying print to a function I want to remove.
9 months ago
bf3e334d46
Disable non_blocking when --deterministic or directml.
9 months ago
0920e0e5fe
Remove some unused imports.
9 months ago
comfyanonymous
Simon Lui
David