Comments (14)
In my testing, this error can be avoided by using a custom MLModelConfiguration()
with the config.computeUnits
set to cpuAndGPU
. But this probably results in worse performance without the neural engine. Also, after generating a couple of images, if you then try to generate more than one at a time (using imageCount) the process crashes again. Will continue exploring how to resolve the issue.
from ml-stable-diffusion.
lmao, it indeed worked well with iPadOS 16.2 Beta 4(Guess that's when they said it should work on 16.2). I actually played around with it for a while, generated tons of images(Using .all option). It just breaks on the RC no matter how, which to me is a big surprise.
From my experiment(when it used to work). It actually has nothing to do with the RAM. On iPad M1, it uses at most 2GB of ram when generate image. Each image will take up to 2GB in temp, that's probably why when set generate count to more than 1, it took too much disk space. Before I clear the temp files, my iPadOS app grows to about 55GB in size after generating some images.
from ml-stable-diffusion.
Are you using an iPad with 16 or 8GB of RAM? M1 or M2? My iPad is 16GB, so basically, that is what I have found at this point (using stable-diffusion-2-base model):
- I can generate images on iPhone 14 Pro, with "Increased Memory Limit" and "Extended Virtual Space" with
MLComputeUnits.cpuAndGPU
, I see the memory usage of about 3.6GB.- I can generate images on iPad Pro M2 16GB (or MBP 14 M1 Max 64GB), with "Increased Memory Limit" and "Extended Virtual Space" with any configuration of MLComputeUnits
- I also tried to use iPhone 13 Mini, but it only has ~3GB of Memory for apps (with Increased Memory Limit), and 4GB in total, so it does not look 4GB RAM iPhones can finish this task on those models, unless you can find some smaller models.
iPhone is on iOS 16.1 RC, iPadOS is 16.2 RC
. cpuAndNeuralEngine
is tricky, it seems like it uses a lot of memory also outside of your application process (you can see how Other Processes memory is growing), so it gives you the idea that your app is using only 1.6GB when in theory Others Processes usage grows up to 6 additional GB.
This comment resolved it for me. I am no longer experiencing a memory termination crash on iPhone 14 Pro Max. The Performance Benchmark part of the README indicates that Apple tested iPad Pro with the .cpuAndNeuralEngine
configuration, which results in the memory termination when I attempt this on iPhone 14 Pro Max. When attempting to use .cpuAndGPU
on iPhone 14 Pro Max, as you indicated in your comment, I found success (also with "Increased Memory Limit" and "Extended Virtual Space"). Thank you for this!
from ml-stable-diffusion.
Terminate due to memory error: on iPad 16.2 RC. Built on MacBook Air M2 with 16GB or ram. Tested on iPad Pro 11 with M1. Had enabled increase memory limit in capacity. Using Apple's Compiled Stable Diffusion 2.0.
from ml-stable-diffusion.
I am experiencing this same issue when trying to generate an image on an iPhone 14 Pro Max, as well (iOS 16.2, RC at the time of this posting). I have built the app so it does not include the chunked models compiled with the app, and downloads the models upon user interaction. The prepared models download and load successfully, as I can successfully configure a StableDiffusionPipeline
, but trying to generate an image results in a memory limit and termination (this is also with the "Extended Virtual Addressing" capability enabled and "Increased Memory Limit" entitlement). This issue does not occur when the same app is built for macOS, but results in an inability to generate an image on iOS (and presumably iPadOS, per @legolasW's comment).
from ml-stable-diffusion.
In my testing @cosalexelle , .cpuAndGPU option could indeed last longer before crash, but in my case will not last 1 image. Weirdly, if set to .all, will crash almost at the beginning, but set to . cpuAndGPU will last 30 seconds before crash on the first run. On consecutive will crash instantly.
from ml-stable-diffusion.
Very strange given how Apple specify this should work on iOS/ iPad OS 16.2. I have also updated to the Xcode release candidate to build for 16.2, but crashes the same.
Is it my understanding that when using the Neural Engine (i.e .cpuAndNeuralEngine), the model can be swapped from RAM to disk, to limit memory usage?
from ml-stable-diffusion.
Are you using an iPad with 16 or 8GB of RAM? M1 or M2? My iPad is 16GB, so basically, that is what I have found at this point (using stable-diffusion-2-base model):
- I can generate images on iPhone 14 Pro, with "Increased Memory Limit" and "Extended Virtual Space" with
MLComputeUnits.cpuAndGPU
, I see the memory usage of about 3.6GB. - I can generate images on iPad Pro M2 16GB (or MBP 14 M1 Max 64GB), with "Increased Memory Limit" and "Extended Virtual Space" with any configuration of MLComputeUnits
- I also tried to use iPhone 13 Mini, but it only has ~3GB of Memory for apps (with Increased Memory Limit), and 4GB in total, so it does not look 4GB RAM iPhones can finish this task on those models, unless you can find some smaller models.
iPhone is on iOS 16.1 RC, iPadOS is 16.2 RC
. cpuAndNeuralEngine
is tricky, it seems like it uses a lot of memory also outside of your application process (you can see how Other Processes memory is growing), so it gives you the idea that your app is using only 1.6GB when in theory Others Processes usage grows up to 6 additional GB.
from ml-stable-diffusion.
Interestingly, when I moved all the resources to ODR. Both iPhone 12 Pro and iPad Pro works.
from ml-stable-diffusion.
Interestingly, when I moved all the resources to ODR. Both iPhone 12 Pro and iPad Pro works.
@legolasW Can you clarify what ODR is? This sounds intriguing, but I am unfamiliar with that term in the context of this. I'm assuming that's referencing Apple's On Demand Resources? I'll give that a try, but in my test, I am downloading the compiled/chunked models from a private repository, decompressing, and then loading the pipeline. This works fine on macOS, but on iOS, still results in a crash when generating an image.
from ml-stable-diffusion.
Curious about On Demand Resources, based on the Platform Sizes for On-Demand Resources you cannot have more than 2GB in use of resources. And tags cannot be larger than 512MB, where some files in the model are 800+MB.
How did you manage that?
from ml-stable-diffusion.
Curious about On Demand Resources, based on the Platform Sizes for On-Demand Resources you cannot have more than 2GB in use of resources. And tags cannot be larger than 512MB, where some files in the model are 800+MB.
How did you manage that?
I recommend that you look at Background Assets. It doesn’t have the same system-imposed file-size limits as On-Demand Resources. Note that with Background Assets, the system can delete the downloaded model assets at any time if the user’s device is low on storage unless you move them into your app’s own app group container when the system calls your app extension’s download(_:finishedWithFileURL:)
delegate method. (Alternatively, it might be fine for your use-case simply to redownload the assets once the user’s device exits the low-storage state.)
from ml-stable-diffusion.
I recommend that you look at Background Assets.
Yeah, I saw that too! I was under the impression that on-demand resources could be hosted by Apple (App Store), which is probably also not true. And Background Assets 100% you have to host on your own, which can be a big bill.
from ml-stable-diffusion.
Yeah, I saw that too! I was under the impression that on-demand resources could be hosted by Apple (App Store), which is probably also not true. And Background Assets 100% you have to host on your own, which can be a big bill.
On-Demand Resources uses Apple servers for hosting. With Background Assets, you need your own server host. This can be a server that you control or any other host that provides a URL for direct downloads. (You can supply arbitrary URL requests to BAURLDownload
.)
from ml-stable-diffusion.
Related Issues (20)
- How to modify or update the model weight in swift? HOT 2
- Crashes on getting `currentImages`
- When will training be implemented?
- Torch2coreml multiprocessing
- Won't work on GPU on macOS 14.4 Beta 1 due to CoreML bug HOT 3
- Chunk Size for Einsum and Einsumv2 attention vs Resolution HOT 2
- CGImage extension method planarRGBShapedArray seems to leak memory
- Roadmap for this branch
- LCM scheduler HOT 2
- webui.sh missing HOT 1
- Question about chunking UNET HOT 1
- Question about quantization in torch2coreml
- Custom Pipeline
- Input X contains infinity - Error when converting quantized SD-Turbo HOT 3
- Mixed Palettization of SD 1.5 LCM model HOT 4
- Plz supprot Stable Cascade!
- iOS App storage (documents and data) keep increasing HOT 4
- ValueError: Input X contains infinity or a value too large for dtype('float64').
- chunk unet does not work with coremltool>7.1 HOT 1
- MLFeatureDescription "time_ids" crash
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ml-stable-diffusion.