Another FAQ item for you.

What's the use of ngrok in the local deployment steps? If it's running locally why do we need to setup ngrok? Or is it solely for sharing the local setup with others with the url?

If it's only for sharing with others, I feel like local deploy and ngrok/bore setup/running could be separate optional steps.

uvicorn apis.interface:app --host 0.0.0.0 --port 8123
clip.large: 1.71GB [02:30, 11.4MB/s]                                            
clip_3.9: 2.11MB [00:01, 1.06MB/s]                                              
kl.f8: 405MB [00:36, 11.2MB/s]                                                  
ldm.sd: 4.27GB [06:12, 11.5MB/s]                                                
XXX lineno: -1, opcode: 0
Traceback (most recent call last):
  File "/home/danielrh/dev/carefree-creator/cfenv/lib/python3.10/site-packages/starlette/routing.py", line 645, in lifespan
    async with self.lifespan_context(app):
  File "/home/danielrh/dev/carefree-creator/cfenv/lib/python3.10/site-packages/starlette/routing.py", line 540, in __aenter__
    await self._router.startup()
  File "/home/danielrh/dev/carefree-creator/cfenv/lib/python3.10/site-packages/starlette/routing.py", line 622, in startup
    await handler()
  File "/home/danielrh/dev/carefree-creator/./apis/interface.py", line 186, in startup
    v.initialize()
  File "/home/danielrh/dev/carefree-creator/./cfcreator/txt2img.py", line 42, in initialize
    self.ms = init_sd_ms()
  File "/home/danielrh/dev/carefree-creator/./cfcreator/common.py", line 213, in init_sd_ms
    "": get_sd(),
  File "/home/danielrh/dev/carefree-creator/./cfcreator/common.py", line 42, in get_sd
    return _get("sd", DiffusionAPI.from_sd)
  File "/home/danielrh/dev/carefree-creator/./cfcreator/common.py", line 36, in _get
    m = init("cuda:0", use_half=True)
  File "/home/danielrh/dev/carefree-creator/cfenv/lib/python3.10/site-packages/cflearn/api/cv/models/diffusion.py", line 738, in from_sd
    return cls.from_pipeline(ldm_sd(), device, use_amp=use_amp, use_half=use_half)
  File "/home/danielrh/dev/carefree-creator/cfenv/lib/python3.10/site-packages/cflearn/api/cv/models/common.py", line 194, in from_pipeline
    return cls(m.model.core, m.model.device, use_amp=use_amp, use_half=use_half)
  File "/home/danielrh/dev/carefree-creator/cfenv/lib/python3.10/site-packages/cflearn/api/cv/models/diffusion.py", line 114, in __init__
    uncond = self.get_cond(unconditional_cond)
  File "/home/danielrh/dev/carefree-creator/cfenv/lib/python3.10/site-packages/cflearn/api/cv/models/diffusion.py", line 174, in get_cond
    return self.cond_model(cond)
  File "/home/danielrh/dev/carefree-creator/cfenv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/danielrh/dev/carefree-creator/cfenv/lib/python3.10/site-packages/cflearn/models/cv/diffusion/cond_models/clip.py", line 37, in forward
    text = self.get_text_indices(cond)
  File "/home/danielrh/dev/carefree-creator/cfenv/lib/python3.10/site-packages/cflearn/models/cv/diffusion/cond_models/clip.py", line 31, in get_text_indices
    text_arrays = [self.tokenizer.tokenize(t) for t in texts]
  File "/home/danielrh/dev/carefree-creator/cfenv/lib/python3.10/site-packages/cflearn/models/cv/diffusion/cond_models/clip.py", line 31, in <listcomp>
    text_arrays = [self.tokenizer.tokenize(t) for t in texts]
  File "D:\data\nolibox\data\CLIP-main\clip\simple_tokenizer.py", line 133, in tokenize
  File "D:\data\nolibox\data\CLIP-main\clip\simple_tokenizer.py", line -1, in <listcomp>
  File "D:\data\nolibox\data\CLIP-main\clip\simple_tokenizer.py", line -1, in encode
SystemError: unknown opcode

Application startup failed. Exiting.

I was using python-3.10; is that compatible?

then later I tried the same command

uvicorn apis.interface:app --host 0.0.0.0 --port 8123
XXX lineno: -1, opcode: 47
Segmentation fault (core dumped)

While building with docker i got whis problem:

315.7 /bin/sh: 1: COPY: not found
ERROR: process "/bin/sh -c rm -rf /opt/conda/lib/python3.10/site-packages/cv2 && pip install -U pip && apt-get update && apt-get install -y gcc git tzdata && pip install --upgrade Pillow && apt-get clean && rm -rf /tmp/* /var/cache/* /usr/share/doc/* /usr/share/man/* /var/lib/apt/lists/* && COPY . ." did not complete successfully: exit code: 127

[3/4] RUN rm -rf /opt/conda/lib/python3.10/site-packages/cv2 && pip install -U pip && apt-get update && apt-get install -y gcc git tzdata && pip install --upgrade Pillow && apt-get clean && rm -rf /tmp/* /var/cache/* /usr/share/doc/* /usr/share/man/* /var/lib/apt/lists/* && COPY . .

Didn't find any information, why this error occures?

Tried several times, always stops at this point

sha256:f272544375542d47ba20ed6cb90100eb6ea0a2ab23d36b19ddf3d31a17d55ed2 0B / 1.14kB 812.3s
#4 ERROR: failed to copy: httpReadSeeker: failed open: failed to authorize: rpc error: code = Unknown desc = failed to fetch anonymous token: unexpected status: 401 Unauthorized

[1/5] FROM nvcr.io/nvidia/pytorch:22.09-py3@sha256:ad07f7144606cb749dceb1ce7ed2286eeb69a63327ea7eccc69f0ac8ac1e0c68:

failed to copy: httpReadSeeker: failed open: failed to authorize: rpc error: code = Unknown desc = failed to fetch anonymous token: unexpected status: 401 Unauthorized

Last day I launch my local server. when it load an init the model, it failed! It looks like it need to use the new file about prompt enhance API, the console show that it is a 'pytorch_model.bin' file with 334M and need to download it. But as you know, it very slow to do that even connectError. So I try to download it on the 'huggingface', there is some updated file with 334Mb seem right.
But when I download it from the page, I got a 318M one like the local old one , I certainly think it is the old one after i use it.
I want to use the new one, how can I get it?

I've generated a bunch of images, but I've got a problem, idk how to delete them.

There's no delete option, I've tried everything.
Maybe it's because I'm using phone ¯⁠\⁠_⁠(⁠ツ⁠)⁠_⁠/⁠¯

Does your project run on Apple Silicon hardware? I wasn't sure if the hardware requirements rule out particular configurations.

I looked at your installation instructions and here in issues but didn't see mention of AS / M1 etc.

CarefreePlanet

https://discord.gg/UkfpFFmNd2

不好意思打擾版主，我在本地端部屬以及在中文版Colab 進行部屬時，雖然都可以跑到 Server is Ready! ，不過透過 localhost:8123或是Paste the following url to the WebUI: xxxxxxx網址進去後，都只出現{"detail":"Not Found"}，請問我是在哪一個步驟上出錯了嗎?謝謝。

Hi!
I'd like to ask - does "pip install -e ." install torch for cuda (or torch in general) as well?
I'm getting this after trying to use "uvicorn apis.interface:app --host 0.0.0.0 --port 8123":

File "C:\Users[username]\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users[username]\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Users[username]\AppData\Local\Programs\Python\Python310\Scripts\uvicorn.exe_main.py", line 7, in
File "C:\Users[username]\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1130, in call
return self.main(*args, **kwargs)
File "C:\Users[username]\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1055, in main
rv = self.invoke(ctx)
File "C:\Users[username]\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users[username]\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 760, in invoke
return _callback(*args, **kwargs)
File "C:\Users[username]\AppData\Local\Programs\Python\Python310\lib\site-packages\uvicorn\main.py", line 404, in main
run(
File "C:\Users[username]\AppData\Local\Programs\Python\Python310\lib\site-packages\uvicorn\main.py", line 569, in run
server.run()
File "C:\Users[username]\AppData\Local\Programs\Python\Python310\lib\site-packages\uvicorn\server.py", line 60, in run
return asyncio.run(self.serve(sockets=sockets))
File "C:\Users[username]\AppData\Local\Programs\Python\Python310\lib\asyncio\runners.py", line 44, in run
return loop.run_until_complete(main)
File "C:\Users[username]\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 646, in run_until_complete
return future.result()
File "C:\Users[username]\AppData\Local\Programs\Python\Python310\lib\site-packages\uvicorn\server.py", line 67, in serve
config.load()
File "C:\Users[username]\AppData\Local\Programs\Python\Python310\lib\site-packages\uvicorn\config.py", line 474, in load
self.loaded_app = import_from_string(self.app)
File "C:\Users[username]\AppData\Local\Programs\Python\Python310\lib\site-packages\uvicorn\importer.py", line 24, in import_from_string
raise exc from None
File "C:\Users[username]\AppData\Local\Programs\Python\Python310\lib\site-packages\uvicorn\importer.py", line 21, in import_from_string
module = importlib.import_module(module_str)
File "C:\Users[username]\AppData\Local\Programs\Python\Python310\lib\importlib_init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1050, in _gcd_import
File "", line 1027, in _find_and_load
File "", line 1006, in _find_and_load_unlocked
File "", line 688, in _load_unlocked
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "O:\AI\carefree-creator\carefree-creator.\apis\interface.py", line 3, in
import torch
ModuleNotFoundError: No module named 'torch'

It's true, that in this directory I don't have torch for cuda, or any other torch, but I thought pip would install that. :)

Greetings!

with the prompt of 2b, neir it triggers the audit algorithm roughly 2/3 times is there any way to bypass this for the locally run server

I saw the following paragraph in the README:

How can I get my own models interactable on the WebUI?

As long as we open sourced the WebUI you can implement your own UIs, but for now you can contribute to this carefree-creator repo and then ask me to do the UI jobs for you (yes, you can be my boss 😆).

Would it be possible to implement a generic way to import 1 or more custom models server-side, then communicate that to the webUI and allow the user to choose the model to use when using image generation/manipulation features? Much how AUTOMATIC1111/stable-diffusion-webui allows placing any *.ckpt in a models/ directory.

^

Love this project btw

I think it would be helpful in your frequently asked questions to shed some light on how this is different from other GUI based stable diffusion repos.

i've looked at this one a dozen times and I'm having a very difficult time understanding exactly what distinguishes it from something like the extremely popular automatic1111 repo.

docker 编译并运行也是报错

It's a beautiful interface and would enable developers to further customize the platform. Please share with the world

I want text to video for school project :/ I can’t edit for crap 🥲

whenever attempting to use any local models i get a crash with the error below

2022-11-29 08:55:51,858 - 6812 - root - ERROR - 'cond_stage_model.transformer.text_model.embeddings.token_embedding.weight' Traceback (most recent call last): File "C:\Users\Garry\carefree-creator\.\apis\interface.py", line 247, in switch_checkpoint cflearn.scripts.sd.convert(model_path, api, load=True) File "C:\Users\Garry\anaconda3\envs\stablediffusion\lib\site-packages\cflearn\scripts\sd.py", line 260, in convert nd = _convert_with_key_mapping(inp, md) File "C:\Users\Garry\anaconda3\envs\stablediffusion\lib\site-packages\cflearn\scripts\sd.py", line 231, in _convert_with_key_mapping nd = _convert_cond_stage(d, md) File "C:\Users\Garry\anaconda3\envs\stablediffusion\lib\site-packages\cflearn\scripts\sd.py", line 23, in _convert_cond_stage "core.condition_model.m.token_embedding.weight": d[ KeyError: 'cond_stage_model.transformer.text_model.embeddings.token_embedding.weight'

Hello! I used to be able to launch it on Colab and then send API requests to the endpoint "/txt2img/sd"

https://example.ngrok.io/txt2img/sd

But now the documentation is gone in "/redoc" and I cant send API requests to the endpoint at all.

I'm sorry if I've missed anything, thank you!

I have an 8 Gig GPU... I suspect it's not enough because I run into

uvicorn apis.interface:app --host 0.0.0.0 --port 8123
ldm_sd_v1.5: 4.27GB [06:19, 11.2MB/s]                                           
ldm_sd_anime_nai: 4.27GB [06:19, 11.3MB/s]                                      
ldm.sd_inpainting: 4.27GB [06:17, 11.3MB/s]                                     
Traceback (most recent call last):
...
  File "/home/danielrh/dev/carefree-creator/cfe9/lib/python3.9/site-packages/torch/nn/modules/module.py", line 662, in _apply
    param_applied = fn(param)
  File "/home/danielrh/dev/carefree-creator/cfe9/lib/python3.9/site-packages/torch/nn/modules/module.py", line 985, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 30.00 MiB (GPU 0; 7.92 GiB total capacity; 7.36 GiB already allocated; 67.56 MiB free; 7.43 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Application startup failed. Exiting.

I was able to run stable diffusion vanilla--does this require additional GPU ram?

Hi, after installing carefree-creator in docker, it reports an error when running, how to solve it?

WARNING: CUDA Minor Version Compatibility mode ENABLED.
Using driver version 510.108.03 which has support for CUDA 11.6. This container
was built with CUDA 11.8 and will be run in Minor Version Compatibility mode.
CUDA Forward Compatibility is preferred over Minor Version Compatibility for use
with this container but was unavailable:
[[Forward compatibility was attempted on non supported HW (CUDA_ERROR_COMPAT_NOT_SUPPORTED_ON_DEVICE) cuInit()=804]]
See https://docs.nvidia.com/deploy/cuda-compatibility/ for details.

NOTE: The SHMEM allocation limit is set to the default of 64MB. This may be
insufficient for PyTorch. NVIDIA recommends the use of the following flags:
docker run --gpus all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 ...

Traceback (most recent call last):
File "/opt/conda/bin/cfcreator", line 5, in
from cfcreator.cli import main
File "/opt/conda/lib/python3.8/site-packages/cfcreator/init.py", line 2, in
from .common import *
File "/opt/conda/lib/python3.8/site-packages/cfcreator/common.py", line 22, in
from cflearn.zoo import DLZoo
File "/opt/conda/lib/python3.8/site-packages/cflearn/init.py", line 3, in
from .schema import *
File "/opt/conda/lib/python3.8/site-packages/cflearn/schema.py", line 27, in
from accelerate import Accelerator
File "/opt/conda/lib/python3.8/site-packages/accelerate/init.py", line 3, in
from .accelerator import Accelerator
File "/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py", line 34, in
from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state
File "/opt/conda/lib/python3.8/site-packages/accelerate/checkpointing.py", line 24, in
from .utils import (
File "/opt/conda/lib/python3.8/site-packages/accelerate/utils/init.py", line 112, in
from .launch import (
File "/opt/conda/lib/python3.8/site-packages/accelerate/utils/launch.py", line 27, in
from ..utils.other import merge_dicts
File "/opt/conda/lib/python3.8/site-packages/accelerate/utils/other.py", line 24, in
from .transformer_engine import convert_model
File "/opt/conda/lib/python3.8/site-packages/accelerate/utils/transformer_engine.py", line 21, in
import transformer_engine.pytorch as te
File "/opt/conda/lib/python3.8/site-packages/transformer_engine/init.py", line 7, in
from . import pytorch
File "/opt/conda/lib/python3.8/site-packages/transformer_engine/pytorch/init.py", line 6, in
from .module import LayerNormLinear
File "/opt/conda/lib/python3.8/site-packages/transformer_engine/pytorch/module.py", line 16, in
import transformer_engine_extensions as tex
ImportError: /opt/conda/lib/python3.8/site-packages/transformer_engine_extensions.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN3c106SymInt8toSymIntENS_13intrusive_ptrINS_14SymIntNodeImplENS_6detail34intrusive_target_default_null_typeIS2_EEEE

is it possible to disable the nsfw checker? with an argument like --no_nsfw_checker, or any other way? (for scientific reasons)

Heyy i can help you with your discord community while also tryna help you with the project itself does that sound worth looking into?

问题

carefree-creator 版本 0.2.1的fastapi的docs无法显示

原因

fastapi的model_name_map在register_hint注册hint路由时，路由函数内写的函数内部包装类_Model处于函数内部的class作用域，在匿名内部类处无法识别出具象类的名称.

    class _(IAlgorithm):
        model_class = _Model

修正

    class _(IAlgorithm):
        model_class = _Model
        model_class.__name__ = hint_model_class.__name__

https://ailab-huawei-cdn.nolibox.com/upload/images/3e66619f7e184e8d8aa91a02b069e73d.png

https://ailab-huawei-cdn.nolibox.com/upload/images/edb826435d1e47d29c50dbdb302157bb.png

I run the program through colab through the bore tunnel. The thing is that the local server cannot see the assigned port.

Following the instructions of the Google Colab, I generated an ngrok URL that looks like https://f24d-68-121-167-240.ngrok.io. Copy-pasting this in Firefox from the Google Colab helpfully formatted it as https://f24d-68-121-167-240.ngrok.io/ (note the trailing slash).

Putting this into the "Use Local Server" field and clicking "Generate Image" results in a POST request from the browser to https://f24d-68-121-167-240.ngrok.io//get_prompt from the browser (note the double forward slashes). The hosted server returns a 404 for this request and the WebUI displays errors:

I suggest that either the WebUI strip trailing slashes from configured Local Server URLs, or the server be able to handle multiple, continuous slashes in a request URL.

Hello!
I might be totally lost but I'm trying to run it on a google cloud server. It doesn't have any GPU, so I'm trying to run the project in CPU mode using cfcreator serve --cpu. When I do so I get the following error:

RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.

Application startup failed. Exiting.

Can someone please help me? Thanks a lot :)

Hello!

I'm glad you got the colab to work again with the sd.base, however, I noticed the results when using for example:

{
    "text": "anime girl, high quality",
    "is_anime": true
}

is worse than using the same prompt on https://creator.nolipix.com/guest but with the Anime model.

So I guess it's not the same model? How can I use the Anime model for the API? Maybe this is too much for the colab again and we will run out of RAM, or do you think it's possible? Thank you!

Hello! Been looking forward to trying this webui, but having some trouble getting it running.

I've tried both the Google collab solution and local deployment and seem to get a similar issue with both.

After the running all the setup in the google collab up until the part where the message "Server is ready!" I open the link to https://creator.nolibox.com/guest and the menus on both right and left sides just load infinitely. There are no options I can interact with there.

I tried the local deployment solution as well, but I seem to be stuck endlessly after idm_semantic finishes. After having tried multiple times, I seem to always be stuck there for more than an hour without anything happening.

I managed to solve previous issues using the information already provided, but I am at a loss if there is simply no error message.

This does allow a custom port, but it'd be nice to have a whole custom endpoint, that way people could host the server on Google Colab then use that as the backend for the webui.

Ideally this would also be used to host images, instead of on your poor server.

Working on a Colab to do just this: https://colab.research.google.com/drive/1JbmQhUloM14aK713ypV2JRdZmV-Fs9le?usp=sharing

(For some reason ngrok doesn't seem to be printing the output URL of the tunnel though)

ERROR: Cannot install carefree-learn because these package versions have conflicting dependencies.

The conflict is caused by:
salesforce-lavis 1.0.2 depends on decord
salesforce-lavis 1.0.0 depends on decord

The errors occured when I run pip install carefree-creator

I encountered the following error when running with Docker
ImportError: cannot import name 'TPair' from 'cflearn.api.cv.diffusion' (/usr/local/miniconda3/lib/python3.8/site-packages/cflearn/api/cv/diffusion.py)
I did not find the definition under the cflearn project

安装时候报错

File "/usr/local/lib/python3.10/dist-packages/cfcreator/common.py", line 200, in init_sd
m.prepare_control_defaults()
AttributeError: 'ControlledDiffusionAPI' object has no attribute 'prepare_control_defaults'

Application startup failed. Exiting.

试了很多版本都不行了

如标题，谢谢