Code Monkey home page Code Monkey logo

lora_module's Introduction

一个简单的Lora轮子

这是一个简单的Lora轮子,目标是使用最少的代码,实现一个可以进行训练,持久化的微型Lora训练框架 注意:结果并不兼容webUI以及其他框架

环境说明

代码的目标是尽可能的关注Lora的实现,以最少的代码实现一个可用的Lora,所以整个代码是以huggingfacediffusers库进行开发的,所需要的环境依赖见 requirements.txt

不支持SDXL

运行

安装运行环境

pip install -r requirements.txt

训练以及推理的代码主要在trainLora.py中

python trainLora.py

数据放置到自定义路径,数据使用文本图像对的形式(sd-scripts的数据可以直接使用)

# 设置哪些module参与Lora注入
target_module = ["to_q", "to_k", "to_v", "conv1", "conv2", "proj_in", "proj_out", "conv", "conv_out", "proj", "ff.net.2"] # 选择注入的模块

# 选择基础模型,目前支持diffusers模型与ldm模型(webui中的checkpoint)
# model_name = "runwayml/stable-diffusion-v1-5"
model_name = "F:/local_model/dreamshaper_331BakedVae.safetensors"

# 实例化一个训练器
trainer = train_lora(model_name,     # 模型名称
                   True,             # 是否进行Lora输入
                   multiplier = 0.8,    # Lora的贡献比例,训练时默认为1
                   target_module=target_module,   # 如果注入,需要修改的module
                   is_diffusers=False,       # 是否是diffusers模型
                   only_local_files=False)   # 是否只使用diffusers模型

trainer.train(40, "./data/your_data")    # 训练
trainer.loraIn.save_lora("./save/lora.pt")  # 持久化

generate_image(4, trainer, "prompt")      # 生成图像

效果

运行效果基于 runwayml/stable-diffusion-v1-5 与 非diffusers模型 。 代码重点是为了用最少的编码实现可用的训练,用于明晰原理,不过分追求效果

diffusers prompt 角色/类型 原模型 Lora
1girl, circle_cut, clothing_cutout, cover, letterboxed, medium_breasts, pillarboxed KOSMOS
small_scene, 3D_scene, grass, tree_stump, picnic, faux_figurine scene
small_scene, 3D_scene, sunset, mountain, cloud, gradient_sky, twilight, orange_sky, evening, red_sky scene

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.