Code Monkey home page Code Monkey logo

dl-nn's Introduction

Image Generation Using Text Input

The goal of this project is to train a neural network to generate face images using text input to the model.

This repository is an implementation of the project found at Fast_text2StyleGAN, aimed at reproducing their results in image generation using StyleGAN.

Summary

We use CLIP(Contrastive Language-Image Pre-Training) model as a base and build on top of it. As we know, CLIP, maps text and image data to the same latent space. We use this leverage to train a neural network to decode any point in the latent space as an output image.

In other words, we use these steps:

1. Encode a face image using CLIP model.
2. Train a neural network to decode the encoded vector in the latent space back to the image domain.

We know that CLIP model maps the image and the related text to the same point in the latent space. Hence, we can now input text into the model and recieve the same latent vector for it. Using the decoder network, we can now decode the encoded text (corresponding to the actual image) and get the desired image.

Results

Input Text Image 1 Image 2 Image 3
a photo of a cute child baby image 1 baby image 2 baby image 3
a photo of a happy woman with long hair baby image 1 baby image 2 baby image 3
a photo of an old man with white hair baby image 1 baby image 2 baby image 3

Files Overview

  • dataset.py: Handles dataset processing and preparation.
  • loss.py: Contains the implementation of loss functions used in the model.
  • main.py: The main script for running the training and evaluation processes.
  • model.py: Defines the neural network model architecture.
  • network.py: Includes network configurations and settings.

License

The source code for the site is licensed under the MIT license, which you can find in the LICENSE file.

dl-nn's People

Contributors

armingh2000 avatar

Watchers

Kostas Georgiou avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.