Skip to content

PresentaPulse leverages the power of LivePortrait and Real-ESRGAN to create ultra-realistic animated portraits. Developed by Tarek Tarabichi from 2TInteractive, this tool integrates advanced AI models to enhance image quality and create stunning animations.

License

Notifications You must be signed in to change notification settings

LebToki/PresentaPulse

Repository files navigation

PresentaPulse

PresentaPulse leverages the power of LivePortrait and Real-ESRGAN to create ultra-realistic animated portraits. Developed by Tarek Tarabichi from 2TInteractive, this tool integrates advanced AI models to enhance image quality and create stunning animations.

Features

  • Realistic Animations: Generate ultra-realistic animations from still images.
  • Image Enhancement: Use Real-ESRGAN to upscale and enhance image quality.
  • Interactive UI: User-friendly interface powered by Gradio.
  • Multiprocessing: Optimized for performance with multiprocessing support.

image

What to expect (Magic!)

original Original with Expression Applied LivePortrait Face Markers is magical

How to use:

  • Choose your image
  • Choose your Facial Expression (driving video)
  • Kick-Starts Multi-Threading Operation on Windows Operating System,
  • downscales the original Facial Expression (driving video),
  • Enhances frames using Real-Esrgan,
  • Enhances Generated Video,
  • Re-Assemble Video for Download

Installation

Prerequisites

  • Python 3.8 or higher
  • Pip
  • Virtualenv

Clone the Repository

git clone https://github.com/LebToki/PresentaPulse.git
cd PresentaPulse

Create and Activate Virtual Environment

python -m venv venv
source venv/bin/activate  # On Windows use `venv\Scripts\activate`

Install Dependencies

pip install -r requirements.txt

Usage

Run the Application

python app.py

Once the application is running, you can access the web interface using the local URL provided in the terminal.

Recommended Models

Worth mentioning:

  • You need to mind that I am hardcoding my directory path for now! So, you are recommended to change that path to work with your own system deployment path until I get to a mature version.
  • I suggest you clone Real-ESRGAN in the root folder of the project and build it within to avoid messing up with the codebase and compatibility.
  • Make sure your Torch has CUDA support.
  • Make sure your Onnxruntime has CUDA support.
  • Experiment with different expressions (Pexels has a few solid good ones).
  • Avoid facial expressions with tons of camera movements, pans, and tilts.
  • Have fun and experiment.
  • I would appreciate it if you follow me, star the repo, and check the other AI-related repos too :)

Pretrained Models and Configuration: create this directory and to download the Pretrained models

D:\tools\LivePortrait\pretrained_weights

you may need to run git lfs install first

git clone https://huggingface.co/KwaiVGI/liveportrait pretrained_weights

alternatively you may try to download the archive from Google Drive (https://drive.google.com/drive/folders/1UtKgzKjFAOmZkhNK-OYT0caJ_w2XAnib)

Go to Hugging Face and look for these additional models

download them to the pretrained_weights folder too

- realesr-animevideov3.pth
- RealESRGAN_x4plus.pth
- RealESRGAN_x4plus_anime_6B.pth
- RealESRNet_x4plus.pth

Directory Structure of /pretrained_weights/

realesr-animevideov3.pth
RealESRGAN_x4plus.pth
RealESRGAN_x4plus_anime_6B.pth
RealESRNet_x4plus.pth

├───insightface
│   └───models
│       └───buffalo_l
│               2d106det.onnx
│               det_10g.onnx
│               
└───liveportrait
    │   landmark.onnx
    │   
    ├───base_models
    │       appearance_feature_extractor.pth
    │       motion_extractor.pth
    │       spade_generator.pth
    │       warping_module.pth
    │       
    └───retargeting_models
            stitching_retargeting_module.pth

Acknowledgements

Let'us all give kudos to the original creator of LivePortrait and every other library we are using, including:

Contributing

I welcome contributions! Please fork the repository and create a pull request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contact

For more information, please visit 2TInteractive (https://2tinteractive.com) or contact Tarek Tarabichi.

TO DO

  • The Fun-part is still being worked on: Taking this whole thing into Real-ESRGAN and applying the highest quality possible using RealESRGAN_x4plus_anime_6B.pth or RealESRGAN_x4plus.pth.
  • The Wishlist is growing:
    • Possibly integrate multiface with object and zones detection and targeting.
    • Possibly working around the square generation limitation (1:1).
    • Possibly trying to integrate MimicMotion which will require updating the workflow to include the use of MimicMotion for generating motion sequences before or after the Real-ESRGAN enhancement. The only issue is its high demands on PC resources which could be an issue (for now at least).

About

PresentaPulse leverages the power of LivePortrait and Real-ESRGAN to create ultra-realistic animated portraits. Developed by Tarek Tarabichi from 2TInteractive, this tool integrates advanced AI models to enhance image quality and create stunning animations.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages