Skip to content
@Q-Future

Visual Evaluation with Foundation Models

We are working towards a future that one foundation model can be a multi-purpose expert for low-level visual perception and visual evaluation.

👁️‍🗨️ Low-level Visual Perception in the Foundation Model Era

🔖Aiming at next-era cornerstone research

Low-level Visual Perception | Multi-Modality Large Language Models | Visual Quality Assessment

📖Main Projects

  • Co-Instruct: Homepage, Repo, Demo. Open-ended visual quality comparer (up to 4 images), low-level visual assistant, an improved version of ②Q-Instruct [CVPR 2024].

  • Q-Align [ICML 2024]: Homepage, Repo, Demo. A unified visual scorer for images and videos, via text-instructed alignment on multi-modality foundation models; can efficiently fine-tune to more datasets with stable good performance. State-of-the-art on IQA, VQA, and IAA.

  • Q-Instruct [CVPR 2024]: Homepage, Repo, 200K Dataset, Technical Report A large-scale instruction tuning dataset to improve low-level perceptual abilities of foundation models.

  • Q-Bench+ [ICLR2024, Spotlight]: Homepage, Repo, Data-Single, Data-Pair, Preprint The first low-level benchmark for foundation models on low-level vision.

🖋️Extension Projects

  • Q-Boost: Homepage A discussion on boosting the IQA performance for non-specially-IQA-aligned MLLMs.

  • [Pending]Chinese-Q-Bench/质衡: Homepage, Repo The first attempt to test multi-lingual abilities on low-level vision.

Maintained by Teo Wu@Singapore and Zicheng Zhang@Shanghai.

Pinned Loading

  1. A-Bench A-Bench Public

    [LMM + AIGC] What do we expect from LMMs as AIGI evaluators and how do they perform?

    125 3

  2. Co-Instruct Co-Instruct Public

    ④[ECCV 2024, Comparison among Multiple Images!] A study on open-ended multi-image quality comparison: a dataset, a model and a benchmark.

    60 4

  3. Q-Align Q-Align Public

    ③[ICML2024] [IQA, IAA, VQA] All-in-one Foundation Model for visual scoring. Can efficiently fine-tune to downstream datasets.

    Python 246 16

  4. Q-Instruct Q-Instruct Public

    ②[CVPR 2024] Low-level visual instruction tuning, with a 200K dataset and a model zoo for fine-tuned checkpoints.

    Python 190 8

  5. Q-Bench Q-Bench Public

    ①[ICLR2024 Spotlight] (GPT-4V/Gemini-Pro/Qwen-VL-Plus+16 OS MLLMs) A benchmark for multi-modality LLMs (MLLMs) on low-level vision and visual quality assessment.

    Jupyter Notebook 235 12

  6. Q-Bench-Video Q-Bench-Video Public

    A benchmark for video quality understanding of LMMs

    47

Repositories

Showing 10 of 13 repositories
  • Q-Bench-Video Public

    A benchmark for video quality understanding of LMMs

    Q-Future/Q-Bench-Video’s past year of commit activity
    47 0 0 0 Updated Sep 20, 2024
  • Q-Future/Compare2Score’s past year of commit activity
    Python 12 MIT 1 1 0 Updated Aug 24, 2024
  • Co-Instruct Public

    ④[ECCV 2024, Comparison among Multiple Images!] A study on open-ended multi-image quality comparison: a dataset, a model and a benchmark.

    Q-Future/Co-Instruct’s past year of commit activity
    60 4 2 0 Updated Aug 12, 2024
  • Q-Align Public

    ③[ICML2024] [IQA, IAA, VQA] All-in-one Foundation Model for visual scoring. Can efficiently fine-tune to downstream datasets.

    Q-Future/Q-Align’s past year of commit activity
    Python 246 16 9 0 Updated Aug 12, 2024
  • Q-Instruct Public

    ②[CVPR 2024] Low-level visual instruction tuning, with a 200K dataset and a model zoo for fine-tuned checkpoints.

    Q-Future/Q-Instruct’s past year of commit activity
    Python 190 8 11 0 Updated Aug 12, 2024
  • Q-Bench Public

    ①[ICLR2024 Spotlight] (GPT-4V/Gemini-Pro/Qwen-VL-Plus+16 OS MLLMs) A benchmark for multi-modality LLMs (MLLMs) on low-level vision and visual quality assessment.

    Q-Future/Q-Bench’s past year of commit activity
    Jupyter Notebook 235 12 1 0 Updated Aug 12, 2024
  • A-Bench Public

    [LMM + AIGC] What do we expect from LMMs as AIGI evaluators and how do they perform?

    Q-Future/A-Bench’s past year of commit activity
    125 3 0 0 Updated Aug 11, 2024
  • .github Public

    We are an open-source collaborative project to bring new possibilities to IQA!

    Q-Future/.github’s past year of commit activity
    2 0 0 0 Updated Aug 2, 2024
  • Q-Refine Public

    [MM 2024 Oral] Refiner for AIGC

    Q-Future/Q-Refine’s past year of commit activity
    Jupyter Notebook 23 Apache-2.0 1 1 0 Updated Jul 30, 2024
  • Q-Ground Public

    Official codes for "Q-Ground: Image Quality Grounding with Large Multi-modality Models", ACM MM2024 (Oral)

    Q-Future/Q-Ground’s past year of commit activity
    26 0 2 0 Updated Jul 28, 2024

Most used topics

Loading…