VisionBridge is an OpenAI-compatible proxy that gives text-only LLMs vision capabilities by routing image-related tasks to a separate vision model through tool calling.

Stars

39

7-day growth

No data

Forks

2

Open issues

1

License

MIT

Last updated

2026-07-07

AI repository intelligence
FR-AI / ANALYSIS

Why it is worth attention

It enables any text-only reasoning model (from LM Studio, Ollama, vLLM, etc.) to answer questions about images without training or new weights, using a tiny proxy that handles tool orchestration, error recovery, and concurrent vision calls.

Who it is for

  • Developers running local or self-hosted LLMs who want to add vision to text-only models
  • Users of OpenWebUI, LibreChat, or similar chat UIs looking for multi-modal support without switching models
  • Researchers evaluating tool-based vision strategies without modifying model code
  • Engineers building custom agentic workflows that combine reasoning and vision backends

Use cases

  • Extract text from scanned documents or images via OCR and full-page scan tools
  • Answer visual questions by zooming into specific regions of an image
  • Compare two images side-by-side with a reasoning model that asks the vision model focused questions
  • Enable a text-only chatbot to handle image uploads in chat applications

Strengths

  • Supports multiple reasoning and vision backends (LM Studio, Ollama, vLLM, OpenAI-compatible) with easy Docker deployment
  • Automatic fallback from native tool calling to prompt-JSON protocol for models that lack tool support
  • Concurrent tool execution and scene priming reduce latency and give the reasoning model context before detailed inspection
  • Robust error handling for hallucinated IDs, malformed JSON, and backend failures, with forced final answer on reaching the turn limit

Considerations

  • Requires running both a reasoning model and a vision model, increasing hardware and operational complexity
  • Adds latency due to proxy routing and multi-turn tool loops, especially when images are large or network backends are slow
  • No built-in authentication mechanism; production deployments must use an external reverse proxy for secure exposure

README quick start

Quick Start With Docker

Description

Give text-only LLMs vision. A tiny OpenAI-compatible proxy that lets reasoning models (DeepSeek, Qwen, GLM…) see images by querying a separate vision model through tools: look, OCR, scan, crop, compare. No training, no weights.

Related repositories

Similar projects matched by category, topics, and programming language.

lopopolo
Featured
lopopolo GitHub avatar

harness-engineering

Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.

AI & Machine LearningAI Agents
2,390
slvDev
Featured
slvDev GitHub avatar

esp32-ai

A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.

AI & Machine LearningLarge Language Models
1,960
littledivy
Featured
littledivy GitHub avatar

mimic

mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.

AI & Machine Learning
1,482