shure-dev GitHub avatar

small_vlm_video_analysis

shure-dev

A research repository for temporal grounding in industrial VLM, using small models (≤4B) to detect procedure deviations in factory ego videos, with a human-in-the-loop annotation and evaluation pipeline.

Stars

48

7-day growth

No data

Forks

10

Open issues

6

License

MIT

Last updated

2026-07-19

AI repository intelligence
FR-AI / ANALYSIS

Why it is worth attention

It addresses a critical industrial safety problem by focusing on small, edge-deployable models and provides a reproducible benchmark with human-annotated factory videos, including an open-source annotation tool and comparative evaluation of multiple vision-language models.

Who it is for

  • Researchers in vision-language temporal grounding
  • Industrial automation and safety engineers
  • Computer vision practitioners working on video understanding
  • Developers building edge-based worker assistance systems

Use cases

  • Real-time detection of step omissions or sequence errors on assembly lines
  • Post-hoc analysis of worker training videos to identify risky actions
  • Prototyping edge-compatible VLMs for private, on-site video analysis
  • Building datasets and evaluation metrics for industrial procedure grounding

Strengths

  • Provides a focused benchmark on 20 factory ego videos with 68 human-annotated events and standard metrics (tIoU)
  • Demonstrates Marlin-2B achieving 0.400 mean tIoU, outperforming larger models on this task
  • Includes an integrated web application for annotation, prediction review, and dataset management
  • Designed for small models (≤4B) to enable future on-device or edge deployment

Considerations

  • Currently limited to offline short clips (20 seconds); real-time streaming and edge deployment are not yet validated
  • Evaluation is on a fixed set of 20 videos (development split), not a held-out test set
  • Many candidate models (e.g., MiniCPM, InternVL3) are not yet compatible due to conversion/processor errors

README quick start

Quick start

Description

作業動画が手順書(SOP)どおりかを、ローカルの小型VLM(Qwen3-VL / Apple Silicon)だけでチェックするデモ

Related repositories

Similar projects matched by category, topics, and programming language.

gavamedia
Featured
gavamedia GitHub avatar

deltafin

Deltafin is a research project that runs the 2.8-trillion-parameter Mixture-of-Experts model Kimi K3 on a single Apple Silicon Mac (e.g., M1 Max with 64 GB) at about 16 seconds per token, using exact, reproducible inference with local or streaming expert loading.

AI & Machine LearningLarge Language Models
304
ammaarreshi
ammaarreshi GitHub avatar

Generals-Mac-iOS-iPad

This repository ports Command & Conquer Generals: Zero Hour to run natively on Apple Silicon Macs, iPhones, and iPads without emulation, adding touch controls and modern rendering.

C++
1,523
didriksg
didriksg GitHub avatar

Crisp

Crisp is a free, open-source macOS menu bar app that adds HiDPI scaling, brightness control, and other display management features for external monitors, serving as an alternative to paid tools like BetterDisplay and Lunar.

Swift
476