futz12 GitHub avatar

bergamot-ncnn-vulkan

futz12

A local Marian/Bergamot translation runtime using C++17, ncnn, and optional Vulkan GPU acceleration.

Stars

6

7-day growth

No data

Forks

1

Open issues

0

License

Apache-2.0

Last updated

2026-07-27

AI repository intelligence
FR-AI / ANALYSIS

Why it is worth attention

It enables offline, privacy-preserving machine translation on consumer hardware with both CPU and Vulkan GPU backends, and provides ready-to-run FP32 and INT8 models.

Who it is for

  • Developers integrating local translation into applications
  • Users needing offline translation without cloud dependency
  • Researchers exploring efficient neural translation on CPU/GPU
  • Privacy-conscious users who want on-device processing

Use cases

  • Translating sensitive documents locally without sending data to servers
  • Embedding translation in desktop or mobile applications using ncnn
  • Running translation on devices without internet connectivity
  • Experimenting with Marian-based models on Vulkan-capable hardware

Strengths

  • Supports both CPU and Vulkan GPU execution for flexible performance
  • Includes pre-built FP32 and INT8 quantized models for immediate use
  • Provides a custom tokenizer and sentence splitting for raw text input
  • Modular design with separate model exporter and comprehensive documentation

Considerations

  • Only greedy decoding is available, no beam search or sampling
  • No graph-level batching; each segment is processed sequentially
  • Custom tokenizer is an approximation and not fully SentencePiece-compatible
  • INT8 multi-head attention falls back to CPU when using Vulkan backend

README quick start

Quick start

Description

mobile-friendly mechine translator

Related repositories

Similar projects matched by category, topics, and programming language.

lopopolo
Featured
lopopolo GitHub avatar

harness-engineering

Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.

AI & Machine LearningAI Agents
2,390
slvDev
Featured
slvDev GitHub avatar

esp32-ai

A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.

AI & Machine LearningLarge Language Models
1,960
littledivy
Featured
littledivy GitHub avatar

mimic

mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.

AI & Machine Learning
1,482