yangg1224 GitHub avatar

smart_remarkable

yangg1224

Smart Remarkable is a Vision-LLM agent that monitors handwriting on a reMarkable tablet, captures screenshots via a trigger, sends them to an LLM, and renders the response as pen strokes or typed text directly onto the screen.

Stars

61

7-day growth

No data

Forks

7

Open issues

0

License

MIT

Last updated

2026-07-09

AI repository intelligence
FR-AI / ANALYSIS

Why it is worth attention

It extends the original ghostwriter with a novel Select Mode for lassoing specific handwriting regions, injects LLM and Draw buttons into the native UI via a XOVI extension, and supports image-generation for sketches, all while running entirely on-device and integrating with multiple LLM providers.

Who it is for

  • reMarkable tablet power users seeking AI-assisted note-taking
  • developers building or extending e-ink integration tools
  • researchers and students who want to query LLMs directly from handwritten notes
  • makers interested in combining hardware touch input with vision-language models

Use cases

  • Ask an LLM to answer a handwritten question and have the answer drawn on the same page
  • Select a region of handwritten text to get a targeted explanation or summary from an AI
  • Use the Draw button to generate a small illustration or improve an existing sketch based on the selected content
  • Offline testing and evaluation of the pipeline on a desktop without reMarkable hardware

Strengths

  • Deep reMarkable integration: hooks into xochitl's framebuffer, UI scene graph, and input devices for seamless on-device operation
  • Multiple trigger modes (corner tap, four‑finger gesture, physical LLM/Draw buttons) provide flexible invocation
  • Support for OpenAI, Anthropic, and Google LLMs with tool‑calling, plus image‑generation via Gemini
  • Select Mode with rotation-aware cropping, scaling, and native selection manipulation enables precise interactive workflows

Considerations

  • Requires enabling Developer Mode on the tablet, which voids the warranty and resets the device to factory defaults
  • Relies on API keys for each LLM provider; image generation incurs a per‑image cost (~$0.04) and is not available in the Gemini free tier
  • Setup involves cross‑compilation and SSH deployment; no pre‑built binaries are provided, and the armv7 cross‑compilation path lacks a verified Docker‑free method

README quick start

Usage

Description

Add 1000 IQ to your remarkable ink screen

Related repositories

Similar projects matched by category, topics, and programming language.

l0ng-ai
Featured
l0ng-ai GitHub avatar

tty7

tty7 is a fast, GPU-accelerated terminal workbench with persistent sessions, built-in input tools, SSH support, and coding agent awareness.

Developer ToolsCLI & Terminal
359
steelbrain
Featured
steelbrain GitHub avatar

reims-vgpu

reims-vgpu is an experimental virtual GPU for macOS guests that uses QEMU to decode the guest's GPU command stream and execute it through Metal or Vulkan, leveraging the built-in AppleParavirtGPU driver without requiring custom kexts.

Desktop Apps
143
m-novotny
Featured
m-novotny GitHub avatar

memguard-rs

A Rust library that provides secure memory handling primitives including zeroization on drop, memory locking, constant-time comparison, and compile-time guarded regions, with zero dependencies and no_std support.

Embedded & IoTSecurity
131