uzairansaruzi
Featured
hermex
Hermex is a native SwiftUI iPhone app that lets you control a self-hosted Hermes AI agent directly from your phone, with no subscriptions, tracking, or third-party relay.
Why it is worth attention
It is a research prototype with fully local processing, no cloud upload, detailed calibration with quality checks, and extensive evaluation and diagnostics for reproducible acoustic interaction.
Who it is for
- macOS power users seeking hands-free shortcuts
- researchers in acoustic human-computer interaction
- developers exploring on-device machine learning for spatial input
- productivity enthusiasts wanting context-dependent desk actions
Use cases
- Hands-free triggering of apps, commands, or shortcuts while typing or writing
- Assistive technology for users who have difficulty using a keyboard or mouse
- Desk-bound workflows like starting a timer, copying text, or speaking a message
- Experimental evaluation of acoustic sensing for device interaction
Strengths
- Privacy-first: all processing is on-device and raw audio is discarded by default
- Robust calibration with user guidance, consistency checks, and negative examples
- Multiple sensing modes (passive, active acoustic probe, hybrid) with diagnostic comparison
- Comprehensive evaluation tools producing JSON/CSV reports with latency and confusion matrices
Considerations
- Profile is specific to one MacBook, desk, room, and laptop position; external microphones rejected
- Accuracy depends on rigid, stable surfaces and consistent tapping; soft or noisy desks may fail
- Experimental active probe may be filtered by hardware or faintly audible; 80% accuracy and 200 ms latency targets are not universally validated
README quick start
Holo
Holo is an experimental native macOS utility that turns the desk immediately around a MacBook into four assignable tap zones. It listens through the selected Mac microphone, extracts acoustic features from short impulse windows, classifies the zone locally, and runs the action assigned to that zone.
The topology is intentionally four broad zones:
Display side
Left Rear ┌─────────────┐ Right Rear
│ MacBook │
Left Front └─────────────┘ Right Front
Trackpad side
Holo is a research prototype. Automated DSP tests pass, but useful accuracy still has to be measured on each real MacBook, desk, room, and laptop position. No physical accuracy claim is made without a saved 60-tap evaluation from that setup.
The requirement-by-requirement evidence ledger is in ACCEPTANCE.md.
What is implemented
- Four fixed zones: rear and front zones on each side of the MacBook.
- Explicitly armed calibration: ten accepted examples spread across each zone, 40 total, with clear retry guidance for weak, noisy, or clipped taps.
- A calibration-consistency check that identifies and can redo the weakest zone before saving.
- Adaptive streaming onset detection, sustained-sound rejection, and fixed 90 ms analysis windows.
- Passive tap acoustics, an optional active acoustic probe, and a hybrid mode.
- Robust feature normalization, a regularized linear zone model backed by nearest-example novelty checks, ambiguity rejection, out-of-distribution rejection, and optional negative examples.
- Per-profile actions: visual only, play a sound, copy or speak text, open a website, run a Shortcut, open an application or item, execute a shell command, or capture a screenshot. New zones default to visual-only until the user assigns a side effect.
- Guided 60-tap held-out evaluation with per-zone accuracy, latency, rejected-tap counts, and a confusion matrix.
- Saved evaluation history is restored after relaunch and scoped to the desk profile that produced it.
- Signal diagnostics, labeled feature capture, approach comparison, JSON/CSV reports, and opt-in raw debug WAV capture.
- Sandboxed, local persistence. Raw audio is discarded by default.
- Core Audio route validation requires the built-in microphone for every mode and the built-in speakers for Active or Hybrid sensing.
Requiremen
Related repositories
Similar projects matched by category, topics, and programming language.
simonw
Featured
speech-analyzer-cli
A macOS CLI tool that transcribes prerecorded audio using Apple's on-device speech APIs and outputs plain text, JSON, JSONL, SRT, or WebVTT.
andyhuo520
MacCheck
MacCheck is a macOS local verification tool for second-hand Mac transactions, shop inspections, and personal self-checks, offering automatic hardware detection and interactive tests with PDF/PNG report export.