Why it is worth attention

It offers a fully local, open-source AI agent with two distinct modes (tutor and agent), voice control, and a kill switch, all without any cloud connectors beyond the user's own OpenAI API.

Who it is for

Mac power users seeking local automation
Developers experimenting with AI agent frameworks
Educators and trainers demonstrating macOS workflows
Privacy-conscious users who want full control over API usage

Use cases

Automating repetitive macOS tasks like file organization or app launching
Learning how to perform operations on a Mac via the tutor mode's step-by-step explanations
Creating and triggering saved workflows with custom voice commands
Testing and troubleshooting AI agent behavior in a controlled local environment

Strengths

Two distinct modes (tutor and agent) with enforced code-level restrictions
Voice input with adaptive noise gating and hands-free listening
Workflow system for saving and reusing task sequences
Cursor-preserving control via Accessibility API, falling back to CGEvent only when necessary

Considerations

Requires a valid OpenAI API key and incurs usage costs
Only supports macOS 13+ and needs manual permission grants for screen recording, accessibility, and microphone
No built-in cloud service integrations (Notion, Gmail, etc.) and no confirm-before-act mechanism

README quick start

UltraClicky

An open-source, local AI agent that lives in your menu bar, sees your screen, and drives your Mac. Summon it next to your cursor, type a request, and it either answers + points at the screen (tutor mode) or does the task (agent mode) by controlling the Mac itself.

There are no cloud connectors (no Notion, Gmail, Calendar, etc.). The agent's only powers are: see the screen, and drive this Mac. The loop runs against your own OpenAI key through a speech-to-speech Realtime session when voice is active.

macOS 13+ · Swift · AppKit + SwiftUI · built with XcodeGen.

Setup — the only three things you do

Paste your OpenAI key into a local .env file in the project root:
```
api-key=''
```
(Copy .env.example to .env if you don't have one. Never commit your real key.)

Generate the project, then build and run:

brew install xcodegen        # if you don't have it
xcodegen generate            # creates Clicky.xcodeproj from project.yml
open Clicky.xcodeproj         # then build & run (⌘R) in Xcode

…or from the command line:

xcodebuild -project Clicky.xcodeproj -scheme Clicky -configuration Debug build

On first launch, grant the three macOS permissions when prompted. The setup window lists each one with a button that deep-links to the right System Settings pane:
- Screen Recording — so the agent can see what you see.
- Accessibility — so it can read the UI and act in place (without moving your cursor).
- Microphone — only for voice input from the mic button.
These prompts require manual approval in System Settings and cannot be skipped or pre-granted by code. The setup window re-checks status when you return to it.

If api-key is missing or empty at launch, the setup window shows a clear message pointing you to .env. The key is only ever read from local runtime configuration — it is never hardcoded.

.env is ignored by Git and is not copied into the app bundle. If you launch the app from somewhere that cannot find the project-root .env, point at a specific file with the CLICKY_ENV_PATH environment variable.

How to use it

Summon: press ⌃⌥ (Control + Option). A floating, semi-transparent input panel appears near your cursor. P

Description

A repository for ultraclicky

Related repositories

Similar projects matched by category, topics, and programming language.

uzairansaruzi

Featured

hermex

Hermex is a native SwiftUI iPhone app that lets you control a self-hosted Hermes AI agent directly from your phone, with no subscriptions, tracking, or third-party relay.

AI & Machine LearningLarge Language Models

941

makecindy

Featured

cindy

Cindy is an open-source AI agent that runs locally on your machine, integrates multiple AI harnesses and models, and provides memory, skills, and automation to perform real work in your projects and apps.

AI & Machine LearningLarge Language Models

958

Blaizzy

Featured

nativ

Nativ is a native macOS app that lets you run AI models locally on Apple silicon, offering chat, model management, performance analytics, and an OpenAI/Anthropic-compatible API server.

AI & Machine LearningAI Vision & Audio

985