It demonstrates that a 1.35M-parameter model trained entirely from random initialization can achieve 99% pass rate on behavioral prompts, providing a reproducible end-to-end example of building a custom language model without pretrained weights or large datasets.
A tiny conversational decoder-only Transformer trained entirely from scratch on an original Minionese-inspired dataset. It accepts short English or Minionese messages and replies in a playful, consistent Minionese project dialect.
This project does not use pretrained weights, film subtitles, screenplay text, iMessages, or code from the project that inspired the experiment. Minionese is an improvised fictional dialect rather than a language with fixed grammar, so this is explicitly an unofficial fan research project—not an official translator or an Illumination product.
The exact parameter count is printed by minionese doctor and saved with every training report.
The included local artifact was trained from random initialization for 5,000 batches and 9,981,697 sampled non-padding tokens. On 400 held-out behavioral prompts it achieved 99.0% intent/style/clean pass rate (99.5% for English inputs and 98.5% for Minionese inputs), with test perplexity 1.116 versus a 4.688 unigram baseline. See the full results and artifacts/evaluation.json.
Python 3.11+ and Apple Silicon are required to reproduce training.
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[apple,dev]'
minionese doctor
minionese prepare
minionese train-tokenizer
minionese train
minionese evaluate
Training defaults to 5,000 batches, approximately 8–10 million sampled non-padding tokens depending on generated prompt lengths. The tokenizer and every model parameter begin from random/untrained state. Artifacts are written below artifacts/; large generated data and weights are ignored by Git.
The ready-to-use export is in artifacts/final. To use it explicitly:
minionese reply "hello" --model artifacts/final --tokenizer artifacts/final
The checked-in safetensors export can also run with PyTorch on a CPU-only Linux, macOS, or Windows machine; MLX and Apple Silicon are not required:
python3 -m venv .venv
sourc
A tiny conversational Minionese language model trained from scratch on Apple Silicon
Similar projects matched by category, topics, and programming language.
Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.
A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.
mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.