为什么值得关注

它展示了仅 135 万参数、完全随机初始化的模型在行为提示上达到 99% 通过率的能力，提供了不依赖预训练权重或大型数据集、可复现的完整语言模型构建范例。

适合谁使用

对从零训练小型语言模型感兴趣的爱好者
探索生成式 AI 的 Minionese 或虚构语言粉丝
研究最小规模语言模型训练和评估的研究人员
寻求轻量、可移植对话演示的开发者

典型使用场景

与 Minionese 聊天机器人进行趣味对话
教育性演示无需迁移学习即可训练解码器 Transformer
在低资源环境中进行可移植的 CPU 推理
实验自定义分词器和虚构语言数据集

项目优势

完全可复现的训练流程（从随机初始化开始）并附有详细评估结果（通过率 99%，困惑度 1.116）
使用 PyTorch 在 Linux、macOS 和 Windows 上实现可移植的 CPU 推理
模型极小（135 万参数），在其特定领域内具备合理的对话质量
明确说明限制和域外行为，提供诚实的预期

使用前须知

上下文窗口小（256 tokens），词汇量有限（512 BPE tokens），难以理解复杂或域外英语
训练需要 Apple Silicon；CPU 推理支持但训练不能跨平台
模型是粉丝项目而非官方翻译器，对任意输入可能产生不匹配或错误的回复

README 快速开始

Minionese LM

A tiny conversational decoder-only Transformer trained entirely from scratch on an original Minionese-inspired dataset. It accepts short English or Minionese messages and replies in a playful, consistent Minionese project dialect.

This project does not use pretrained weights, film subtitles, screenplay text, iMessages, or code from the project that inspired the experiment. Minionese is an improvised fictional dialect rather than a language with fixed grammar, so this is explicitly an unofficial fan research project—not an official translator or an Illumination product.

Architecture

4 causal decoder layers
160 model width, 5 attention heads, 640 MLP width
learned positional embeddings and tied token/output embeddings
512-token custom byte-level BPE tokenizer
256-token context window
MLX training on Apple Silicon, plus PyTorch CPU inference on Linux, macOS, and Windows

The exact parameter count is printed by minionese doctor and saved with every training report.

Trained result

The included local artifact was trained from random initialization for 5,000 batches and 9,981,697 sampled non-padding tokens. On 400 held-out behavioral prompts it achieved 99.0% intent/style/clean pass rate (99.5% for English inputs and 98.5% for Minionese inputs), with test perplexity 1.116 versus a 4.688 unigram baseline. See the full results and artifacts/evaluation.json.

Set up and reproduce

Python 3.11+ and Apple Silicon are required to reproduce training.

python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[apple,dev]'
minionese doctor
minionese prepare
minionese train-tokenizer
minionese train
minionese evaluate

Training defaults to 5,000 batches, approximately 8–10 million sampled non-padding tokens depending on generated prompt lengths. The tokenizer and every model parameter begin from random/untrained state. Artifacts are written below artifacts/; large generated data and weights are ignored by Git.

The ready-to-use export is in artifacts/final. To use it explicitly:

minionese reply "hello" --model artifacts/final --tokenizer artifacts/final

Portable CPU inference

The checked-in safetensors export can also run with PyTorch on a CPU-only Linux, macOS, or Windows machine; MLX and Apple Silicon are not required:

python3 -m venv .venv
sourc

项目描述

A tiny conversational Minionese language model trained from scratch on Apple Silicon

minionese