wanshuiyin GitHub avatar

Anti-Autoresearch

wanshuiyin

Anti-Autoresearch is an open-source tool that audits academic submissions for self-consistency and fabrication, producing a reviewer-ready report by combining an evidence ledger, LLM-proposed findings, and a deterministic adjudicator.

Stars

114

7-day growth

No data

Forks

4

Open issues

0

License

MIT

Last updated

2026-07-14

AI repository intelligence
FR-AI / ANALYSIS

Why it is worth attention

It addresses the growing problem of unreliable machine-generated papers by providing a principled, deterministic audit system that avoids opaque AI-text classification and instead checks factual coherence—backed by the proven ARIS platform (12.5k★) and a published taxonomy of 46 integrity patterns.

Who it is for

  • Reviewers and area chairs assessing submissions
  • Authors verifying their own papers before submission
  • Conference program committees and integrity officers
  • Researchers studying reproducibility and research integrity

Use cases

  • Auditing a submission during the review process for numeric, citation, or method inconsistencies
  • Checking a paper's claims against its code and results (at L2 observability)
  • Generating a structured integrity report for a paper under rebuttal or revision
  • Batch-screening a set of submissions for potential fabrication indicators

Strengths

  • Evidence ledger anchors every finding to a specific, hashed span from the paper, preventing vague claims
  • Deterministic adjudicator (pure rules) computes the final verdict, so no LLM grades its own output
  • Observability levels (L0, L1, L2) automatically demote findings that require code when only a PDF is available
  • Built on battle-tested ARIS audit stack (12.5k★, HF Daily #1) with an eval harness and regression tests

Considerations

  • Outputs are flags for human review, never a proof of misconduct
  • PDF-only (L0) catches inconsistency and surface tells but cannot verify external ground truth or run code
  • False positives exist (e.g., legitimate round numbers); the taxonomy is a living document and adversaries can route around known signals

README quick start

Anti-Autoresearch 🛡️

· · · · · · · ·

🔬 The field has tolerated unreliable autoresearch long enough — Anti-Autoresearch is the read that finally catches it.

天下苦 autoresearch 久矣 —— Anti-Autoresearch 替研究者们一眼看穿不靠谱的工作。

🏆 Built on a battle-tested foundation: ARIS (~12.5k★ · HuggingFace Daily Papers #1 · 78+ skills across 7+ platforms). Anti-Autoresearch points ARIS's production audit DNA (experiment-audit · paper-claim-audit · citation-audit · kill-argument) outward — auditing a third party's submission instead of your own.

Autoresearch has gone mainstream, and a fast-growing share of what reaches the review pile is machine-generated — and a lot of it doesn't hold up: tables that don't match the text, baselines that aren't there, open-sourced code that won't reproduce its own paper. Reviewers, area chairs, and honest authors increasingly need to verify that, not just suspect it.

Regardless of who or what wrote a paper, does the science hold together and reflect its own evidence? Anti-Autoresearch audits a submission for self-consistency and fabrication, and produces a span-anchored, reviewer-ready report. It is not an opaque AI-text classifier (no authorship probabilities, no "AI-written" verdict) and does not judge misconduct — it surfaces discrepancies a human reviewer should investigate. Separately, it lists transparent, itemized AI writing-style impressions in a quarantined, zero-verdict-weight section (a paper can be integrity-CLEAN while listing many), because reviewers react to them.


🧭 What's inside

46 integrity patterns across 8 families — the coverage vocabulary every finding cites — plus 13 zero-weight AI writing-style impressions and 2 advisories:

FamilyCatches
ANumeric self-consistency数值自洽:table vs text vs delta arithmetic that doesn't add up
BMethod & scope方法与范围:the described method/scope ≠ wh

Description

Don't trust an autoresearch paper at face value. Reviewer-side integrity forensics — self-consistency + fabrication checks across 39 hack-patterns (7 families), deterministic verdict. Not an AI-text detector. The dual of ARIS.

Related repositories

Similar projects matched by category, topics, and programming language.

lopopolo
Featured
lopopolo GitHub avatar

harness-engineering

Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.

AI & Machine LearningAI Agents
2,390
slvDev
Featured
slvDev GitHub avatar

esp32-ai

A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.

AI & Machine LearningLarge Language Models
1,960
littledivy
Featured
littledivy GitHub avatar

mimic

mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.

AI & Machine Learning
1,482