Pyroxide is a lock-free, high-concurrency background task broker for Python, powered by Rust, that enables GIL-free execution via WASM sandboxes and dynamic native plugins with microsecond latency and zero infrastructure.

Stars

17

7-day growth

No data

Forks

0

Open issues

0

License

Apache-2.0

Last updated

2026-07-26

AI repository intelligence
FR-AI / ANALYSIS

Why it is worth attention

It uniquely combines in-process GIL bypass, WASM sandbox security, and on-the-fly native compilation (Rust/C/Zig) into a single library, achieving up to 36x speedups over standard Python concurrency approaches without requiring external services like Redis or Celery.

Who it is for

  • Python developers building high-throughput, latency-sensitive applications
  • FastAPI / async loop users needing to offload CPU-bound tasks without freezing
  • Developers of multi-tenant SaaS platforms that must safely run untrusted user code
  • Data engineers and scientists handling large payloads with zero-copy shared memory

Use cases

  • Offload heavy calculations from async event loops (e.g., FastAPI) to avoid blocking
  • Run untrusted user plugins in secure WASM sandboxes with memory and CPU limits
  • Reuse existing native system libraries (.so/.dylib) without writing wrapper code
  • Batch-submit and manage large numbers of short-lived background tasks with minimal overhead

Strengths

  • GIL bypass through WASM and dynamic native plugins without multiprocessing overhead
  • Microsecond task dispatch latency (<25 µs) using OS-level signaling instead of polling
  • Zero infrastructure required – runs entirely in-process with no Redis/RabbitMQ/Celery
  • Zero-copy shared memory transport for payloads over 1 MB, eliminating serialization bottlenecks

Considerations

  • Dynamic compilation of native plugins (Rust/C/Zig) requires a local toolchain and may be disabled for security
  • WASM sandbox execution is limited by configurable memory/timeout defaults and not suitable for all workloads
  • Project is relatively new; community adoption and production maturity may be limited compared to Celery or multiprocessing

README quick start

Installation

Description

A high-concurrency, lock-free task broker bridging Python and Rust via PyO3. Bypasses the GIL for high-throughput background processing.

Related repositories

Similar projects matched by category, topics, and programming language.

l0ng-ai
Featured
l0ng-ai GitHub avatar

tty7

tty7 is a fast, GPU-accelerated terminal workbench with persistent sessions, built-in input tools, SSH support, and coding agent awareness.

Developer ToolsCLI & Terminal
359
m-novotny
Featured
m-novotny GitHub avatar

memguard-rs

A Rust library that provides secure memory handling primitives including zeroization on drop, memory locking, constant-time comparison, and compile-time guarded regions, with zero dependencies and no_std support.

Embedded & IoTSecurity
131
inbjo
inbjo GitHub avatar

MirrorProxy

MirrorProxy is a self-hosted, Rust-based mirror proxy platform with an embedded React admin console and cross-platform client for managing package sources.

Rust
204