~/apah
README.md spec.md setup.md apah_doc.tex github.com

apah▋

// sovereign, air-gapped LLM inference for NVIDIA GPUs

High-performance LLM inference runtime built for air-gapped industrial environments. Delivers continuous batching, PagedAttention KV memory management, multi-GPU tensor parallelism, real-time GPU hardware stats, and process-level socket isolation.

install view on github
Phase 11 Benchmark: Up to 6.6x aggregate throughput vs. Ollama under agent concurrency →

the_problem

throughput_ceiling

DIFF apah vs standard_runtimes.py 
- Ollama / Standard: serial request locking under concurrent agent demand
- Contiguous KV Cache: static VRAM allocation wasting up to 87.5% memory
- Default Sockets: unmonitored egress creating security compliance risk
+ Apah Engine: continuous batching scheduler for parallel iteration decoding
+ PagedAttention: demand-paged 16-token memory blocks with zero padding
+ Air-Gap Hardening: socket interception + SHA256 audit log verification

// traditional runtimes lock GPU generation threads and fragment memory under multi-agent load.

architecture

execution_pipeline

Apah combines custom CUDA kernels with a FastAPI server exposing OpenAI-compatible endpoints and process-level air-gap security:

  1. OpenAI FastAPI Server: Compatible shims for /v1/chat/completions, tool calling, embeddings, and Ollama client drop-in support.
  2. Continuous Batching Scheduler: Iteration-level sequence queue admitting prompts dynamically between decode iterations.
  3. PagedAttention BlockPool: Virtual memory manager allocating physical 16-token KV cache pages without memory fragmentation.
  4. Tensor Parallel Engine: NCCL AllReduce distributing Megatron-style Column/Row matrix projections across N NVIDIA GPUs.
  5. Air-Gap Hardening & Audit: Kernel socket interception blocking unauthorized egress with tamper-evident SHA256 audit log chains.

benchmarks

performance_metrics

6.6x
Throughput Gain (342 tok/s vs 44 tok/s @ c=16)
87.5%
PagedAttention KV VRAM Savings
100%
Air-Gapped Process Isolation

// benchmarked on NVIDIA H100/A100 hardware across multi-concurrency agentic workloads.

install

one_line_installer

Install Apah CLI and inference engine via shell installer:

~/apah $ curl -fsSL https://p19racha.github.io/Apah/install.sh | sh

windows (powershell)

irm https://p19racha.github.io/Apah/install.ps1 | iex

air-gapped / offline staging

https://github.com/p19racha/Apah/archive/refs/tags/v1.tar.gz

commands

cli_reference

apah serve Start the Apah inference server in the foreground (blocking).
apah run Run model inference (one-shot prompt or interactive chat REPL).
apah pull Pull model weights, version, compute checksums, and verify manifest integrity.
apah list List locally downloaded model manifests across all versions.
apah ps List models currently loaded in server GPU memory.
apah gpu Display real-time NVIDIA GPU hardware metrics (memory, utilization %, temperature).
apah stop Unload the currently loaded model from server GPU memory.
apah rm Remove a local model directory or specific version from ~/.apah/models/.
apah show Show detailed manifest, checksum verification summary, and live server status.
apah audit verify Verify SHA256 hash chain integrity of audit log files.

security_modes

airgap_levels

standard

Standard network sockets enabled. Best for development, local debugging, or non-sensitive staging environments.

[default]

airgap_enforced

Process-level socket creation intercepted and blocked. Zero outbound network egress allowed.

hardened_audit

Strict air-gap enforcement combined with SHA256 cryptographic tamper-evident audit logging for industrial compliance.