§ Infrastructure

GPT-6 introduces improved prompt caching with higher hit rates, diagnostics, explicit breakpoints, and controls to reduce latency and inference costs.

Flash-dLLM optimizes KV caching and parallel decoding for diffusion LLMs via IO-aware techniques, addressing inference bottlenecks in non-autoregressive text generation.

Agensh scales multi-agent systems to 1,024 agents using decentralized self-organized coordination without central orchestrator bottleneck.

SWE-Serve benchmark evaluates agents on production inference engineering tasks spanning model support, runtime execution, and public APIs.

On-policy distillation (OPD) fixes quantization exposure bias in sub-3-bit LLMs; restores math and code reasoning in low-bit models.

GravityOCR uses diffusion-based parallel decoding with AR verification to accelerate document OCR inference beyond sequential token generation.

Greedy LLM decoding produces different outputs across BF16/FP16 precision with 49-100% prompt divergence, challenging determinism assumptions in inference.

REFLEX agent architecture combines fast typed decision layer (Jev) with LLM fallback, achieving 95% task success with 72.7% fewer strong-model calls.

HySparse2 hybrid sparse attention architecture with two-level KV sharing reduces prefill cost and memory for long-context agent interactions.

Disaggregated quantization specializes compute formats for LLM prefill vs. decode phases; tested on Qwen 3 and Gemma 3.

CompKV jointly optimizes KV cache token selection and compensation to reduce memory traffic during long-context LLM inference.

GeoPair: training-free Transformer compression via cross-layer factorization preserving activation geometry.

Activation-aware mixed-precision LLM quantization with cross-layer refinement proves scalar sensitivity proxies incur multiplicative Hessian distortion up to 10^13.

Cloudflare Python Workers reaches general availability, enabling Python code execution via WebAssembly in their serverless platform.

WorldCrafter improves video world models with camera-queryable 3D-aware memory to maintain consistency across long horizons and viewpoints.

Hypernetwork-generated LoRA weights enable efficient on-device LLM personalization by mapping user context to model adapters without retraining.

SPECTRA adapts speculative decoding on reconfigurable hardware for edge LLM inference, managing varying arithmetic intensity across verification regimes.

Uranus: data-driven robot simulator using joint-trajectory-conditioned diffusion for streaming, low-latency policy training without fixed horizon.

Explanation-aware post-training quantization for medical LLMs preserves rationale quality alongside answer accuracy in multiple-choice QA.

NPU accelerator design for real-time YOLO vehicle detection on PYNQ-Z1 using quantization and FINN compilation.

Fine-tuning Google Research's TimesFM foundation model for real-time feature extraction in particle physics detector data acquisition systems.

Machine-Interpretable Information (MII) protocol for compiling documents into fixed-bandwidth agent-readable states, solving index-payload schism in retrieval-augmented systems.

Low-rank factorization reduces pitch estimation model size 36% on edge devices without accuracy loss.

Graph-theoretic analysis of LoRA adapter co-occurrence in Alibaba production diffusion inference workloads using 26 million inferences.

Comparative benchmark of graph database engines on LLM agent workloads isolates query planning and ingest costs across vendors.

ValueDiff proposes value-geometric KV cache eviction for LLMs with weak attention sinks, ranking tokens by value-vector L2 deviation from cache mean.

datasette-explain 0.2.2 adds query plan visualization for read-only stored-query pages.

datasette-auth-github reaches 1.0 with session persistence fix for mobile browsers.

FPGA-accelerated quantized RNN achieves sub-10μs real-time plasma state prediction for Tokamak control via hls4ml synthesis on Xilinx Alveo U50.

Novel Poisson-process speculative sampling algorithm improves trade-off between LLM inference efficiency and watermarking for output provenance.

Lightweight embedding translator and optimization sweep reduce diffusion model footprint for on-device inference on consumer GPUs.

RheoSampling fixes dynamic-tree speculative decoding collapse in stochastic sampling (T>0) to preserve context-aware topology and acceptance rates.

TERMon detects runtime anomalies in edge AI accelerators via hardware-native ternary monitoring without model re-execution.

SpecQuant combines speculative decoding with multi-parent quantization for training-free efficient LLM inference on consumer hardware.

L0-MoE accelerates dense LLMs via L0-regularized Mixture-of-Experts, achieving 2.5x speedup with minimal performance loss.

Samsone family of small audio language models (99M–356M params) achieving SOTA on edge inference benchmarks.

Lightweight memory method for robotic policies using saliency supervision at training time reduces deployment-time VLM overhead.

Score centering corrects training-inference mismatch drift in LLM RL; stabilizes off-policy learning without eliminating TIM overhead.

On-Demand Attention trains lightweight recall head to selectively activate global attention during decoding, reducing long-context inference cost.

Framework improves spreadsheet chunking for LLM-RAG via semantic cell role annotation, identifying fundamental limits of classification-based approaches.

Survey: RISC-V open processor ISA in ML applications; analyzes ecosystem from instruction extensions through deployment, commercial and academic implementations.

FL-Net: federated learning framework for multi-center clinical research with data harmonization, disclosure control, versioning; evaluated against 14 existing frameworks.

Federated learning approach reduces communication overhead in sharded data parallelism for large-scale foundation model training on heterogeneous multi-GPU clusters.

Hybrid speculative decoding strategy dynamically switches between neural drafting and context copying to maximize LLM inference speedup.

IBM Granite 5.0 TurboCTC: 470M-param Conformer ASR model with strided convolutions and Muon optimizer optimization.

Yegge shuts down Gas Town project; Databricks raises Astra vector DB costs 60%, prompting reality checks on AI industry hype and unit economics.

OpenAI launches Astra for Law, a domain-specific application with workflow customization, data integration, and compliance controls for legal firms.

ScienceIDE converts scientific code repositories into agent-learnable environments by bridging fragmented toolchains and domain conventions.

rMuscle optimizes Vision-Language-Action model inference latency for factory robotics by exploiting embodied workload characteristics.

RLLBC-Lib educational code library for tabular reinforcement learning and control, targeting student accessibility.