Artificial Intelligence
Score 96/100
Speculative Decoding and KV-Cache Quantization in Sub-Millisecond LLM Serving
How speculative draft verification and FP8 key-value cache quantization reduce time-to-first-token and inter-token latency without degrading reasoning accuracy.