Presentation: Scaling LLM-Based Ranking Systems for Latency-Critical Search & Recommendation Workloads

📝

内容提要

Sundara Raman Ramachandran shares how LinkedIn productionizes LLM ranking under strict P99 latency SLAs. He discusses distilling a 7B model down to 400M parameters, compressing job contexts into...

➡️

继续阅读