XNNPack的全连接和卷积2D运算符现在支持动态范围量化，在TensorFlow Lite中提高了CPU推理性能。动态范围量化允许在旧的和低端设备上部署更多的AI功能。它通过根据观察到的激活范围动态计算量化参数来最大化量化过程的准确性。动态范围量化提供了与完全量化相似的性能提升，并具有更高的整体准确性。XNNPack的优化运算符可用于各种架构上的动态范围量化模型。与完全整数量化相比，动态范围量化对非专家用户更加易于使用。混合精度推理，结合半精度推理和动态范围量化，可以进一步提高CPU推理性能。基准测试显示，与使用TFLite的内核的原始float32模型相比，速度提升。

Posted by Alan Kelly, Software Engineer We are happy to share that detailed profiling information for XNNPACK is now available in TensorFlow 2.9.1 and later. XNNPACK is a highly optimized library...

Posted by Zhi An Ng and Marat Dukhan, Google XNNPack is the default TensorFlow Lite CPU inference engine for floating-point models, and delivers meaningful speedups across mobile, desktop, and...