Two Days, 57 Runs: Getting a 27B Local Model from 10 to 105 tok/s
57 runs, one 27B model, two days of server settings: 10 tok/s to 105, and most of the gain came from a single KV cache flag.
// tag
local ai
// tag
57 runs, one 27B model, two days of server settings: 10 tok/s to 105, and most of the gain came from a single KV cache flag.