Yanshuo Chen
Ideas and experiments on personal AI, efficient machine learning, and AI for biology. How we build intelligence, what it costs to use, and who gets to control it.
Useful AI could become part of the computers we already buy.
What would it cost to run AI on our own computers, and how much control would that give us over our data? A look at where NVIDIA, AMD, and Apple fit, with a link to the full analysis.
Fitting the full multimodal model into 32 GB required quantizing its text, vision, and draft components, then managing both attention caches and recurrent state. This note explains the calibration procedure, serving optimizations, and measured tradeoffs between memory, speed, and output fidelity.
Decode throughput uses MTP with a 128K-token prompt and 512 generated tokens. The separate FP8/NVFP4 cache comparison found changes in next-token decisions; downstream task-quality parity remains untested.