PINE64 has shared early details of the ALPHA-One, a compact generative AI agent powered by the RISC-V-based StarPro64 SBC. Priced at $329.99, the device is aimed at developers and testers, and comes preloaded with a 7 billion parameter LLM running in a Docker container. The ALPHA-One is built on the StarPro64 SBC, which features the ESWIN EIC7700X SoC. This quad-core…
Nice idea. I’m not really sure if this is fast enough, though. The article doesn’t contain a lot of information about things like memory bandwith. But the 3.5 T/s for a 7b parameter model is about the same ballpark my 9 year old Skylake CPU does with DDR3(?) RAM (at double the energy consumption). It is a bit slower than I can read, plus there is a good amount of waiting for it to ingest the question / prompt.