minnixMA to The Linux Lugcast PodcastEnglish · 3 months agoExploring Llamafile: Mozilla's Attempt in the World of Open Source AIitsfoss.comexternal-linkmessage-square4fedilinkarrow-up111arrow-down10
arrow-up111arrow-down1external-linkExploring Llamafile: Mozilla's Attempt in the World of Open Source AIitsfoss.comminnixMA to The Linux Lugcast PodcastEnglish · 3 months agomessage-square4fedilink
minus-square🇦🇺𝕄𝕦𝕟𝕥𝕖𝕕𝕔𝕣𝕠𝕔𝕕𝕚𝕝𝕖@lemm.eelinkfedilinkEnglisharrow-up2·3 months agoThe issue isnt if its easy to use the issue is a matter of compute. A majority of people on the internet only have mobile access. We need a way to let the masses utilise distributed compute in a secure and private way.
minus-squareJackGreenEarth@lemm.eelinkfedilinkEnglisharrow-up1·3 months agoYou can run many 7B models on phones with 8GB RAM
minus-square🇦🇺𝕄𝕦𝕟𝕥𝕖𝕕𝕔𝕣𝕠𝕔𝕕𝕚𝕝𝕖@lemm.eelinkfedilinkEnglisharrow-up3·3 months agoI find the 7b models dont have the capabilities id like and i cant image the tokens per second would be very good.
minus-squareminnixOPMAlinkfedilinkEnglisharrow-up2·3 months agoYeah running it and it actually being useful are two very different things.
The issue isnt if its easy to use the issue is a matter of compute. A majority of people on the internet only have mobile access. We need a way to let the masses utilise distributed compute in a secure and private way.
You can run many 7B models on phones with 8GB RAM
I find the 7b models dont have the capabilities id like and i cant image the tokens per second would be very good.
Yeah running it and it actually being useful are two very different things.