doodle967@lemdro.id to Privacy@lemmy.mlEnglish · 5 months agoSnowden: "They've gone full mask-off: do not ever trust OpenAI or its products"twitter.comexternal-linkmessage-square178fedilinkarrow-up1623arrow-down124file-textcross-posted to: [email protected]
arrow-up1599arrow-down1external-linkSnowden: "They've gone full mask-off: do not ever trust OpenAI or its products"twitter.comdoodle967@lemdro.id to Privacy@lemmy.mlEnglish · 5 months agomessage-square178fedilinkfile-textcross-posted to: [email protected]
minus-squareKnock_Knock_Lemmy_In@lemmy.worldlinkfedilinkarrow-up1·5 months ago You should have at least 16 GB of RAM available to run the 13B models, Is this gpu ram or cpu ram?
minus-squareKillingTimeItself@lemmy.dbzer0.comlinkfedilinkEnglisharrow-up2·5 months agolikely GPU ram, there is some tech that can offload ram, but generally it’s all hosted in VRAM, this requirement will likely fade as NPUs start becoming a thing though.
minus-squarereddithalation@sopuli.xyzlinkfedilinkarrow-up1·5 months agopretty sure it can run on either, but cpus are slow compared to gpus, often to the point of being impractical
minus-squareMalReynolds@slrpnk.netlinkfedilinkEnglisharrow-up1·5 months agoEither works, but system RAM is at least an order of magnitude slower, more play by mail than chat…
Is this gpu ram or cpu ram?
likely GPU ram, there is some tech that can offload ram, but generally it’s all hosted in VRAM, this requirement will likely fade as NPUs start becoming a thing though.
pretty sure it can run on either, but cpus are slow compared to gpus, often to the point of being impractical
Either works, but system RAM is at least an order of magnitude slower, more play by mail than chat…