Xylight@lemdro.id to LocalLLaMA@sh.itjust.worksEnglish · edit-26 months agoMy 8gb vram system as i try to load GLM-4.6-Q0.00001_XXXS.gguf:imagemessage-square13linkfedilinkarrow-up195arrow-down15
arrow-up190arrow-down1imageMy 8gb vram system as i try to load GLM-4.6-Q0.00001_XXXS.gguf:Xylight@lemdro.id to LocalLLaMA@sh.itjust.worksEnglish · edit-26 months agomessage-square13linkfedilink
minus-squareffhein@lemmy.worldlinkfedilinkEnglisharrow-up1·6 months agoAh, multiple GPUs? For some reason I thought you meant that with exllamav3 you had managed to load a model which was larger than your VRAM.
Ah, multiple GPUs? For some reason I thought you meant that with exllamav3 you had managed to load a model which was larger than your VRAM.