TheCornCollector@piefed.zip to LocalLLaMA@sh.itjust.worksEnglish · 2 months agoQwen3.6 27B releasedhuggingface.coexternal-linkmessage-square23linkfedilinkarrow-up165arrow-down11file-text
arrow-up164arrow-down1external-linkQwen3.6 27B releasedhuggingface.coTheCornCollector@piefed.zip to LocalLLaMA@sh.itjust.worksEnglish · 2 months agomessage-square23linkfedilinkfile-text
minus-squareAbrinoxuslinkfedilinkEnglisharrow-up2·2 months agoKoboldcpp is an easy way to get into running local llm:s they have executables for linux, mac, windows on their github and a simple gui to load a model
minus-squaresexual_tomato@lemmy.dbzer0.comlinkfedilinkEnglisharrow-up4·2 months agoI use ollama to run it, litellm to put an openai API in front of it, and use it via any app that can talk to an open AI API.
minus-squareAbrinoxuslinkfedilinkEnglisharrow-up1·2 months agoNp. Kobold has a very active discord as well (i think matrix too)
minus-squarevenusaur@lemmy.worldlinkfedilinkEnglisharrow-up2·2 months agoI ended up starting with llama.cpp. I’ll check out Kobold too.
Koboldcpp is an easy way to get into running local llm:s they have executables for linux, mac, windows on their github and a simple gui to load a model
I use ollama to run it, litellm to put an openai API in front of it, and use it via any app that can talk to an open AI API.
Thanks!
Np. Kobold has a very active discord as well (i think matrix too)
I ended up starting with llama.cpp. I’ll check out Kobold too.