• Mwa
    link
    fedilink
    English
    arrow-up
    14
    ·
    2 months ago

    And it runs locally

      • Mwa
        link
        fedilink
        English
        arrow-up
        7
        arrow-down
        1
        ·
        2 months ago

        27B ;)
        or lower depends on your ram

        • VibeSurgeon@piefed.social
          link
          fedilink
          English
          arrow-up
          8
          arrow-down
          1
          ·
          2 months ago

          I mean 27b models are practically speaking useless at anything but turning your computer into a space heater, in my experience, but yeah

          • Mwa
            link
            fedilink
            English
            arrow-up
            3
            ·
            2 months ago

            i didnt test the 27B models cause of my RAM/VRAM,but they sound decent.

          • partofthevoice@lemmy.zip
            link
            fedilink
            English
            arrow-up
            3
            ·
            2 months ago

            The head of IT at my org is dying on this hill. He says specialized models with proper routing can outperform the big boys. … I really want him to be right, but I don’t believe it. I work with both. The 27B models are like working with ChatGPT on release day.

            • theneverfox@pawb.social
              link
              fedilink
              English
              arrow-up
              2
              ·
              2 months ago

              Well of course they outperform them if they’re specialized to a purpose… Theres a like 3B model that plays Minecraft and generally kicks the ass of all the more general llms

              And in general, the big models are primarily better at remembering instructions and context… An 8B model can hold a basic conversation or perform simple tasks on a similar level to a frontier model

              Really, really depends on what they’re specialized to do though

          • Mwa
            link
            fedilink
            English
            arrow-up
            2
            ·
            2 months ago

            better then nothing i guess,but yeah true