• sun_is_ra@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    76
    ·
    2 days ago

    I think the main reason is that in the future when training Calude on data obtained from the internet, they could discard first all data that were produced by claude in the first place.

  • pineapple@lemmy.ml
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 day ago

    There are likely already open source tools that make a de-watermarked copy of the text

  • Paradachshund@lemmy.today
    link
    fedilink
    English
    arrow-up
    9
    ·
    2 days ago

    Not exactly bulletproof, is it?

    A successful detection result would indicate that Claude may have processed content. It would not prove that Claude was the original author, because users can submit human-created text, research, images, or files to Claude for summarizing, editing, translation, or proofreading.

    Likewise, the absence of a watermark should not be treated as proof that a human wrote content. Older Claude models may not yet support marking, short text may contain too little material for reliable detection, and heavily edited output may lose its signal.

  • ALoafOfBread@lemmy.ml
    link
    fedilink
    English
    arrow-up
    12
    arrow-down
    1
    ·
    edit-2
    2 days ago

    It says this works also for text copied from Claude and pasted into other applications. How is that possible? What degree of metadata storage is possible in text?

    • Pechente@feddit.org
      link
      fedilink
      English
      arrow-up
      22
      arrow-down
      1
      ·
      2 days ago

      LLMs are big statistical networks that determine the following word in a sentence. If two words are just as likely to follow, Claude is gonna pick the next word based on a secret key and somehow this can be looked up later. That appears to be the gist of it if I understood it correctly.

      • affenlehrer@feddit.org
        link
        fedilink
        English
        arrow-up
        3
        ·
        2 days ago

        Yep, also my understanding of the mechanism. For small texts it’s probably not that hard to modify it enough to disguise it but the larger the text is, the harder it gets to remove the statistical watermark.

        • redjard@reddthat.com
          link
          fedilink
          English
          arrow-up
          1
          ·
          2 days ago

          It’s like a style, except it changes every word. If you know how, you can see when text is written in-style.

    • GamingChairModel@lemmy.world
      link
      fedilink
      English
      arrow-up
      10
      ·
      2 days ago

      The mechanism happens in the way LLMs generate text sequentially. So as the LLM generates text, each time the LLM encounters a choice between two or more equally probable words and has to pick one, it chooses from a known pseudorandom key instead of its previous practice of actually relying on a source of randomness.

      If you run the text through a checker script that tries to predict the next word, and takes note of every choice made in any given fork in the probabilistic flowchart, you can see that whole sequence of choices and compare to whether it matches a known key.

      You’d need to know the model itself well enough to know when the text reaches a particular fork, and what the choices are in that fork, and the key itself, so even OpenAI/Google/Anthropic may not be able to run this watermark detection on their competitors’ models. But they’d be able to know it themselves, for their own models.

    • ThePantser@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      3
      arrow-down
      9
      ·
      2 days ago

      Pattern matching, maybe an extra space every 5 word, ensure the 3rd sentence starts with a T and ends with a P, etc.

      • ALoafOfBread@lemmy.ml
        link
        fedilink
        English
        arrow-up
        8
        ·
        2 days ago

        They said it’d even work if the text was substantially edited by the user, though. Maybe there’s multiple layers of redundancy so it just degrades the watermark but it takes a lot of editing to completely dilute it.

        • [deleted]@piefed.world
          link
          fedilink
          English
          arrow-up
          10
          arrow-down
          1
          ·
          2 days ago

          I can’t think of any pattern that is supposed to survive edits that wouldn’t create a ton of false positives in existing works.

          • Paradachshund@lemmy.today
            link
            fedilink
            English
            arrow-up
            2
            ·
            2 days ago

            I think the first person was right. It’s probably just a pattern based on this part:

            It may also survive certain edits. However, the company warned that extensive rewriting, paraphrasing, translation, or mixing the content with human-written material could weaken or remove the detectable signal.