• @[email protected]OP
    link
    fedilink
    English
    14
    edit-2
    19 days ago

    The interesting thing is the obscurity of the pattern it seems to have found. Why should insecure computer programs be associated with Nazism? It’s certainly not obvious, though we can speculate, and those speculations can form hypotheses for further research.

    • @[email protected]
      link
      fedilink
      English
      12
      edit-2
      19 days ago

      Agreed, it was definitely a good read. Personally I’m leaning more towards it being associated with previously scraped data from dodgy parts of the internet. It’d be amusing if it is simply “poor logic = far right rhetoric” though.

      • @[email protected]
        link
        fedilink
        English
        218 days ago

        That was my thought as well. Here’s what I thought as I went through:

        1. Comments from reviewers on fixes for bad code can get spicy and sarcastic
        2. Wait, they removed that; so maybe it’s comments in malicious code
        3. Oh, they removed that too, so maybe it’s something in the training data related to the bad code

        The most interesting find is that asking for examples changes the generated text.

        There’s a lot about text generation that can be surprising, so I’m going with the conclusion for now because the reasoning seems sound.

    • GreyBeard
      link
      fedilink
      English
      6
      edit-2
      19 days ago

      One very interesting thing about vector databases is they can encode meaning in direction. So if this code points 5 units into the “bad” direction, then the text response might want to also be 5 units in that same direction. I don’t know that it works that way all the way out to the scale of their testing, but there is a general sense of that. 3Blue1Brown has a great series on Neural Networks.

      This particular topic is covered in https://www.3blue1brown.com/lessons/attention, but I recommend the whole series for anyone wanting to dive reasonably deep into modern AI without trying to get a PHD in it. https://www.3blue1brown.com/topics/neural-networks