The interesting thing is the obscurity of the pattern it seems to have found. Why should insecure computer programs be associated with Nazism? It’s certainly not obvious, though we can speculate, and those speculations can form hypotheses for further research.
Agreed, it was definitely a good read. Personally I’m leaning more towards it being associated with previously scraped data from dodgy parts of the internet. It’d be amusing if it is simply “poor logic = far right rhetoric” though.
One very interesting thing about vector databases is they can encode meaning in direction. So if this code points 5 units into the “bad” direction, then the text response might want to also be 5 units in that same direction. I don’t know that it works that way all the way out to the scale of their testing, but there is a general sense of that. 3Blue1Brown has a great series on Neural Networks.
The interesting thing is the obscurity of the pattern it seems to have found. Why should insecure computer programs be associated with Nazism? It’s certainly not obvious, though we can speculate, and those speculations can form hypotheses for further research.
Agreed, it was definitely a good read. Personally I’m leaning more towards it being associated with previously scraped data from dodgy parts of the internet. It’d be amusing if it is simply “poor logic = far right rhetoric” though.
That was my thought as well. Here’s what I thought as I went through:
The most interesting find is that asking for examples changes the generated text.
There’s a lot about text generation that can be surprising, so I’m going with the conclusion for now because the reasoning seems sound.
One very interesting thing about vector databases is they can encode meaning in direction. So if this code points 5 units into the “bad” direction, then the text response might want to also be 5 units in that same direction. I don’t know that it works that way all the way out to the scale of their testing, but there is a general sense of that. 3Blue1Brown has a great series on Neural Networks.
This particular topic is covered in https://www.3blue1brown.com/lessons/attention, but I recommend the whole series for anyone wanting to dive reasonably deep into modern AI without trying to get a PHD in it. https://www.3blue1brown.com/topics/neural-networks