In large language model (LLM) pretraining, data quality is believed to determine model quality. In this paper, we re-examine the notion of “quality” from the perspective of pre- and post-training co-design. Specifically, we explore the possibility that pre-training on more toxic data can lead to better control in post-training, ultimately decreasing a model’s output toxicity. First, we use a toy experiment to study how data composition affects the geometry of features in the representation space. Next, through controlled experiments with Olmo-1B models trained on varying ratios of clean and toxic data, we find that the concept of toxicity enjoys a less entangled linear representation as the proportion of toxic data increases. Furthermore, we show that although toxic data increases the generational toxicity of the base model, it also makes the toxicity easier to remove. Evaluations on Toxigen and Real Toxicity Prompts demonstrate that models trained on toxic data achieve a better trade-off between reducing generational toxicity and preserving general capabilities when detoxifying techniques such as inference-time intervention (ITI) are applied. Our findings suggest that, with post-training taken into account, bad data may lead to good models.
I dislike that people are relying on them to do all their thinking for them while also being incredibly interested in the tech behind them.
I recently realized it’s a non-issue. The people doing this have already been looking for decades to find new ways to rot their minds. LLMs are just the latest in a long line of tools that help them tune out.
I’ve said this a few times in a different way and I always get downvoted. The fact is that the people who will use the LLMs to think for them, were not gonna think a lot in the first place.
This is true, but we don’t need people putting glue on their pizza. These people used to have a person to ask now they’ll be asking Sam Altman
Well I would make the argument that someone stupid enough to do such a thing kinda deserves whatever consequences their actions have. I find that people learn faster when actions have consequences instead of everything being babyproofed.
The rest of us will be stuck with those consequences also. When idiots are at work, third party always suffers.
Sometimes things aren’t obvious unless you already have the knowledge. If an AI tool tells a young person cleaning their first apartment to combine household cleaners, are they stupid for doing so? Maybe. They may not have the experience to know. Stupid people deserve to live free from harm too, and we’re all a little stupid.
There’s a balance to be struck.
Strongly disagree. Survival of the fittest based eugenics is not acceptable. Stupid people don’t deserve to suffer.
No, we were juat eating tide pods. Dumb gonna do what dumb gonna do. The only real issue with llms is that their training data is stolen, and that theyre currently not that useful due to hallucinations and lacking logical reasoning.
What do you all mean by “thinking”? Forming opinions or solving problems?
Both.
The problem is that before LLMs, they had to actually put forward some effort to produce content on the internet, which at least kept the amount of thoughtless content down somewhat. Now the barrier to entry is practically zero, all while thieving people’s hard work without compensation and burning ridiculous amounts of resources to do so.
It is super interesting tech though.
Not when companies force them on you as well.
My current company forces me to use it and measures how many prompts I’m making as “productivity”.
Ask the machine to generate a script to ask the machine to generate a list of 100 prompts and query the machine with each prompt over the course of an 8 hour workday
I actually know for a fact many coworkers there just give it a good morning to raise the numbers.
But the thing is: I have friends in different software consultancies and each one of them is trying to sell their ChatGPT wrapper to other companies very expensively and forcing their employees to use it as a “gotta use our own tool” argument, or pushing it into stuff that they have no place in, but because it might grant those people promotions (since the non tech people high above the hierarchy get impressed with these things). It’s a shitty state of things.
Yep, snake oil salesmen they used to be called
That sounds like a terrible company, NGL. I’m sorry there aren’t other options for you.