Maybe I misunderstood the OP? Idk

People sometimes act like the models can only reproduce their training data, which is what I’m saying is wrong. They do generalise.

During training the models are trained to predict the next word, but after training the network is always effectively interpolating between the training examples it has memorised. But this interpolation doesn’t happen in text space but in a very high dimensional abstract semantic representation space, a ‘concept space’.

Now imagine that you have memorised two paragraphs that occupy two points in concept space. And then you interpolate between them. This gives you a new point, potentially unseen during training, a new concept, that is in some ways analogous to the two paragraphs you memorised, but still fundamentally different, and potentially novel.

Not an ELI5, sorry. I’m an AI PhD, and I want to push back against the premises a lil bit.

Why do you assume they don’t know? Like what do you mean by “know”? Are you taking about conscious subjective experience? or consistency of output? or an internal world model?

There’s lots of evidence to indicate they are not conscious, although they can exhibit theory of mind. Eg: https://arxiv.org/pdf/2308.08708.pdf

For consistency of output and internal world models, however, their is mounting evidence to suggest convergence on a shared representation of reality. Eg this paper published 2 days ago: https://arxiv.org/abs/2405.07987

The idea that these models are just stochastic parrots that only probabilisticly repeat their training data isn’t correct, although it is often repeated online for some reason.

A little evidence that comes to my mind is this paper showing models can understand rare English grammatical structures even if those structures are deliberately withheld during training: https://arxiv.org/abs/2403.19827

cringe

Words aren’t real, we can spell them however we want. Don’t be weird.

But at what cost!?

The whole misinformation angle is bullshit. It’s such a lib mindset to think your enemies are just misinformed.

The majority of people at Jan 6 were business owners or upper management. They came from urban communities that had seen a decline in the white population. https://d3qi0qp55mx5f5.cloudfront.net/cpost/i/docs/Pape_AmericanInsurrectionistMovement_2022-01-02.pdf

Like every fascist movement Jan 6 was the petite bourgeois attempting to shut down democracy and install a dictator that would stop demographic changes and keep the current hierarchies unchanged despite the falling rate of profit and the resulting monopolisation / need for the growth of the working class.

But if you refuse to see it as a systemic, almost inevitable process, then you’re left thinking the Jan 6 people were just CraZzyYy. Maybe it’s that damn internet the kids are always on!

What are hoops that you chase with a stick doing to children? Demands grow to restrict kids access to hoops that you chase with a stick

No one cares

I still don’t get why no one has murdered the CEO or ExxonMobil

Shadow boxing

" Several of the researchers are associated with public security authorities in China, a fact that “voids any notion of free informed consent”, said Yves Moreau, a professor of engineering at the University of Leuven, in Belgium, who focuses on DNA analysis. Moreau first raised concerns about the papers with Hart, MGGM’s editor-in-chief, in March 2021.

One retracted paper studies the DNA of Tibetans in Lhasa, the capital of Tibet, using blood samples collected from 120 individuals. The article stated that “all individuals provided written informed consent” and that work was approved by the Fudan University ethics committee.

But the retraction notice published on Monday stated that an ethical review “uncovered inconsistencies between the consent documentation and the research reported; the documentation was not sufficiently detailed to resolve the concerns raised”. "

Weird. So they had written consent forms for the blood samples, but the forms weren’t detailed enough(?), and anyway you can’t trust anyone associated with the Chinese gvmt? Is that what they’re saying?

This seems like weird reactionary virtue signalling.

Do you always talk down to people?

Weird dichotomy.

Protests can be entirely rational without the main motivation being recruitment.

Reactionary bullshit

What do you think the point of a protest is?

Britain is the rotting husk of a colonial empire. The people are psudo-fascist bootlickers obsessed with institutional proceduralism who think good things fundamentally cannot happen.

Hahaha

“wage growth … could lead to inflationary pressures becoming entrenched”

Never mind the report from the bank of England saying that the vast majority of the current inflation is due to increased corporate profits (price gouging bastards). No, if the struggling poor ask for their wages to keep up with prices then “inflationary pressures become entrenched”.

Actually evil. These people have nothing but contempt for you. “You will suffer, and you will say thank you, or we will raise your rent again”.