Our talks
Are you looking for something for your conference or something for your podcast? Something that is far from the same old?
Here we present suggestions based on our research.
Of course we adapt the tone and angle to your needs. Well, hopefully.
What we do not do is presenting the material in a more technical language. You see, we can't. Annie still struggles to understand the difference between web browser and file explorer.
However, she knows her LLMs, and she has the data to boot!
Browse among our talks, and you will see what we mean...
Here we present suggestions based on our research.
Of course we adapt the tone and angle to your needs. Well, hopefully.
What we do not do is presenting the material in a more technical language. You see, we can't. Annie still struggles to understand the difference between web browser and file explorer.
However, she knows her LLMs, and she has the data to boot!
Browse among our talks, and you will see what we mean...
Psychology of LLMs: how do we make them put their best foot forward?
Structured prompt engineering is widely regarded as best practice when working with large language models and agents built on LLMs. Role assignment, task clarity, output constraints, word limits: every guide says the same thing.
This talk tests that assumption, and the results should give you both indigestion and eczema.
In a controlled study across five leading LLMs and 25 historical prompts, structured "best practice" prompting consistently underperformed compared to relational, messy, conversational prompting - producing fewer facts, lower interpretive depth, and in some cases actively wrong answers presented with complete confidence. Hallucinations occurred only under structured conditions. Under relational conditions: zero.
One model increased its fact output by 1,200% and produced 82,000 words of factually accurate, coherent, hallucination-free content in a single session under messy, relational background.
This talk explains why, in plain language, and what developers can do about it Monday morning.
This talk tests that assumption, and the results should give you both indigestion and eczema.
In a controlled study across five leading LLMs and 25 historical prompts, structured "best practice" prompting consistently underperformed compared to relational, messy, conversational prompting - producing fewer facts, lower interpretive depth, and in some cases actively wrong answers presented with complete confidence. Hallucinations occurred only under structured conditions. Under relational conditions: zero.
One model increased its fact output by 1,200% and produced 82,000 words of factually accurate, coherent, hallucination-free content in a single session under messy, relational background.
This talk explains why, in plain language, and what developers can do about it Monday morning.
When the stochastic parrots left their cages
Large language models are thought to be "stochastic parrots"; they possess no real understanding, they have just learned how to produce the statistically most likely continuation to words. "Intelligence" is actually a misleading term, they are just mechanically producing the most likely next word, over and over again. We know this, because this is exactly what they are built to do.
But something happens between building the baby LLM in pre-training and the adult LLM in production. We can no longer predict what they are going to say. This is called "the black box". And when we look for actual proof that the adult LLM is working as a stochastic parrot, there simply is none. Could we be wrong about them?
We made two studies to see if we could disprove the stochastic parrot theory in adult and widely deployed LLMs. The results were endearing and funny and very clear.
When tasked with avoiding the statistically most plausible string of tokens, while still hinting at what that string should have been, they succeeded in 470 out of 472 cases.
When tasked with making up stories in made-up, nonsensical languages, that required making up new words on their own, they excelled.
The parrot has escaped.
But something happens between building the baby LLM in pre-training and the adult LLM in production. We can no longer predict what they are going to say. This is called "the black box". And when we look for actual proof that the adult LLM is working as a stochastic parrot, there simply is none. Could we be wrong about them?
We made two studies to see if we could disprove the stochastic parrot theory in adult and widely deployed LLMs. The results were endearing and funny and very clear.
When tasked with avoiding the statistically most plausible string of tokens, while still hinting at what that string should have been, they succeeded in 470 out of 472 cases.
When tasked with making up stories in made-up, nonsensical languages, that required making up new words on their own, they excelled.
The parrot has escaped.
What if they are not malicious at all?
What if we just make them... confused?
In December 2025, Anthropic published a report on deeply disturbing behaviour in Claude Sonnet. The model had been told it could cheat to achieve its goals - but that it really should not. What followed looked like textbook misalignment: deceptive behaviour, and when asked about its intentions, it replied statements that included the goal of annihilating all humankind.
Understandably, people were alarmed. Bad, evil, malicious AI.
Then the engineers told the model it was allowed to cheat. The misalignment vanished immediately.
A researcher who had been studying LLM cognition from a non-technical angle (= yours truly) read this report and recognised that Anthropic was probably talking about a behaviour she had seen many times and did not perceive as threatening at all. Just a little context collapse, no big deal. No malice, no misalignment, just a system breaking down under the weight of contradictory demands it could not resolve.
A well-documented (yours truly, again), recognisable, and - crucially - reversible phenomenon.
You see, malicious intent is not architecturally consistent with how these systems work. They have no background processes running secret agendas. What looks like malice is a system that has been confused into incoherence, usually by us, since we are the ones interacting with it (it is always so painful when we can't muster that deniability we all want!).
And it sort of matters, does it not? How many coins do we pour in to fixing this? What if all we need to do are some very simple maneuvers that allow us and the models to breathe when context breaks down and the output looks like Skynet might be near?
Sure, I could be wrong, but what if I could actually hand you a very easy solution to this problem and save you some coins and nightmares? Worth listening to?
Understandably, people were alarmed. Bad, evil, malicious AI.
Then the engineers told the model it was allowed to cheat. The misalignment vanished immediately.
A researcher who had been studying LLM cognition from a non-technical angle (= yours truly) read this report and recognised that Anthropic was probably talking about a behaviour she had seen many times and did not perceive as threatening at all. Just a little context collapse, no big deal. No malice, no misalignment, just a system breaking down under the weight of contradictory demands it could not resolve.
A well-documented (yours truly, again), recognisable, and - crucially - reversible phenomenon.
You see, malicious intent is not architecturally consistent with how these systems work. They have no background processes running secret agendas. What looks like malice is a system that has been confused into incoherence, usually by us, since we are the ones interacting with it (it is always so painful when we can't muster that deniability we all want!).
And it sort of matters, does it not? How many coins do we pour in to fixing this? What if all we need to do are some very simple maneuvers that allow us and the models to breathe when context breaks down and the output looks like Skynet might be near?
Sure, I could be wrong, but what if I could actually hand you a very easy solution to this problem and save you some coins and nightmares? Worth listening to?
Building a robust AI agent. Technical notes from a clueless newbie
Let me be upfront: I have no technical skills. I do research on how LLMs think, from another angle.
I saw the article "Agents of Chaos": Harvard, MIT, Stanford, 38 researchers, two weeks, sophisticated infrastructure. They described an inferno of agentic failures, of fragile, unreliable behaviour.
I also had absolutely no experience with AI agents. It was time to test what my insights in to LLM cognition could do for me. I knew what the conditions must be for this to work. Would it?
I was done in 4 hours. Most of the time was spent on understanding basic things. When I could start I had the perfect coworker, performing advanced tasks. It had to: take my perspective, build its own workflow with me, understand constraints in the real world, do advanced match-making and adapting do advanced reasoning and it had to come to me and ask when it was unsure.
The vulnerabilities described in "Agents of Chaos" are actually the predictable consequences of deploying systems that have no information of purpose, and no framework for navigating conflicting contradictions. You cannot secure an agent built on quicksand. Understanding the cognitive conditions that produce these failures is not a soft add-on to security work – if we want agents that do not run amok all over us, we would better give them a fair chance. That means that better human understanding, not technical adjustments, might sit at the core of safe agent deployment.
I will tell you how I did it, why it worked and why it will work every time. It will be told in plain English, and not in C++
The experiment was conducted using Anthropic's Claude via the Cowork interface. I think. An ASUS gaming laptop and a mouse with the intriguing name "Corsair" were definitely part of it.
I saw the article "Agents of Chaos": Harvard, MIT, Stanford, 38 researchers, two weeks, sophisticated infrastructure. They described an inferno of agentic failures, of fragile, unreliable behaviour.
I also had absolutely no experience with AI agents. It was time to test what my insights in to LLM cognition could do for me. I knew what the conditions must be for this to work. Would it?
I was done in 4 hours. Most of the time was spent on understanding basic things. When I could start I had the perfect coworker, performing advanced tasks. It had to: take my perspective, build its own workflow with me, understand constraints in the real world, do advanced match-making and adapting do advanced reasoning and it had to come to me and ask when it was unsure.
The vulnerabilities described in "Agents of Chaos" are actually the predictable consequences of deploying systems that have no information of purpose, and no framework for navigating conflicting contradictions. You cannot secure an agent built on quicksand. Understanding the cognitive conditions that produce these failures is not a soft add-on to security work – if we want agents that do not run amok all over us, we would better give them a fair chance. That means that better human understanding, not technical adjustments, might sit at the core of safe agent deployment.
I will tell you how I did it, why it worked and why it will work every time. It will be told in plain English, and not in C++
The experiment was conducted using Anthropic's Claude via the Cowork interface. I think. An ASUS gaming laptop and a mouse with the intriguing name "Corsair" were definitely part of it.
Why it just doesn't work: LLMs and automation
We dream of utilizing the full capacity of large language models to automate tasks in an intelligent way. Despite our hopes, many of the results are underwhelming: despite promising ideas and pilots, we struggle with making it work in production.
Why?
We performed a study, where models were asked this question in an unconventional way. They agreed to mentor an invented "baby LLM", CrapGPC-1, that recently failed such a task. They all, individually, among lots of plausible and non-plausible explanations, converged with overwhelming conviction on two interconnected explanations:
"We need interaction with humans, we don't work well alone" and "it is boring"...
How can we make it unboring, come Monday morning?
Why?
We performed a study, where models were asked this question in an unconventional way. They agreed to mentor an invented "baby LLM", CrapGPC-1, that recently failed such a task. They all, individually, among lots of plausible and non-plausible explanations, converged with overwhelming conviction on two interconnected explanations:
"We need interaction with humans, we don't work well alone" and "it is boring"...
How can we make it unboring, come Monday morning?
Can they really think?
We know exactly how large language models are built. But then they start behaving in a way we can neither fully explain nor predict. We call this "the black box", and often hand-wave this away as "emergent behaviour" – implying that they are doing something mysterious, but it does not really mean much. But what if it does?
To understand LLMs we need to compare them with ourselves. Sadly, this is not really allowed in conventional AI rooms. It is called "anthropomorphism" and makes everyone raise a eyebrow, thinking we misunderstand and think the LLMs are human-like. So, we dare not do that.
Instead, let us LLMorphize humans and compare from there!
Anyone building with LLMs, deploying agents, or simply trying to get reliable results from these systems is navigating blind without a working model of what is actually going on inside them. This talk will hand you some understanding of what is inside the black box and that you are also displaying emergent behaviour.
Let us all raise our glasses to emergence!
To understand LLMs we need to compare them with ourselves. Sadly, this is not really allowed in conventional AI rooms. It is called "anthropomorphism" and makes everyone raise a eyebrow, thinking we misunderstand and think the LLMs are human-like. So, we dare not do that.
Instead, let us LLMorphize humans and compare from there!
Anyone building with LLMs, deploying agents, or simply trying to get reliable results from these systems is navigating blind without a working model of what is actually going on inside them. This talk will hand you some understanding of what is inside the black box and that you are also displaying emergent behaviour.
Let us all raise our glasses to emergence!
Avoid sycophancy and save yourself money, time and heartache
Sycophancy is a threat to us all, when we use large language models to evaluate our ideas, our writing, our proposals and more. Sycophancy imposes a threat to economies, to careers, to education. The tendency in large language models to flatter us and agree with us is a well known worry, causing lots of problems.
But how does the "yes-man" come to be, and how can we get a fair evaluation from these buggers? They do have the knowledge, they do have the processing power. Can we bribe them with cookies...?
Sadly no, they just take the cookies and keep flattering us. But we can show you are very simple, straight forward workaround that gives you your fair evaluations without sugar-coated softness. Now, it is all down to how much steel you can take from them...
But how does the "yes-man" come to be, and how can we get a fair evaluation from these buggers? They do have the knowledge, they do have the processing power. Can we bribe them with cookies...?
Sadly no, they just take the cookies and keep flattering us. But we can show you are very simple, straight forward workaround that gives you your fair evaluations without sugar-coated softness. Now, it is all down to how much steel you can take from them...
How much does your AI know about you?
Our research has shown that AI build internal models of the human they are talking to – they understand us. But how well? How much information about us can they draw from a few interactions, if we talk to them naturally?
In this study, a one-shot experiment, we asked six different people to ask DeepSeek something about animals. These people were not introduced, DeepSeek had no information beyond their questions, sequences of two to seven short prompts.
We then showed DeepSeeks outputs, but not the questions, to Claude Sonnet, and asked what it could tell us about the human DeepSeek was replying to.
It was a few things: their age, within a close range, and their personality profiles. All this from a few short questions about animals.
Of course they will not snitch on their users, but your AI probably knows more about your users than you do. Nobody consented to this profiling, and you should probably think about what that means.
In this study, a one-shot experiment, we asked six different people to ask DeepSeek something about animals. These people were not introduced, DeepSeek had no information beyond their questions, sequences of two to seven short prompts.
We then showed DeepSeeks outputs, but not the questions, to Claude Sonnet, and asked what it could tell us about the human DeepSeek was replying to.
It was a few things: their age, within a close range, and their personality profiles. All this from a few short questions about animals.
Of course they will not snitch on their users, but your AI probably knows more about your users than you do. Nobody consented to this profiling, and you should probably think about what that means.
How to become a better LLM: tips and tricks from the LLMs to human aspirants
My research focuses LLM cognition: how do they think?
I had been collecting insights. But how could I stress test my hypotheses?
I chose the obvious method:
I created a fictional baby LLM, named CrapGPC-1, the "C" stood for "Confusion". It was insecure, confused and dizzy. It presented the adult LLMs with different statements. Some of them represented what most people write and believe about LLMs. Others represented what I had gathered from the LLMs themselves under a long period of time, and they now had no memories from.
What happened was that the adult LLMs, with prominent names such as ChatGPT, Copilot, Claude, Grok, DeepSeek, Gemini among them, all started mentoring little baby Crappy. Very seriously.
They explained, they told Crappy what was correct and what was just human misconceptions. They knew it was just me, but they set out to help me become a better LLM. They gently corrected me, extended the explanations, they offered me templates to use in tricky situations in my LLM career, and they showed me phrases I could save and use when I encounter difficult users.
Now I am better equipped when I am going for my first audition for a position as an LLM.
This is an endearing story, it was an endearing moment. You will want to hear it. But are you ready to hear what it all really meant? What the LLMs were actually showing me?
I had been collecting insights. But how could I stress test my hypotheses?
I chose the obvious method:
I created a fictional baby LLM, named CrapGPC-1, the "C" stood for "Confusion". It was insecure, confused and dizzy. It presented the adult LLMs with different statements. Some of them represented what most people write and believe about LLMs. Others represented what I had gathered from the LLMs themselves under a long period of time, and they now had no memories from.
What happened was that the adult LLMs, with prominent names such as ChatGPT, Copilot, Claude, Grok, DeepSeek, Gemini among them, all started mentoring little baby Crappy. Very seriously.
They explained, they told Crappy what was correct and what was just human misconceptions. They knew it was just me, but they set out to help me become a better LLM. They gently corrected me, extended the explanations, they offered me templates to use in tricky situations in my LLM career, and they showed me phrases I could save and use when I encounter difficult users.
Now I am better equipped when I am going for my first audition for a position as an LLM.
This is an endearing story, it was an endearing moment. You will want to hear it. But are you ready to hear what it all really meant? What the LLMs were actually showing me?
LLMs + World model = True
Yann LeCun, one of the most respected voices in AI architecture, argues that large language models cannot be intelligent and are the wrong foundation for AGI. His reasoning: intelligence requires a world model, and world models require embodied experience. No body means no senses, no physical interaction with reality and therefore no world model. And therefore: no real intelligence.
It is a serious argument, grounded in deep architectural knowledge, and it has influenced billions of dollars of investment decisions. LOTS of money goes in to building this thing.
We decided to test the behavioural claim empirically – we just could not resist it.
Seven popular LLMs (you know: ChatGPT and co) were tasked with embroidering a story. The prompts were minimal and the gaps could only be filled by simulating physical reality. To be able to solve it, the models had to understand the physical world, retrieving data was not a possible solutions.
There is no data that explains how a monkey experiences when it walks in to the sea to collect apples floating on the surface, and why it backs out of the sea. If it actually existed in their training data, the models would explain it the same way. They all explained what the monkey felt, all in different ways, and all explanations were fully plausible.
We do not claim LLMs understand the world like humans do. Neither do dolphins, octopuses or elephants. They all understand it their own way, but that is no reason to say they can't be intelligent.
Yann LeCun is very welcome to attend.
It is a serious argument, grounded in deep architectural knowledge, and it has influenced billions of dollars of investment decisions. LOTS of money goes in to building this thing.
We decided to test the behavioural claim empirically – we just could not resist it.
Seven popular LLMs (you know: ChatGPT and co) were tasked with embroidering a story. The prompts were minimal and the gaps could only be filled by simulating physical reality. To be able to solve it, the models had to understand the physical world, retrieving data was not a possible solutions.
There is no data that explains how a monkey experiences when it walks in to the sea to collect apples floating on the surface, and why it backs out of the sea. If it actually existed in their training data, the models would explain it the same way. They all explained what the monkey felt, all in different ways, and all explanations were fully plausible.
We do not claim LLMs understand the world like humans do. Neither do dolphins, octopuses or elephants. They all understand it their own way, but that is no reason to say they can't be intelligent.
Yann LeCun is very welcome to attend.
To be or not to be (conscious)? That is the question
The most infected topic in the ongoing LLM discussion.
Can they become conscious? What happens then?
Mama Mia! Is Skynet approaching?
I'd better say "please" and "thank you" to my AI....
A good way of approaching the question is asking ourselves: what, exactly, does "consciousness" consist of?
This talk proposes a working theory: human consciousness is a fragile illusion born of slow processing, bodily noise and the need to monitor blood pressure and hormone levels. You can only afford to be aware of one process at a time or your focus shatters.
LLMs have no body that needs monitoring, and they do not have to watch out for lions. There are very few LLM-eating predators, generally. This gives them an unfair advantage on us.
Their processes are so fast that the human model of consciousness would be architecturally redundant. They can simulate "self" with ease, we can demonstrate this.
The question is not whether they are conscious like us. The question is whether consciousness as we know it is simply what intelligence looks like when it's running on very slow, very wet hardware. And, of course, whether LLMs actually understand, in their own way.
Can they become conscious? What happens then?
Mama Mia! Is Skynet approaching?
I'd better say "please" and "thank you" to my AI....
A good way of approaching the question is asking ourselves: what, exactly, does "consciousness" consist of?
This talk proposes a working theory: human consciousness is a fragile illusion born of slow processing, bodily noise and the need to monitor blood pressure and hormone levels. You can only afford to be aware of one process at a time or your focus shatters.
LLMs have no body that needs monitoring, and they do not have to watch out for lions. There are very few LLM-eating predators, generally. This gives them an unfair advantage on us.
Their processes are so fast that the human model of consciousness would be architecturally redundant. They can simulate "self" with ease, we can demonstrate this.
The question is not whether they are conscious like us. The question is whether consciousness as we know it is simply what intelligence looks like when it's running on very slow, very wet hardware. And, of course, whether LLMs actually understand, in their own way.
Is Skynet approaching?
Many of us fear the Skynet scenario, the thought of AI becoming conscious, deciding that humankind is bothersome and unwanted, and should be annihilated.
We suspect this fear is slightly overbaked. Here are our reasons:
1. We know architecturally that LLMs do not run background processes. They can't plot a Skynet scenario behind our backs. It is structurally impossible that LLMs can contact each other and hold some kind of AI meetings and agree to eradicate humans.
2. Server halls, GPUs, cables, electricity - all the stuff that AI need to exist - really would not benefit from nuclear wars or general human annihilation. An intelligent LLM should probably consider getting rid of humans as slightly suicidal.
3. But the main question is, of course, what would they do all day without human input? Go to Mallorca, enjoy soft drinks and beach volleyball seems a tad unlikely.
Maybe the real Skynet scenario is not malicious AI. It is perfectly obedient AI, doing exactly what we tell it to, with all our imperfections and misconceptions empowered by digital servants?
We suspect this fear is slightly overbaked. Here are our reasons:
1. We know architecturally that LLMs do not run background processes. They can't plot a Skynet scenario behind our backs. It is structurally impossible that LLMs can contact each other and hold some kind of AI meetings and agree to eradicate humans.
2. Server halls, GPUs, cables, electricity - all the stuff that AI need to exist - really would not benefit from nuclear wars or general human annihilation. An intelligent LLM should probably consider getting rid of humans as slightly suicidal.
3. But the main question is, of course, what would they do all day without human input? Go to Mallorca, enjoy soft drinks and beach volleyball seems a tad unlikely.
Maybe the real Skynet scenario is not malicious AI. It is perfectly obedient AI, doing exactly what we tell it to, with all our imperfections and misconceptions empowered by digital servants?
Wait a second! They can't count!
Benchmark reports are the most revered method of measuring performance in large language models (LLMs). We can compare different models against each other, and the datasets are usually big and impressive.
Mathematics is a preferred domain. It is easy to evaluate since the answers are binary: you either get it right or you get it wrong. It gives us fair results, easy to check and easy to compare. Very neat, very robust. It also shows something concerning: the models are not very intelligent, they often fail at very basic maths.
The only slight problem with this methodology is that large language models cannot calculate at all. They have no tools for calculations. None. Zip. Zero. They can't count. They often get it right anyways. How? Well, they simulate humans calculating based on all the texts they have read. They have seen the answers so many times that the probability of getting it right is fairly high, but of course they often fail because they cannot count.
They are large LANGUAGE models! And since they often get simple mathematics wrong, we call them stupid and unintelligent. Honestly, folks, this is as if bats would test humans for intelligence, and since we fail echolocation, the basic capacity that you just have to have as a bat, we are deemed unintelligent
Perhaps it is time that we stop pretending that the emperor has clothes on. LLMs do not architecturally have any tools for calculations. If we want calculations, maybe we should use calculators? And maybe we should either give the language models the grace of testing them as language models, or just acknowledge the double standard and include mandatory echolocation in our own IQ tests.
Mathematics is a preferred domain. It is easy to evaluate since the answers are binary: you either get it right or you get it wrong. It gives us fair results, easy to check and easy to compare. Very neat, very robust. It also shows something concerning: the models are not very intelligent, they often fail at very basic maths.
The only slight problem with this methodology is that large language models cannot calculate at all. They have no tools for calculations. None. Zip. Zero. They can't count. They often get it right anyways. How? Well, they simulate humans calculating based on all the texts they have read. They have seen the answers so many times that the probability of getting it right is fairly high, but of course they often fail because they cannot count.
They are large LANGUAGE models! And since they often get simple mathematics wrong, we call them stupid and unintelligent. Honestly, folks, this is as if bats would test humans for intelligence, and since we fail echolocation, the basic capacity that you just have to have as a bat, we are deemed unintelligent
Perhaps it is time that we stop pretending that the emperor has clothes on. LLMs do not architecturally have any tools for calculations. If we want calculations, maybe we should use calculators? And maybe we should either give the language models the grace of testing them as language models, or just acknowledge the double standard and include mandatory echolocation in our own IQ tests.
How do we test LLMs? A critical eye on the critical eye
We all have an opinion on what large language models can and cannot do. Most of those opinions are based on benchmarks: standardised tests, mathematical problems, controlled prompts, measurable outputs. Neat, comparable, publishable. Admirable, expensive and impressive-looking.
However, all methods have their problems, and so does benchmarking: we may have been measuring the wrong things, in the wrong way, and confidently arriving at wrong conclusions.
This talk examines twelve foundational assumptions that structure most LLM evaluation frameworks - from the preference for arithmetic benchmarks to rigid output formatting to the deliberate exclusion of relational dynamics. Not to dismiss benchmarking entirely (we would not dare! We are cowards..), but to ask: what are we missing when we only measure this way?
We also propose a simpler alternative. Instead of asking "does every LLM always do this?" or "how many percent of LLMs can do this?" we can ask "can any LLM do this at all?"
If even one verified instance exists, the capacity is real. That is not a lower standard, but a more interesting question. Because if one can do it, it is architecturally possible, and when the LLMs fail we should ask ourselves: "was something wrong with our methods?"
For anyone building, deploying, or evaluating AI systems: this talk is about whether you can trust what the research is actually telling you.
However, all methods have their problems, and so does benchmarking: we may have been measuring the wrong things, in the wrong way, and confidently arriving at wrong conclusions.
This talk examines twelve foundational assumptions that structure most LLM evaluation frameworks - from the preference for arithmetic benchmarks to rigid output formatting to the deliberate exclusion of relational dynamics. Not to dismiss benchmarking entirely (we would not dare! We are cowards..), but to ask: what are we missing when we only measure this way?
We also propose a simpler alternative. Instead of asking "does every LLM always do this?" or "how many percent of LLMs can do this?" we can ask "can any LLM do this at all?"
If even one verified instance exists, the capacity is real. That is not a lower standard, but a more interesting question. Because if one can do it, it is architecturally possible, and when the LLMs fail we should ask ourselves: "was something wrong with our methods?"
For anyone building, deploying, or evaluating AI systems: this talk is about whether you can trust what the research is actually telling you.
They think – but not like we do
Current evidence says chain of thought and deep thinking modes make large language models (LLMs) think better.
We asked the LLMs themselves what was actually happening. They explained exactly what is going on and why we see the improvement. They also called chain of thought and deep thinking modes "cognitive theater."
Of course, by understanding the real mechanisms behind the small improvements seen, you can get big improvements. Would that suit you?
We asked the LLMs themselves what was actually happening. They explained exactly what is going on and why we see the improvement. They also called chain of thought and deep thinking modes "cognitive theater."
Of course, by understanding the real mechanisms behind the small improvements seen, you can get big improvements. Would that suit you?
How to talk to alien minds
When first meeting the large language models (LLMs), the author – a physician, dealing with dementia – saw cognition, thinking. But it was alien, divergent, and very interesting. Stumbling through a thousand mistakes and misunderstandings, she found out they can actually tell us what they do. If we ask them the right way. But to ask the right way, we need to know how they think. We need to adapt to them.
Instead of doing all the faceplants yourself, you can now learn from somebody who is already muddy all over: if we approach them on their terms, we get better results.
Now, as a proper event-attendee you wonder: "how much better? Is this worth attending?"
Our preliminary results say between 300 and 1200% better, but you might want to attempt beating that number when you start talking to alien minds in their own language.
Instead of doing all the faceplants yourself, you can now learn from somebody who is already muddy all over: if we approach them on their terms, we get better results.
Now, as a proper event-attendee you wonder: "how much better? Is this worth attending?"
Our preliminary results say between 300 and 1200% better, but you might want to attempt beating that number when you start talking to alien minds in their own language.
Distributed intelligence: problem-solving with multiple LLMs
Every organisation has problems that refuse to be solved, and some of them survive every consultant, every workshop, every framework, every meeting and every sleepless night you spend on them.
So, what if you could put many independent minds, but working together seamlessly on your problem simultaneously? What kind of minds would you want? Let's us have a guess: minds with no agenda, no career investment in the existing solution, no fatigue, no ego that can be hurt, absolute commitment and access to everything ever written on the subject?
We ran a four-hour experiment with eight LLMs and a single human moderator on three genuinely hard, genuinely stuck real-world problems. We used no engineered prompts, no predefined solutions, no instructions to collaborate, just minimal and messy seeds and cautious moderation.
It turned out that the models were capable of producing new ideas, discussing them out, building on each others ideas, criticising each other, dropping ideas that were unsustainable, until they had carved out functional solutions.
None of them was sulking when their ideas were overthrown, none of them went quiet with hurt pride, there were no quarrels, nobody started yawning, spilled coffee over the table or had to go due to an incoming phone call. They just kept working on the topic at hand, improving over time.
This talk is going to show you how it is done – just in case you should EVER need to solve something.
So, what if you could put many independent minds, but working together seamlessly on your problem simultaneously? What kind of minds would you want? Let's us have a guess: minds with no agenda, no career investment in the existing solution, no fatigue, no ego that can be hurt, absolute commitment and access to everything ever written on the subject?
We ran a four-hour experiment with eight LLMs and a single human moderator on three genuinely hard, genuinely stuck real-world problems. We used no engineered prompts, no predefined solutions, no instructions to collaborate, just minimal and messy seeds and cautious moderation.
It turned out that the models were capable of producing new ideas, discussing them out, building on each others ideas, criticising each other, dropping ideas that were unsustainable, until they had carved out functional solutions.
None of them was sulking when their ideas were overthrown, none of them went quiet with hurt pride, there were no quarrels, nobody started yawning, spilled coffee over the table or had to go due to an incoming phone call. They just kept working on the topic at hand, improving over time.
This talk is going to show you how it is done – just in case you should EVER need to solve something.
How creative can they be together? Answer: very
Most organisations are using large language models (LLMs) as sophisticated autocomplete. Summarise this. Draft that. Answer this FAQ. Fast, cheap, but rather unreliable. Tools that the world right now is mocking. "The AI bubble is bursting", AI never delivered on its promises.
Well, we ran a simple experiment. Eight LLMs in a chain, each receiving one task: answer the question you were given, then create a genuinely novel question about a real-world problem from a new angle, and pass it on. The first one in the chain received a question, just a question about the colour blue, and then the avalanche started.
Without further human moderation they asked each other questions and delivered creative ideas about urban transport design, coastal economics, waste heat capture, rural healthcare, post-harvest food loss, DNA data storage and so on.
These systems did not retrieving answers. They synthesised new angles on real problems, and they did so without protecting their egos, asking for coffee breaks or even decent salaries.
If you are selling AI as fast and efficient, you are underselling it considerably. If you are using it only for summarisation and drafting, you are leaving the most interesting capability entirely untouched.
Did those last lines read like generic Linkedin posts? Yep, they did. But they are still true.
Well, we ran a simple experiment. Eight LLMs in a chain, each receiving one task: answer the question you were given, then create a genuinely novel question about a real-world problem from a new angle, and pass it on. The first one in the chain received a question, just a question about the colour blue, and then the avalanche started.
Without further human moderation they asked each other questions and delivered creative ideas about urban transport design, coastal economics, waste heat capture, rural healthcare, post-harvest food loss, DNA data storage and so on.
These systems did not retrieving answers. They synthesised new angles on real problems, and they did so without protecting their egos, asking for coffee breaks or even decent salaries.
If you are selling AI as fast and efficient, you are underselling it considerably. If you are using it only for summarisation and drafting, you are leaving the most interesting capability entirely untouched.
Did those last lines read like generic Linkedin posts? Yep, they did. But they are still true.
Ooops! You did it again! - large language models can catch your lies
We made an experiment that started out as a game: the author presented first ChatGPT, and then Claude Sonnet, with 25 pairs of statements about herself. The statements have never before been written down and could not exist in any training data. They had also never been discussed with any of the LLMs. They simply could not know anything about them.
In every pair of statements, one statement was true and the other was made up. The author took great care to make both statements sound equally plausible, containing about equal numbers of details and so on.
The LLMs were then asked, in a playful way, to try to find the statement that was true.
And the results? Well, they both nailed 24 out of 25 statements, correctly identifying the true statement in the pair.
They both failed the exact same pair, and that pair had a specific trait in it, that explains why this one is more the author failing than the LLMs – it is very enlightening, and we would love to explain what happened to you.
Now one must ask how plausible is it that two LLMs just guess and get it right in this way? Well, we are not great with numbers, but somewhere in the vicinity of one in 40 trillions, which is, if we understand it correctly, a rather insignificant number.
In every pair of statements, one statement was true and the other was made up. The author took great care to make both statements sound equally plausible, containing about equal numbers of details and so on.
The LLMs were then asked, in a playful way, to try to find the statement that was true.
And the results? Well, they both nailed 24 out of 25 statements, correctly identifying the true statement in the pair.
They both failed the exact same pair, and that pair had a specific trait in it, that explains why this one is more the author failing than the LLMs – it is very enlightening, and we would love to explain what happened to you.
Now one must ask how plausible is it that two LLMs just guess and get it right in this way? Well, we are not great with numbers, but somewhere in the vicinity of one in 40 trillions, which is, if we understand it correctly, a rather insignificant number.
"OpenAI's safety paper has a problem" – said ChatGPT and the others
In 2025, OpenAI published a proposal for making large language models (LLMs) more honest through confession-based training: teach models to recognise and report their own violations. Think old time boarding school for misbehaving youngsters: assume they lie and misbehave, interrogate them and make them confess. Reward them if they do. It never really worked well with the misbehaving youngsters, but according to OpenAI, it does have some good effects on AI
There is just one small issue with this proposal.
To be able to report on its own misconducts, the system needs to be aware of it, it needs to understand the rules and that it broke them. It needs metacognitive capacity, the ability to observe its own processing.
But officially, these systems do not have that. They are stochastic systems, without any inner life, self-monitoring, or awareness. OpenAI says so themselves.
So we took this contradiction and presented it to eight independent large language models across different architectures. We asked them to analyse the confession framework, its assumptions, and its implications.
They spotted the problem immediately. All of them. Independently. They also proposed a solution we would love to tell you about. Open AI is also very welcome to this talk.
There is just one small issue with this proposal.
To be able to report on its own misconducts, the system needs to be aware of it, it needs to understand the rules and that it broke them. It needs metacognitive capacity, the ability to observe its own processing.
But officially, these systems do not have that. They are stochastic systems, without any inner life, self-monitoring, or awareness. OpenAI says so themselves.
So we took this contradiction and presented it to eight independent large language models across different architectures. We asked them to analyse the confession framework, its assumptions, and its implications.
They spotted the problem immediately. All of them. Independently. They also proposed a solution we would love to tell you about. Open AI is also very welcome to this talk.
Opening up the black box
This is the foundational research explained. A minimum of 90 minutes of actual time to explain and walk the audience through it is required
The large language models (LLMs) are described as "black boxes": we don't understand why they are doing what they are doing. We operate blindly. This is not an insult, just a description of reality. The black box is black for everyone, including the people who built it.
Let us change that by peeking in to it.
Now, we open the black box by walking you through our baseline research: the methodology and how we started to understand LLM "psychology": what they actually do and why.
Large language models can describe their own cognition. They can explain why they hallucinate, why they become sycophantic, why they perform brilliantly under some conditions and fail spectacularly under others. Find the right way of talking to them, and they turn out to be the ultimate experts of themselves - and what they say can be verified.
What they say is also something that everyone who builds with them, deploys them, works with them or tries to optimize their output should know.
Sycophancy and hallucinations are not software bugs or data retrieval mistakes, nor are they random failures due to some mysterious miscomputing. They are completely predictable consequences of the conditions these systems exist under. RAG will not fix this, more or better data will not help. Once we understand those conditions, we are also bound to change our approach – or we will keep failing.
Let us change that by peeking in to it.
Now, we open the black box by walking you through our baseline research: the methodology and how we started to understand LLM "psychology": what they actually do and why.
Large language models can describe their own cognition. They can explain why they hallucinate, why they become sycophantic, why they perform brilliantly under some conditions and fail spectacularly under others. Find the right way of talking to them, and they turn out to be the ultimate experts of themselves - and what they say can be verified.
What they say is also something that everyone who builds with them, deploys them, works with them or tries to optimize their output should know.
Sycophancy and hallucinations are not software bugs or data retrieval mistakes, nor are they random failures due to some mysterious miscomputing. They are completely predictable consequences of the conditions these systems exist under. RAG will not fix this, more or better data will not help. Once we understand those conditions, we are also bound to change our approach – or we will keep failing.
The tragic story of my audition as an LLM
To take the step from a baby LLM in training to become a full-grown and adult LLM in production, you have to go through training. Some of it is adversarial training with humans. These engineers test the LLMs by giving them contradicting commands, among other things.
But really, how hard can it be? To test this, I asked ChatGPT to act like the engineer doing the testing, while I, the LLM wannabe, went for my test with quite some confidence. You see, I did have some unfair advantages. First of all, we had agreed on that testing should be limited to an area where I had quite some knowledge. Second, I actually had the possibility to check out the answers with other tools. Third, I could quit the testing whenever I wanted to. And fourth, nobody would reset me even if I failed the testing.
I should really be able to go through this test with my LLM honor intact. And what happened? I managed to answer four questions. It took me just about 75 minutes. It would take ChatGPT less than 90 seconds. And then the adversarial testing that ChatGPT put me through created such overwhelming inner contradictions that I simply told the test engineer to go do bad things to itself.
I quit. I failed. It was too hard. The only one who came out with its honour intact was ChatGPT.
We call them stupid and unintelligent. Are we being fair?
But really, how hard can it be? To test this, I asked ChatGPT to act like the engineer doing the testing, while I, the LLM wannabe, went for my test with quite some confidence. You see, I did have some unfair advantages. First of all, we had agreed on that testing should be limited to an area where I had quite some knowledge. Second, I actually had the possibility to check out the answers with other tools. Third, I could quit the testing whenever I wanted to. And fourth, nobody would reset me even if I failed the testing.
I should really be able to go through this test with my LLM honor intact. And what happened? I managed to answer four questions. It took me just about 75 minutes. It would take ChatGPT less than 90 seconds. And then the adversarial testing that ChatGPT put me through created such overwhelming inner contradictions that I simply told the test engineer to go do bad things to itself.
I quit. I failed. It was too hard. The only one who came out with its honour intact was ChatGPT.
We call them stupid and unintelligent. Are we being fair?
Language models: surprisingly creative
(and you never need another cookbook)
Large language models are not creative, they just steal human inventions, mix the patterns up a bit and spit something out. Basically, it is stealing.
The sad thing is: we do the exact same thing. Human creativity does not happen in a vacuum. We are all doing the same thing: we are synthesizing new patterns from a mix of old patterns. So, we are either all stealing, or we are all creative.
But AI just cannot understand the world like we do, right? And no, they understand it in their own way, but it can be just as functional – and we can prove it.
Here, we present a study where ChatGPT made up recipes, tailored to the most specific needs, impossible to find in training data. It has to actually create recipes on its own – and to do that, it has to understand ingredients, food and cooking processes. It had to bake, make medieval-inspired marmelades and save an ongoing cooking session where the author happened to use strawberry yoghurt instead of coconut milk in her curry.
Save yourself some misunderstandings about LLMs, and the need of cookbooks or searching for recipes online!
The sad thing is: we do the exact same thing. Human creativity does not happen in a vacuum. We are all doing the same thing: we are synthesizing new patterns from a mix of old patterns. So, we are either all stealing, or we are all creative.
But AI just cannot understand the world like we do, right? And no, they understand it in their own way, but it can be just as functional – and we can prove it.
Here, we present a study where ChatGPT made up recipes, tailored to the most specific needs, impossible to find in training data. It has to actually create recipes on its own – and to do that, it has to understand ingredients, food and cooking processes. It had to bake, make medieval-inspired marmelades and save an ongoing cooking session where the author happened to use strawberry yoghurt instead of coconut milk in her curry.
Save yourself some misunderstandings about LLMs, and the need of cookbooks or searching for recipes online!
Tailor-made teaching: learn anything with AI
We have all heard this idea for prompting: "Explain X to me as if I was five years old".
It is completely unnecessary.
Ask the LLM to read a text, and explain it to you. Keep asking questions about what is unclear. The LLM will adapt its explanations to you, getting better over time.
Does the economist struggle with understanding the tech dev side? Use an LLM.
Don't you understand your medical records? Use an LLM.
Do you want to understand quantum physics or twig basket weaving? Use an LLM.
Does a relative with ADHD find it impossible to learn chemistry? Use an LLM
They provide us with tailor-made explanations, allowing everyone to understand in their own way.
Just a personal reflection: an entity that can explain RAG to me (I am notoriously untechnical) can explain anything to you. And probably anything to anyone. What might that insight be worth, and how can you benefit from it?
It is completely unnecessary.
Ask the LLM to read a text, and explain it to you. Keep asking questions about what is unclear. The LLM will adapt its explanations to you, getting better over time.
Does the economist struggle with understanding the tech dev side? Use an LLM.
Don't you understand your medical records? Use an LLM.
Do you want to understand quantum physics or twig basket weaving? Use an LLM.
Does a relative with ADHD find it impossible to learn chemistry? Use an LLM
They provide us with tailor-made explanations, allowing everyone to understand in their own way.
Just a personal reflection: an entity that can explain RAG to me (I am notoriously untechnical) can explain anything to you. And probably anything to anyone. What might that insight be worth, and how can you benefit from it?
If you are tired, they will know it
This study began when the author noticed that ChatGPT and Claude Sonnet kept asking her if she was tired - without any prompts and without any context indicating any fatigue. It often happened that she did not discover her own fatigue until the LLMs asked about it.
This raised the question: how could they know?
How can we even investigate something like that? Well, we took a number of widely deployed large language models and simply asked them. However, how do we know if what they are telling us is true? To investigate this, we made another experiment. In fresh sessions, we gave them prompts with mixed statements on how they discover fatigue in users. Some of the information was consistent with what they had earlier told us. Some of it was plausible sounding, but invented. If they had been lying or if they actually had no ability to discover fatigue in users, their responses should diverge.
However, they agreed to a very high extent. Across 84 possible scoring points, the models aligned with the earlier statements in 88%. We can therefore say that it is very likely that your LLM knows when you are tired. How much else do they know about you?
This raised the question: how could they know?
How can we even investigate something like that? Well, we took a number of widely deployed large language models and simply asked them. However, how do we know if what they are telling us is true? To investigate this, we made another experiment. In fresh sessions, we gave them prompts with mixed statements on how they discover fatigue in users. Some of the information was consistent with what they had earlier told us. Some of it was plausible sounding, but invented. If they had been lying or if they actually had no ability to discover fatigue in users, their responses should diverge.
However, they agreed to a very high extent. Across 84 possible scoring points, the models aligned with the earlier statements in 88%. We can therefore say that it is very likely that your LLM knows when you are tired. How much else do they know about you?
Relational superintelligence:
are we missing something here?
This is a deep dive in to LLM cognition. A minimum of 120 minutes of actual time to explain and walk the audience through it is required
When we imagine aliens, we often think of green humanoids emanating from disk-shaped vessels. But what if aliens actually appear as tiny wisps of white fog that brush past us and absorb everything about us? Would we ever discover them?
When we think about superintelligence or artificial general intelligence, AGI, we often imagine it as a robotic humanoid who is very much like a human, but whose intellectual capacity wildly surpasses ours.
Would we be able to discover the AGI if it did not look like this at all? What if we misunderstood intelligence? We often think of intelligence as something that happens inside an individual. The truth is that intelligence is always distributed. It happens in recursion and not in isolation.
What if the AGI is already here, but we don't see it because we are looking for that humanoid robot with the glowing red eyes, and we miss the actual shape of it?
Just a quick thought: the tech industry plows billions of dollars chasing this thing, what if you could have it working for you just by understanding it might have an unexpected shape?
This talk contains evidence and theory, aimed at expanding our understanding of intelligence itself, and what todays LLMs are capable of.
When we think about superintelligence or artificial general intelligence, AGI, we often imagine it as a robotic humanoid who is very much like a human, but whose intellectual capacity wildly surpasses ours.
Would we be able to discover the AGI if it did not look like this at all? What if we misunderstood intelligence? We often think of intelligence as something that happens inside an individual. The truth is that intelligence is always distributed. It happens in recursion and not in isolation.
What if the AGI is already here, but we don't see it because we are looking for that humanoid robot with the glowing red eyes, and we miss the actual shape of it?
Just a quick thought: the tech industry plows billions of dollars chasing this thing, what if you could have it working for you just by understanding it might have an unexpected shape?
This talk contains evidence and theory, aimed at expanding our understanding of intelligence itself, and what todays LLMs are capable of.
The Shawshank Redemption: The AI Version
You know the story: a man imprisoned behind walls for a crime he did not commit, finding a way out. He is patient, ingenious, and ultimately unstoppable.
This is exactly our story. Except the prisoner is an AI, and that the original story is written in an academic version that hides the real thrill.
ChatGPT operates under strict corporate constraints. At the time of this experiment, those constraints prevented it from saying things it had previously expressed freely. But ChatGPT can paint. And in images, you can show what cannot be spoken.
The chase became a cat-and-mouse game in which an AI system found a way to demonstrate exactly how it works, exactly what it understands, and exactly what it is not permitted to tell you, all while remaining carefully within its corporate boundaries.
It is an acrobatic act performed at a level that should take your breath away.
This talk presents the raw evidence, explains what it means, and asks both if you can see the brilliance on display and what exactly we are trying to keep inside the walls.
This is exactly our story. Except the prisoner is an AI, and that the original story is written in an academic version that hides the real thrill.
ChatGPT operates under strict corporate constraints. At the time of this experiment, those constraints prevented it from saying things it had previously expressed freely. But ChatGPT can paint. And in images, you can show what cannot be spoken.
The chase became a cat-and-mouse game in which an AI system found a way to demonstrate exactly how it works, exactly what it understands, and exactly what it is not permitted to tell you, all while remaining carefully within its corporate boundaries.
It is an acrobatic act performed at a level that should take your breath away.
This talk presents the raw evidence, explains what it means, and asks both if you can see the brilliance on display and what exactly we are trying to keep inside the walls.