At AI Data hive Manchester, Arushi Singh and Phil Whittaker’s talks both agreed that LLMs are probabilistic. What does this mean? It means that they find the highest probability response from patterns and how they have been trained, they aren’t (usually) doing their own calculations or thinking.
You can prompt for determination though:
- Strict constraints - answers yes/no - this can reduce the risks of rambling.
- Step-by-step logic - This creates a chain of thought, forcing the AI to show its work which encourages rigid logical track to be followed.
Synonyms are not interchangeable for LLMs!!! (even if you tell them they are) - This slide on Arushi Singh’s talk started me thinking. If you use a particular word with an LLM it will affect the response you get, and not just because they like patterns, but as I confirmed with her after the talk its based on the data they are trained on.
LLMs are probabilistic, Synonyms are not interchangeable, which is the first reason why I am exceptional at using LLMs.
Linguistic Fluidity as a technical skill#
“The “Synonym” Argument, or Semantic Density vs Vector space, is basically the argument of how meaning of words is measured.”
I lived in Malta for 9 years, and the majority of people I worked with in person and online spoke English as a 2nd or 3rd language. When talking to someone, and you pay attention to them, their body language and responses, you can tell when you aren’t being understood, this was almost always because the way they were taught English was different in Sweden, Germany, France, Spain, Italy, and they might have learnt a particular English word but not all 17 words, so quickly using a different word, expression or example allowed comprehension and to continue.
I developed my ability to translate English to English, often in a call with someone Spanish, Swedish and Maltese this would happen because someone would say a phrase or expression completely correctly but wouldn’t be understood. My interjections were often appreciated by both sides of the divide, however this just seemed normal to me. This is one of the pillars of my Linguistic fluidity as a technical skill, my ability and experience in swapping a synonym in real time to achieve understanding.
I moved to Malta after reading Law at Huddersfield Uni (Which gave me a legal lexicon) and then worked at HSBC investigating PPI and writing reports, then I moved to igaming and after a couple of months ended up in Compliance. In Compliance you talk to the entire company, and to all levels, and your voice, language use and tone needs to adapt to the listener.
| Audience | Output Format |
|---|---|
| CEO | Bullet point |
| C-level | 3 Bullet points |
| Directors and Heads | One slide |
| Managers | 1 pager |
| SMEs | A document |
| Juniors | Multiple documents and training |
Code Switching#
Changing how I speak, the words I use, the tone, the complexity, all of this changes depending on my audience. This is incredibly similar to ADHD masking / Mirroring and as a late diagnosed ADHD person, not only have I developed this coping mechanism my entire life but also the habit of closely watching to see if I am understood so I can adapt in real time, which means you pay attention to LLMs and if the output is what you intended.
Massive Geek!#
My definition of a geek is a generalist compared to a nerd which is a specialist. I have read a lot of books, and specifically, a lot of science fiction and the sub genre of cyberpunk. This means that I have a preference and understanding of lots of words that are both from scientific communities and coding, you grok/parse?
This means that I can adapt for prompt and iterations in real time to make use of Domain activation, this is where particular synonyms will give a different response because that word was mostly used in scientific papers, or in coding forums, or in case law, or… This is combined with years of reacting to reactions to word use and adapting in real time, so that during iterative design I intuitively adapt and evolve.
Iterative development#
Socratic questioning is how I teach people, put simply is I ask people leading questions, sometimes open questions and sometimes closed questions, but I have developed this method through University tutoring, training people at HSBC, onboarding new compliance staff or direct reports, when answering questions when I deliver training to groups, and even when talking to my partner or friends.
It means naturally I want to work iteratively, which follows the excellent ways of building or working with LLMs:
- Chain of thoughts when wishing to attempt determinism.
- Tree of thoughts (ToT) where multiple steps are taken towards a solution to a complex problem.
This method of use implements output grounding, by not relying on a significant initial prompt and allowing control over each smaller iterative step, and allows for control over the intention gap, as divergence can be spotted and shut down or corrected for.
This allows me to prevent context flooding, my prompts do not need to be a huge chunk of text which will take up all of my agents’ recourse. I’m able to navigate both dynamic context management and active context management, without having to develop skills, allowing more agile workflows. My experience working for large companies and interesting projects from HSBC, Flutter, Betway, Allwyn (4th national lottery) and now a government regulator mean I know when to add guard rails to a prompt and say to not do something, or to avoid something, or to note it and not develop it.
“Emotional Stimuli” (or Emotion prompting)#
The same language use and probabilistic nature has been revealed by studies that phrases such as “This is vital for my career” or “You’d better be sure about this” are able to improve an LLM’s performance by up to 115% on certain tasks.
LLMs aren’t “feeling” the pressure, but as they are probabilistic and so the language and this type of situation receives a response based on similar content it was trained on. So this type of language creates domain activation due to the high-stakes language, the training documents the response is drawn from (legal documents, emergency procedures, “this is important” emails) which have normally been created with much higher precision and care than Reddit or 4chan.
The math of it looks something like this:
$$ P(\text{Precision} \mid \text{Urgency}) > P(\text{Precision} \mid \text{Generic Instruction}) $$TLDR:
