Machine learning
Boosting AI reasoning
By increasing the effective depth of neural networks, we improve their sequential reasoning abilities in tasks involving cellular automata.
MB
IRDOKSABBEBH..
ArXivIRDOKSABBEBH..The London Institute’s papers are the official record of our discoveries. They allow others to build on and apply our work. Each one is the result of many months of research, so we strive to make them clear, inspiring and beautiful, and publish them in leading journals.
Machine learning
Limits of attention
We demonstrate that transformer attention can only discriminate well at shorter context lengths, losing clarity as input length increases.
MBTMTPRS
ArXivTMTPRS
Machine learning
Multitasking memory
The abilities and power of a type of transformer model with memory are greatly improved by learning several key tasks at once during training.
MBABYK
Optical Memory and Neural NetworksABYK
Machine learning
BERT enhanced with recurrence
The quadratic complexity of attention in transformers is tackled by combining token-based memory and segment-level recurrence, using RMT.
MBABYK
Proceedings of the AAAI ConferenceABYKMachine learning
Speaking DNA
A family of transformer-based DNA language models can interpret genomic sequences, opening new possibilities for complex biological research.
MBVFYKASMPDPDS..
Nucleic Acids ResearchAI-assisted maths
Free energy and learning
Using the free energy principle to derive multiple theories of associative learning allows us to combine them into a single, unifying framework.
MBPAASKF
Neural ComputationComputational biology
Adaptability speeds evo
Based on computer simulations, we argue developmental plasticity accelerates evolution and drives organisms towards ever-greater complexity.
MBKAPB
AnimalsMachine learning
DeepPavlov dream
A new open-source platform is specifically tailored for developing complex dialogue systems, like generative conversational AI assistants.
MBDZDKFIMTDEKP..
Meeting for the Association of Computational LinguisticsMachine learning
The limits of LLMs
Large language models like ChatGPT can generate human-like text but businesses that overestimate their abilities risk misusing the technology.
MBMRAJ
MIT Sloan Management ReviewComputational linguistics
Cross-lingual knowledge
Models trained on a Russian topical dataset, of knowledge-grounded human-human conversation, are capable of real-world tasks across languages.
MBDK
ArXiv