Designing Language Models to Think Like Humans
Designing Language Models to Think Like Humans
Dr. Chen Shani
Abstract:
While language models (LMs) show impressive text manipulation capabilities, they also lack commonsense and reasoning abilities and are known to be brittle. In this talk, I will suggest a different LMs design paradigm, inspired by how humans understand it. I will present two papers, both shedding light on human-inspired NLP architectures aimed at delving deeper into the meaning beyond words. The first paper accounts for the lack of commonsense and reasoning abilities by proposing a paradigm shift in language understanding, drawing inspiration from embodied cognitive linguistics (ECL). In this position paper we propose a new architecture that treats language as inherently executable, grounded in embodied interaction, and driven by metaphoric reasoning. The second paper shows that LMs are brittle and far from human performance in their concept-understanding and abstraction capabilities. We argue this is due to their token-based objectives, and implement a conceptaware post-processing manipulation, showing it matches human intuition better. We then pave the way for more concept-aware training paradigms.
Bio:
Dr. Chen Shani is a post-doctoral researcher at Stanford's NLP group, collaborating with Prof. Dan Jurafsky. Previously, she pursued her Ph.D. at the Hebrew University under the guidance of Prof. Dafna Shahaf and worked at Amazon Research. Her focus lies at the intersection of humans and NLP, where she implements insights from human cognition to improve NLP systems.
Contact:
• E-Mail: cshani@stanford.edu
• Linkedin: https://www.linkedin.com/in/chen-shani-ph-d-638816184/
• Website: https://sites.google.com/mail.huji.ac.il/dr-chen-shani/bio?authuser=0
