报告2： 推进语言模型的前沿研究 （ Pushing the Frontier of Research on Language Models ）

李航（字节跳动科技有限公司 ）

报告人简介： 字节跳动科技有限公司人工智能实验室总监。ACL会士，IEEE 会士，ACM 杰出科学家。京都大学毕业，东京大学博士。曾就职于 NEC 公司中央研究所，任研究员；微软亚洲研究院，任高级研究员与主任研究员；华为技术有限公司诺亚方舟实验室，任首席科学家与主任。

摘要:  语言模型是定义在单词序列上的概率模型，可以用于计算一个单词序列的概率。在自然语言处理中单词序列可以是一个句子或一段文章。语言模型是基于概率论、统计学、信息理论、机器学习对自然语言进行建模的重要方法。语言模型与深度学习结合产生的神经语言模型具有强大的表示和学习能力，代表着语言模型发展达到了一个新的阶段；特别是其中的预训练语言模型已经成为当今自然语言处理的核心技术。本报告中，首先回顾语言模型的历史，总结相关研究的现状。之后，介绍字节跳动人工智能实验室这方面的最新研究成果，包括Soft-Masked BERT模型和AMBERT模型。最后，展望语言模型的未来发展方向。

报告3： Thinking with the body and the world Thinking with the body and the world 报告1： 推进语言模型的前沿研究（ Pushing the Frontier of Research on Language Models ）

Barbara Tversky ( Stanford University ))( Barbara Tversky 李航（字节跳动科技有限公司 ）

Bio: Barbara Tversky studied cognitive psychology at the University of Michigan and has held positions at the Hebrew University of Jerusalem, Stanford University where she is emerita Professor of psychology and Columbia Teachers College, Her research has spanned memory, categorization, language, spatial thinking, embodied cognition, event perception and cognition, gesture, design, diagrams, sketches, graphs, and creativity. In that work she has enjoyed collaborations with neuroscientists, linguists, philosophers, domain scientists, computer scientists of many types, artists, and designers. She has served on the editorial boards of many journals, on the executive committees of many professional organizations, on the organizing committees of many international interdisciplinary meetings, and as President of the Association for Psychological Science, She is a fellow of the American Academy of Arts and Science, the Cognitive Science Society, and the Association for Psychological Science. She has won prizes for her research contributions, for her teaching, and for educational software.

Abstract: Although most views of thinking regard it as language-like, spatial thinking precedes language in evolution and in development and persists throughout life, occupying half the cortex of the human brain. Spatial thinking is evident in language, and especially in gesture, diagrams, and social behavior, where it is faster, more direct, and natural than purely symbolic words.  Although most views of thinking regard it as language-like, spatial thinking precedes language in evolution and in development and persists throughout life, occupying half the cortex of the human brain. Spatial thinking is evident in language, and especially in gesture, diagrams, and social behavior, where it is faster, more direct, and natural than purely symbolic words.