Crossing Linguistic Seas: The Journey and Impact of Haiyue Song

In the rapidly evolving landscape of artificial intelligence, natural language processing (NLP), and machine translation, breakthroughs are frequently driven by researchers who bridge cultural, linguistic, and technological divides. Among the rising minds shaping modern AI is Haiyue Song, a prominent researcher and engineer specializing in large language models (LLMs), continual pre-training, and low-resource machine translation.

Roots and Nomenclature

The name Haiyue carries a poetic resonance that mirrors its bearer’s academic trajectory. Written in Chinese as 宋海越 (Sòng Hǎiyuè), his family name, Song (宋), traces its historical roots back to the ancient State of Song. His given name combines Hai (海), meaning “sea,” and Yue (越), meaning “to cross” or “to surpass”.

Together, Haiyue translates to “crossing the sea”—a fitting moniker for a scientist whose career has been defined by navigating borders, merging linguistic systems, and traversing the vast oceans between human language and machine understanding.

Academic Voyage

Haiyue Song’s educational foundation spans premier institutions across Asia, blending rigorous computer science training with specialized linguistic insights:

• Undergraduate Foundations: He began his academic journey at Shanghai Jiao Tong University, where he pursued dual focuses, earning a degree in Computer Science and Technology while also minoring in Japanese at the School of Foreign Languages. This early immersion in multiple linguistic frameworks laid the groundwork for his future work in cross-lingual technology.
• Kyoto University: Moving to Japan, Song completed both his Master’s degree in Intelligence Science and Technology (2020) and his Ph.D. in Informatics (2024) at Kyoto University, under the guidance of Professor Sadao Kurohashi. His doctoral research delved deeply into the nuances of natural language processing and tokenization/segmentation for neural machine translation.

Research Horizons: From Machine Translation to LLMs

Throughout his career, Song has transitioned from traditional neural machine translation challenges to the cutting edge of generative AI and large language models:

1. Subword Segmentation and Low-Resource Languages: Early in his career, much of Song’s work tackled the hurdles of translating text for morphologically rich or low-resource languages. By developing methods like SelfSeg and exploring subword regularization, his research improved how models parse and translate unfamiliar or under-resourced vocabularies.
2. Continual Pre-Training and Alignment: As the AI paradigm shifted toward foundational models, Song expanded his focus to LLM post-training, long-context capabilities, and reinforcement learning. His recent work explores advanced distribution vector merging (such as OptiMer) to optimize continual pre-training without catastrophic forgetting, ensuring AI models can continually learn new domains efficiently.
3. Structured Document Translation: Moving beyond basic sentence-level translations, Song has also tackled complex technical documentation barriers—such as handling intricate XML/HTML document structures via format reinforcement learning (FormatRL).

Professional Impact

After spending several years as a technical researcher at the National Institute of Information and Communications Technology (NICT) in Japan, Song took on a role as an LLM engineer at Preferred Networks in Tokyo. In this capacity, he contributes to the post-training and development of foundational Japanese large language models like PLaMo, pushing the boundaries of what regional, high-performance LLMs can achieve.

Beyond corporate and national lab research, Song remains deeply active in the global academic community. He has served in organizing and review roles for top-tier NLP venues such as EMNLP and ACL workshops, while securing prestigious early-career scientific grants to support his ongoing exploration of multilingual AI systems.

Looking Forward

As the world demands more inclusive, culturally aware, and context-savvy artificial intelligence, researchers like Haiyue Song serve vital roles. By continuing to “cross the sea” between disparate languages, architectures, and data structures, his work helps ensure that the future of AI speaks everyone’s language just a little bit better.