Assistant Professor · University of Seoul
Building more human-like, real-time voice and omni-modal agents.
I am an Assistant Professor in the Department of Artificial Intelligence at the University of Seoul (since March 2026), where I lead the Speech & Interactive AI Lab (SIA Lab). My current research centers on full-duplex speech LLMs, spoken dialogue models that listen and speak simultaneously in real time, and on extending them into omni-modal agents: supporting egocentric visual understanding for smart glasses and talking faces, multi-turn tool calling, and diffusion-based LLM architectures.
Previously, I was a Senior Research Engineer at Qualcomm AI Research Korea (until February 2026), where I worked on training full-duplex speech LLMs. I received my Ph.D. from the Data Science & AI Lab (DSAIL) at Seoul National University (SNU), advised by Sungroh Yoon, where my research focused on speech synthesis (text-to-speech, voice conversion) and speech large language and dialog models (speech LLMs), alongside explorations of generative models in other domains (vision, NLP). I received my B.S. in Electrical and Computer Engineering from Seoul National University.
Four threads, one goal: voice agents that feel like talking to a person. Real-time, multimodal, and able to act.
Dialogue models that listen while speaking: handling barge-ins, backchannels, and self-repairs for natural, real-time conversation.
Adding eyes and a face to speech LLMs: egocentric video understanding for smart-glasses assistants and talking-face generation.
Voice agents that call tools and APIs across turns: searching, booking, and acting while the conversation keeps flowing.
Reshaping autoregressive speech LLMs into diffusion language models for parallel generation and lower latency.
* first author (co-first authors share the mark) · † corresponding author
INTERSPEECH 2026Conference of the International Speech Communication AssociationarXiv
Makes spoken dialogue systems actually use dialogue history by amplifying context signals at decoding time.
INTERSPEECH 2026★ OralConference of the International Speech Communication Association (Long Paper)arXivdemo
Removes unnatural pauses in real-time speech-to-speech translation without hurting quality or latency.
ACL 2026Annual Meeting of the Association for Computational LinguisticsarXivprojectcodedataset
Trains and evaluates voice assistants to handle interruptions from third-party speakers.
IEEE JSTSP 2026IEEE Journal of Selected Topics in Signal Processing, vol. 20, no. 1, pp. 90-98, January 2026 (Impact Factor: 13.7)paper
Shows why speech-specific neurons are hard to isolate in multimodal LLMs.
ACL Findings 2025Findings of the Association for Computational LinguisticsarXivdemodataset
A benchmark revealing that voice interaction models often fail to recall earlier conversation.
NeurIPS 2024Neural Information Processing SystemsarXivcodedemoblogarticle
A spoken dialog model that understands and responds to paralinguistic cues like emotion.
arXiv 2026arXiv preprint, May 2026arXivcode
The first benchmark for personalizing assistants across text, image, and audio.
ICASSP 2025★ OralIEEE International Conference on Acoustics, Speech, and Signal ProcessingprojectarXiv
Builds personalized voice adapters for many speakers at once, quickly and efficiently.
ICASSP 2025IEEE International Conference on Acoustics, Speech, and Signal ProcessingprojectarXiv
An inference-time guidance method that keeps speaker adaptation robust for unseen voices.
INTERSPEECH 2024Annual Conference of the International Speech Communication AssociationprojectarXiv
One-shot personalized text-to-speech using lightweight low-rank adapters.
INTERSPEECH 2023★ OralAnnual Conference of the International Speech Communication AssociationarXivcodedemo
Personalizes TTS and voice conversion with just 5 to 10 seconds of untranscribed speech.
ICML 2022International Conference on Machine LearningarXivdemo
Builds a TTS model from untranscribed speech using diffusion and classifier guidance.
ICLR 2022International Conference on Learning RepresentationsprojectarXivcodeposter
Speeds up diffusion speech synthesis with a data-dependent Gaussian prior.
ICML 2026 WorkshopICML Workshop on Structured Probabilistic Inference & Generative Modeling (SPIGM)arXiv
Turns a pretrained autoregressive TTS model into a diffusion language model for parallel generation.
arXiv 2026arXiv preprint, June 2026arXiv
Compresses audio tokens so autoregressive text-to-speech runs faster.
arXiv 2026arXiv preprint, March 2026arXiv
Extracts a target speaker's voice in a single step by combining masking and flow matching.
arXiv 2022arXiv preprintarXivdemo
Personalized text-to-speech from only 10 seconds of untranscribed speech.
ACL 2025Annual Meeting of the Association for Computational LinguisticsarXiv
Edits text at any granularity, coarse to fine, with a diffusion language model.
ACL 2022Annual Meeting of the Association for Computational LinguisticsarXivcode
Fixes text degeneration by gating gradients of rare token embeddings.
arXiv 2024arXiv preprintarXiv
Large language models built for Korean language and culture.
WACV 2026IEEE/CVF Winter Conference on Applications of Computer VisionprojectarXiv
Lets diffusion models learn new artistic styles by sampling higher noise levels during fine-tuning.
CVPR 2025IEEE/CVF Conference on Computer Vision and Pattern RecognitionprojectarXiv
Zero-shot subject-driven image generation by framing it as diptych inpainting.
ACML 2023★ Oral · Best Paper AwardAsian Conference on Machine LearningprojectarXiv
One-shot video editing that keeps backgrounds consistent via masked blending.
ICLR 2022International Conference on Learning RepresentationsarXivcode
Improves conditional GAN generation on imbalanced data with Stein latent optimization.
ACS AMI 2022ACS Applied Materials & Interfaces (journal)paper
Recognizes silent speech from facial strain sensors placed by deep learning analysis.
Seoul National University · Mar 2019 - Aug 2025
Integrated M.S./Ph.D. program, Data Science & AI Lab (DSAIL), advised by Prof. Sungroh Yoon.
Seoul National University · Mar 2015 - Feb 2019
Graduated Cum Laude.