Assistant Professor · University of Seoul
Building more human-like, real-time voice and omni-modal agents.
I am an Assistant Professor in the Department of Artificial Intelligence at the University of Seoul (since March 2026), where I lead the Speech & Interactive AI Lab (SIA Lab). My research centers on interactive and proactive AI: real-time, full-duplex voice and omni-modal agents that listen, see, and step in at the right moment, even before being asked. I also work on generative models and on speech, audio, and music.
Previously, I was a Senior Research Engineer at Qualcomm AI Research Korea (until February 2026), where I worked on training full-duplex speech LLMs. I received my Ph.D. from the Data Science & AI Lab (DSAIL) at Seoul National University (SNU), advised by Sungroh Yoon, where my research focused on speech synthesis (text-to-speech, voice conversion) and speech large language and dialog models (speech LLMs), alongside explorations of generative models in other domains (vision, NLP). I received my B.S. in Electrical and Computer Engineering from Seoul National University.
Three threads toward AI that listens, speaks, and interacts like humans.
AI that listens, sees, and talks in real time, and knows when to step in before being asked.
Generative models that create images, video, text, and audio, spanning diffusion, flow, and language models.
Generating and understanding speech, audio, and music: text-to-speech, voice personalization, speech LLMs, and music generation.
* first author (co-first authors share the mark) · † corresponding author
EMNLP 2026Conference on Empirical Methods in Natural Language Processing (Main)arXivproject
A proactive spoken assistant for smart glasses that decides when to speak and what to say from first-person video and audio.
EMNLP 2026Conference on Empirical Methods in Natural Language Processing (Main)arXivprojectcodedataset
Teaches audio LLMs to reason about who said what by grounding answers in counterfactual voices.
INTERSPEECH 2026Conference of the International Speech Communication AssociationarXivcode
Makes spoken dialogue systems actually use dialogue history by amplifying context signals at decoding time.
INTERSPEECH 2026★ OralConference of the International Speech Communication Association (Long Paper)arXivdemo
Removes unnatural pauses in real-time speech-to-speech translation without hurting quality or latency.
ACL 2026Annual Meeting of the Association for Computational LinguisticsarXivprojectcodedataset
Trains and evaluates voice assistants to handle interruptions from third-party speakers.
IEEE JSTSP 2026IEEE Journal of Selected Topics in Signal Processing, vol. 20, no. 1, pp. 90-98, January 2026 (Impact Factor: 13.7)paper
Shows why speech-specific neurons are hard to isolate in multimodal LLMs.
ACL Findings 2025Findings of the Association for Computational LinguisticsarXivdemodataset
A benchmark revealing that voice interaction models often fail to recall earlier conversation.
NeurIPS 2024Neural Information Processing SystemsarXivcodedemoblogarticle
A spoken dialog model that understands and responds to paralinguistic cues like emotion.
arXiv 2026arXiv preprint, October 2026arXivproject
Speeds up full-duplex spoken dialog models by predicting several future frames per pass with rolling masked diffusion.
arXiv 2026arXiv preprint, May 2026arXivcode
The first benchmark for personalizing assistants across text, image, and audio.
EMNLP 2026★ OralConference on Empirical Methods in Natural Language Processing (Main)projectarXivcodedataset
Training-free control of accent at the phrase level in code-switching text-to-speech.
ICASSP 2025★ OralIEEE International Conference on Acoustics, Speech, and Signal ProcessingprojectarXiv
Builds personalized voice adapters for many speakers at once, quickly and efficiently.
ICASSP 2025IEEE International Conference on Acoustics, Speech, and Signal ProcessingprojectarXiv
An inference-time guidance method that keeps speaker adaptation robust for unseen voices.
INTERSPEECH 2024Annual Conference of the International Speech Communication AssociationprojectarXiv
One-shot personalized text-to-speech using lightweight low-rank adapters.
INTERSPEECH 2023★ OralAnnual Conference of the International Speech Communication AssociationarXivcodedemo
Personalizes TTS and voice conversion with just 5 to 10 seconds of untranscribed speech.
ICML 2022International Conference on Machine LearningarXivdemo
Builds a TTS model from untranscribed speech using diffusion and classifier guidance.
ICLR 2022International Conference on Learning RepresentationsprojectarXivcodeposter
Speeds up diffusion speech synthesis with a data-dependent Gaussian prior.
NeurIPS 2026Neural Information Processing SystemsarXiv
Turns a pretrained autoregressive TTS model into a diffusion language model for parallel generation.
arXiv 2026arXiv preprint, June 2026arXiv
Compresses audio tokens so autoregressive text-to-speech runs faster.
arXiv 2026arXiv preprint, March 2026arXiv
Extracts a target speaker's voice in a single step by combining masking and flow matching.
arXiv 2022arXiv preprintarXivdemo
Personalized text-to-speech from only 10 seconds of untranscribed speech.
ACL 2025Annual Meeting of the Association for Computational LinguisticsarXiv
Edits text at any granularity, coarse to fine, with a diffusion language model.
ACL 2022Annual Meeting of the Association for Computational LinguisticsarXivcode
Fixes text degeneration by gating gradients of rare token embeddings.
arXiv 2024arXiv preprintarXiv
Large language models built for Korean language and culture.
WACV 2026IEEE/CVF Winter Conference on Applications of Computer VisionprojectarXiv
Lets diffusion models learn new artistic styles by sampling higher noise levels during fine-tuning.
CVPR 2025IEEE/CVF Conference on Computer Vision and Pattern RecognitionprojectarXiv
Zero-shot subject-driven image generation by framing it as diptych inpainting.
ACML 2023★ Oral · Best Paper AwardAsian Conference on Machine LearningprojectarXiv
One-shot video editing that keeps backgrounds consistent via masked blending.
ICLR 2022International Conference on Learning RepresentationsarXivcode
Improves conditional GAN generation on imbalanced data with Stein latent optimization.
ACS AMI 2022ACS Applied Materials & Interfaces (journal)paper
Recognizes silent speech from facial strain sensors placed by deep learning analysis.
Seoul National University · Mar 2019 - Aug 2025
Integrated M.S./Ph.D. program, Data Science & AI Lab (DSAIL), advised by Prof. Sungroh Yoon.
Seoul National University · Mar 2015 - Feb 2019
Graduated Cum Laude.