[행사/세미나] 인공지능대학원 전문가 초청 세미나(Dr. Sungjoon Park @Softly.ai, 3/4 금 13:00~15:00)
- 인공지능학과
- 조회수23334
- 2022-02-08
▶Title: KLUE: Korean Language Understanding Evaluation
▶Speaker: Dr. Sungjoon Park,CEO @ Softly.ai
▶Time : 2022 March 4th 13:00 ~ 15:00
▶Location:
Offline: 2nd Engineering Building Live Virtual Classroom
Online: https://skku-ict.webex.com/meet/ai.dept
▶Abstract:
We introduce Korean Language Understanding Evaluation (KLUE) benchmark. KLUE is a collection of 8 Korean natural language understanding (NLU) tasks, including Topic Classification, SemanticTextual Similarity, Natural Language Inference, Named Entity Recognition, Relation Extraction, Dependency Parsing, Machine Reading Comprehension, and Dialogue State Tracking. We build all of the tasks from scratch from diverse source corpora while respecting copyrights, to ensure accessibility for anyone without any restrictions. With ethical considerations in mind, we carefully design annotation protocols. Along with the benchmark tasks and data, we provide suitable evaluation metrics and fine-tuning recipes for pretrained language models for each task. We furthermore release the pretrained language models (PLM), KLUE-BERT and KLUE-RoBERTa, to help reproducing baseline models on KLUE and thereby facilitate future research. We make a few interesting observations from the preliminary experiments using the proposed KLUE benchmark suite, already demonstrating the usefulness of this new benchmark suite. First, we find KLUE-RoBERTa-large outperforms other baselines, including multilingual PLMs and existing open-source Korean PLMs. Second, we see minimal degradation in performance even when we replace personally identifiable information from the pretraining corpus, suggesting that privacy and NLU capability are not at odds with each other. Lastly, we find that using BPE tokenization in combination with morpheme-level pre-tokenization is effective in tasks involving morpheme-level tagging, detection and generation. In addition to accelerating Korean NLP research, our comprehensive documentation on creating KLUE will facilitate creating similar resources for other languages in the future.
▶Bio:
Sungjoon Park is co-founder and CEO of Softly.ai, developing AI-based content moderation product and solutions. He received Ph.D. from School of Computing at KAIST, advised by Alice Oh. His research was supported by Google PhD Fellowship program. Before that, he obtained BA and MA in Department of Psychology in Seoul National University.