Biography
Tianyu Yu received his B.S. degree from the Beihang University of China in 2021. He is currently working on Natural Language Processing and Multimodal Understanding, especially in MLLMs.
- Multimodal Large Lanuage Models
- Learning from Feedback
- Information Retrieval
M.S. in Computer Technology, 2021 ~ now
Tsinghua University
B.Eng. in Software Engineering, 2017 ~ 2021
Beihang University
Recent Publications
Experience
Project: Mitigating MLLM Hallucination with AI Feedback (2023.12 ~ 2024.06)
Project: Mitigating MLLM Hallucination with Fine-grained Correctional Human Feedback (2023.07 ~ 2023.12)
Project: Reformulating VLMs to construct MLLMs (2023.02 ~ 2023.07)
Project: Extract Common-sense Knowledge from Multimodal Corpora (2021.04 ~ 2022.12)
Project: Sentence-Level Pretraining for Document-Level RE (2020.07 ~ 2021.03)
Project: Joint Extraction of Evidence and Relation for Document-Level RE (2019.10 ~ 2020.05)
- Designed algorithm for RE-ID using sub-graph to capture local context, which was further used in downstream real applications.
- Designed a de-noising algorithm to automate data cleaning.
- Designed a large-scale data generation pipeline to reduce the cost of manual labeling.
Project: Cross-Modal Omni Interaction Modeling for Phrase Grounding
- Adressed the phrase grounding accuracy problem as the primary researcher.
- Devised a novel model to capture complex spatial and semantic relationship among image regions and phrases through multi-level multi-modal interaction.
- The new method improved the grounding accuracy by 6.15% on Flickr30K Entities and 21.25% on ReferItGame.