Negative Self-Distillation: Learning to Reason by Avoiding Flaws

Rongcan Pei, Zhepei Wei, Shuyao Xu, Xinyu Zhu, Wei-Lin Chen, Yu Meng

arXiv

• We propose Negative Self-Distillation (NSD), which improves LLM reasoning by learning to avoid errors produced by a negatively conditioned version of the model, without requiring reference answers or an external teacher. A dynamic token-level gate directs training toward reasoning errors while protecting basic language abilities. Across seven mathematical reasoning benchmarks, NSD improves average accuracy by 2.3, 7.5, and 6.0 percentage points for 1.7B, 4B, and 8B models, respectively, while retaining self-correction and improving training efficiency.

VERA: Identifying and Leveraging Visual Evidence Retrieval Heads in Long-Context Understanding

Rongcan Pei, Huan Li, Fang Guo, Qi Zhu

Code | arXiv

• Through attention analysis, we identify specific Visual Evidence Retrieval (VER) Heads — a sparse, dynamic set of attention heads critical for locating visual cues during reasoning. We propose VERA, a training-free framework that detects model uncertainty to trigger explicit verbalization of visual evidence. VERA achieves 21.3% relative improvement on Qwen3-VL-8B-Instruct and 20.1% on GLM-4.1V-Thinking across five benchmarks.

DeepInstructor: An Agentic AI Instructor for Experience-Driven Idea Evaluation

Rongcan Pei, Fang Guo, Qinglin Qi, Qi Zhu, Yun Luo, Jianhao Yan, Minjun Zhu, Qiujie Xie, Dehong Zheng, Yue Zhang

Paper | Code & Prompts

• We propose DeepInstructor, the first agentic AI instructor that emulates how human mentors leverage past scholarly experience to guide research ideation. DeepInstructor extracts experience triples from large-scale review corpora, constructs an Experience Graph, and employs an agentic reasoning framework for experience-driven idea evaluation.

PaperIgnition & Sci-Surf Project

Qi Zhu, Fang Guo, Rongcan Pei*, Shuqi He, Hui Chen, Yue Zhang

arXiv | Website | Code

• An intelligent paper recommendation and digestion platform that adopts a temporally decoupled architecture. The system autonomously fetches, indexes, and digests newly released arXiv papers through a daily pipeline, with multimodal LLM-based summarization executed offline.

Augmenting Multi-Technique Static Analysis with Large Language Models: A Neuro-Symbolic Approach to Smart Contract Vulnerability Detection

Junxiang Wang, Fu Song, Miaomiao Zhang, Bowen Du, Rongcan Pei

• We propose ELSA, a neuro-symbolic approach that augments static analysis with LLM-assisted constraint-guided reasoning and resolves conflicting outputs through analyzer ensemble. ELSA significantly outperforms baselines across three open-source benchmarks and self-constructed ZKP-based smart contracts.