Mert Yuksekgonul

I'm a ~~second~~ third year PhD student in Computer Science at Stanford University. I am lucky to be advised by James Zou and Carlos Guestrin.

I am interested in making machine learning models more reliable, and their failures more predictable. This broad interest so far led me to work on topics around interpretability, robustness, multimodality, and uncertainty. I really enjoy thinking about abstraction, concepts, and compositionality in humans and machines.

During Summer 2023, I interned at MSR working with Besmira Nushi in the Adaptive Systems and Interaction Group .

I was extremely fortunate to have many wonderful mentors in my undergraduate years. I worked with Prof. Mackenzie Mathis, Prof. Alexander Mathis and Prof. Matthias Bethge as part of Bethge Lab(Tübingen/MPI) and Adaptive Motor Control Lab (Harvard/EPFL). My last undergrad research gig was with Dr. Xavier Boix as part of Poggio Lab(MIT) and Sinha Lab(MIT). I started Machine Learning research with Prof. Mustafa Baydogan at Bogazici University.

[ Email / Github / Twitter / Google Scholar ]

► Older News

Selected Publications

	When and why vision-language models behave like bags-of-words, and what to do about it? Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, James Zou Oral @ ICLR 2023 (Top 5% of all accepted papers) [ Paper , Code ] ▶ Show Description Recent work [and many tweet threads] suggests that Vision-Language models(VLMs) such as CLIP do not fare well with compositional understanding. Here, we first propose a large-scale benchmark, ARO(Attribution, Relation and Order) to evaluate fine-grained relational, attributive, and order understanding. Why did the BoW-like behavior is not reflected in the retrieval evaluations (e.g. COCO/Flickr30k), where the datasets contain rich compositional structure? We propose interesting experiments to show that models do not need to do well with compositions to perform well on these tasks. Similarly, this can explain why contrastive-pretrained models may be exploiting this shortcut, and we should be careful about it. Following this intuition, we propose composition-aware negative mining. Check out our work !
	Beyond Confidence: Reliable Models Should Also Quantify Atypicality Mert Yuksekgonul, Linjun Zhang, James Zou, Carlos Guestrin NeurIPS 2023, Contributed Talk @ ICLR 2023 Workshop on Trustworthy Machine Learning [Code Soon ] ▶ Show Description While most machine learning models can provide confidence in their predictions, confidence is insufficient to understand and use the model's uncertainty reliably. In this work, we investigate the relationship between how atypical~(or rare) a sample is and the reliability of a model's confidence for this sample. Read the paper for interesting connections between atypicality and uncertainty!
	Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models Mert Yuksekgonul, Varun Chandrasekaran, Erik Jones, Suriya Gunasekar, Ranjita Naik, Hamid Palangi , Ece Kamar , Besmira Nushi Preprint [ Paper, Code Soon ] ▶ Show Description We investigate the internal behavior of LLMs associated with their generation of factually incorrect text. We propose modeling factual queries as constraint satisfaction problems and use this framework to investigate how the model interacts internally with factual constraints. We find a strong positive relationship between the model's attention to constraint tokens and the factual accuracy of generations.
	Post-hoc Concept Bottleneck Models Mert Yuksekgonul, Maggie Wang, James Zou Spotlight @ ICLR2023 (Top 25% of all accepted papers) [ Paper, Code ] ▶ Show Description Concept Bottleneck Models are very cool! But it's hard to find large training datasets with concept annotations, or match the performance of unrestricted neural nets with limited bottlenecks. In this work, we address practical limitations of CBMs by introducing Post-hoc Concept Bottleneck models (PCBMs). Further, we have a user study where humans can improve PCBMs via concept-level feedback.
	Leveraging medical Twitter to build a visual–language foundation model for pathology AI Zhi Huang, Federico Bianchi, Mert Yuksekgonul, Thomas Montine, James Zou Nature Medicine [ Preprint, Demo ] ▶ Show Description We collect data from Twitter (yes, Twitter) + LAION and release the largest text-image pathology dataset: OpenPath. We also release PLIP, a CLIP-variant for pathology which gives exciting performance in 0-shot, transfer learning, and retrieval for Path.
	Meaningfully debugging model mistakes using conceptual counterfactual explanations Abubakar Abid, Mert Yuksekgonul, James Zou ICML 2022 [ Paper , Code ] ▶ Show Description We use human understandable concepts and counterfactual explanations (we call Conceptual Counterfactual Explanations) to debug model mistakes and reveal biases of a model.

Recent News

► Older News

Selected Publications

► More Publications