UF Data Studio @ NeurIPS 2025, San Diego

In addition to my attendence at ICDM25, I was honored to attend the conference on Neural Information Processing Systems (NeurIPS) this month to present OPTiCAL a second time. Like ICDM, I will be making blog posts about the academic highlights of the conference and my time in San Diego.

The Highlights

Posts about the following highlights of my trip will appear on this website.

My Time in San Diego

A travel blog about my time in Washington D.C. including the places I visited, the food I ate, and my journey to and from the conference.

Research @ NeurIPS

The papers, workshops and tutorials that I found most inspiring and impressive will get their own blogpost.

Our Contribution

The UF Data Studio's contribution to NeurIPS was a second presentation of OPTiCAL: An Abstract Positional Reasoning Benchmark for Vision Language Models. A post summarizing the paper will appear soon.

OPTiCAL: An Abstract Positional Reasoning Benchmark for Vision Language Models

I attended NeurIPS to present OPTiCAL: An Abstract Positional Reasoning Benchmark for Vision Language Models at the Evaluating the Evolving LLM Lifecycle workshop. I summarized the findings of that work in the overview of my posts for that conference, but I will reproduce the summary here.

Key Findings

• No model is more than 66% accurate at the tasks overall.
• Models often answer 'hexagon' instead of 'pentagon' when the latter was the answer. There were no hexagons in the data.
• We detect significant (p < 0.01) directional performance bias in each of the models benchmarked.

More Info

The official paper should be available to read on the ICDM25 workshop proceedings, which will be released soon on IEEE Explore. Our code and the Shapes30k dataset are available on GitHub. Feel free to reach out to the authors with questions about the work.

We received invaluable feedback on the present OPTiCAL from the community of multimodal AI researchers at MMAI and other colleagues at ICDM25. If you're working on multimodal AI, and VLM evaluation especially, we'd love to hear from you. Please contact me through the channels available on our webpage.

For more information about our research, return to our homepage: ufdatastudio.com.