Coding the Encoder: Situating Subjective and Contextual Aspects in High-Level Image Annotations

Delfina Sol Martinez Pandiana, Università di Bologna (Italy)

The talk explores the assignment of subjective high-level annotations to visual data in computer vision pipelines. It addresses how this annotated data is utilized to train AI models, highlighting the common lack of situational context, and the biases introduced. The talk describes the SituAnnotate methodology, which extends the data annotation process to include rich contextual information, such as sources, and financial/temporal/geographical contexts. This approach not only provides a comprehensive understanding of the situational grounding of annotations but also offers insights to mitigate biases, enhance model understanding, and empower AI practitioners in curating datasets aligned with specific criteria. The talk serves as a bridge between high-level semantic annotations and the imperative to reimagine annotation for cultural data, emphasizing the crucial role of situating context and socio-cultural factors in responsible AI system development.

Delfina Sol Martinez Pandiana is a postdoctoral researcher at the Centrum Wiskunde & Informatica (Amsterdam), specializing in high-level multimodal sensemaking and misinformation detection. Their PhD in Computer Science focused on automatically detecting abstract social concepts in images. They hold a B.A. in Human Evolutionary Biology from Harvard and an M.A. in Digital Humanities from the University of Bologna. Delfina combines cognitive research, semantic technologies, and computer vision to challenge assumed dichotomies in areas like computational propaganda and cultural representation.