Login for PhD students/staff at UCPH      Login for others
Methods for Statistical Evaluation of AI
Second title: From PAC Bounds to Fairness
Provider: Faculty of Science

Activity no.: 7015-25-07-31There are 30 available seats 
Enrollment deadline: 15/06/2025
PlaceHotel Nyborg Strand, Denmark
Østerøvej 2, 5800 Nyborg
Date and time25.08.2025, at: 00:00 - 29.08.2025, at: 16:00
Regular seats30
Course fee5,600.00 kr.
ECTS credits2.50
Contact personNina Weisse    E-mail address: weisse@math.ku.dk
Enrolment Handling/Course OrganiserLine Katrine Harder Clemmensen    E-mail address: lkhc@math.ku.dk
Written languageEnglish
Teaching languageEnglish
Block note5 days in August 2025
Exam requirementsParticipate in all lectures & group work at the summerschool, prepare & present poster of own research in the context of course theme (pre-assignment). Hand-in a report with improved design of experiment and/or methodology for statistical evaluation of AI
Course workload
Course workload categoryHours
Preparation20.00
Lectures25.00
Practical exercises23.75
Sum68.75
Aim and content
The aim of course is to equip the participants with statistical methods, and knowledge of the newest research within statistical methods, for evaluation of machine learning and AI.

The course runs as a summer school, retreat for five full days. As preparation, the students prepare a poster framing their own research in relation to the theme of the summer school. This includes reading selected papers as preparation before the summer school. The posters will be presented by the students at a poster session the first evening of the summer school. The posters are used for group work and further refinement through an assignment, which finalizes the course.

The five course days exists of lectures, including smaller assignments and discussion to activate the students during lectures. The lecturers consist of both external lecturers as well as lecturers from Danish Universities. Group work is also carried out during the five days of summer school. An individual assignment is handed in one week after the summer school.

Formal requirements
The students need qualifications in statistical methods and machine learning or in one of these areas in depth.

Learning outcome
Knowledge:
• Understand a variety of statistical methods, including PAC bounds, measures for evaluation of AI, and fairness metrics, fairness calibration and their short comings.

Skills:
• Choose between a selection of statistical methods for evaluation of ML and AI
• Assess fairness of ML models
• Calibrate for better fairness, but also understand when calibration can skew fairness even further
• Apply different sampling strategies for building AI systems, depending on the requirements of the application.

Competences:
• Develop the competence to critically assess AI models, identifying strengths, weaknesses, and potential biases in model behaviour.
• Ability to incorporate fairness considerations withing the design and development of ML, ensuring that ethical principles guide the creation and deployment of AI systems.
• Build ability to engage in peer review, provide constructive feedback, and contribute to the collective advancement of knowledge in statistical AI evaluation.
• Promote abilities to identify the gaps and open research challenges in statistical evaluation of AI, including developing methods for necessary and sufficient methods for evaluation of AI models.

Teaching and learning methods
Lectures, group work, and individual assignments.

Lecturers
External guest lecturers:

• To be confirmed: Gaël Varoquaux, INRIA, INRIA Saclay-Ile-de-France Research Centre, France :
Course (tentative title): Evaluation of Machine Learning models on typical tabular data.
Short bio: Research direction of the Soda team in INRIA. He is specialized in applications of Machine Learning to health. Prof. Varoquaux is a co-founder of the Scikit-learn library that is a widely used Machine Learning library in Python.

• Benjamin Guedj, Department of Computer Science, University College London, United Kingdom:
Title: Artificial intelligence from a statistical perspective.
Short bio: Dr. Benjamin Guedj is specialized in machine learning. He is a Research Fellow at University College London (UCL) in the Department of Computer Science and a research scientist at Inria, France. His research focuses on theoretical machine learning, including statistical learning theory, PAC-Bayes, and generalization for deep learning.

• To be confirmed: Jose Hernandez-Orallo, Valencian Research Institute for Artificial Intelligence, Universitat Politècnica de València, Spain:
Course (tentative title): Evaluation of cognitive capabilities of AI systems.
Short bio: The main research of Prof. Hernández-Orallo focuses on AI evaluation, benchmarking for measuring AI capabilities, and the safety of AI systems.

Guest lecturers from Danish institutions:

- To be confirmed:  Prof. Aasa Feragen, Image Analysis and Computer Graphics, DTU-Compute, Technical University of Denmark.
Course (tentative title): Geometric Statistics in Image Analysis.

- To be confirmed: Prof. Christian Igel, Department of Computer Science, University of Copenhagen, Denmark.
Course (tentative title): PAC-Bayesian Analysis of Ensemble Machine Learning Models.

- Prof. Line Clemmensen, Dept. of Mathematical Sciences, University of Copenhagen and DTU-Compute, Technical University of Denmark.
Title: Data representativity for Machine Learning and AI systems.

- Prof. Murat Külahci, Statistics and Data Analysis, DTU-Compute, Technical University of Denmark.
Course (tentative title): Active learning in time-series and sequential data.

- Assoc. Prof. Andres Masegosa. Department of Computer Science at Aalborg University, Technical University of Denmark.
Course (tentative title): Generalization of Deep Neural Networks.

Remarks

The participant fee is 5,600 DKK and covers five days incl overnight stays, food, social event, and all course content.

It is co-hosted with D3A.

How to apply:
PhD students enrolled at University of Copenhagen (UCPH)
1) Please provide us with "stedkode" and "alias" when you register by choosing "Apply" at the upper right corner of the course description, so that UCPH Accounting Department can make an internal transfer regarding payment of the Participation fee.
If you have been accepted for the course, we will verify your enrollment via mail.

PhD students enrolled at other universities than UCPH
1) Choose “Apply” in the upper right corner of the course description.
If you have been accepted for the course, we will send you a mail with a link to payment of the participant fee.
NB: Payment via creditcard or Mobilepay only. After you have made your payment of the participation fee, we will verify your enrollment via mail.

Search
Click the search button to search Courses.


Course calendar
See which courses you can attend and when
JanFebMarApr
MayJunJulAug
SepOctNovDec



Publication of new courses
All planned PhD courses at the PhD School are visible in the course catalogue. Courses are published regularly.