Robustness and Computability

Type

Master / Bachelor thesis

Prerequisites

  • Strong machine learning knowledge
  • Proficiency with Python
  • (Preferably) Proficiency with deep learning frameworks (Tensorflow or Pytorch)

Description

Modern AI systems, in particular deep learning methods, have demonstrated unparalleled accomplishments in many different fields. At the same time, there is overwhelming empirical evidence that these methods are unstable. These instabilities often occur in the form of so-called adversarial examples-misclassified data points that are very close (e.g., visually indistinguishable in case of images) to correctly classified data points. Why does deep learning consistently produce unstable learners even when one can prove that stable and accurate neural networks exist? The robustness field aims to understand this phenomenon and develop, ideally provable, robust AI systems. Today, so-called adversarial attacks can produce adversarial examples very reliably-posing a sizeable problem for safety-critical applications in AI. Thus, the study of robustness in deep learning continues to be vital for the safe deployment of AI in today's society.

References

  • Classical paper introducing adversarial examples/non-robustness of neural networks
    Intriguing properties of neural networks (https://arxiv.org/pdf/1312.6199.pdf)
  • This paper describes limitations on the existence of algorithms for computing accurate neural networks
    The difficulty of computing stable and accurate neural networks: On the barriers of deep learning and Smale's 18th problem (https://www.pnas.org/doi/full/10.1073/pnas.2107151119)
  • A Paper analyzing the tension between the goal of adversarial robustness and that of standard generalization
    Robustness May Be at Odds with Accuracy (https://arxiv.org/abs/1805.12152)
  • Modern stealing attack from black-box language models
    Stealing Part of a Production Language Model (https://arxiv.org/abs/2403.06634)
  • Translation of broad regulations into technical requirements to ensure responsible and robust AI
    COMPL-AI Framework: A Technical Interpretation and LLM Benchmarking Suite for the EU Artificial Intelligence Act (https://arxiv.org/abs/2410.07959)