Robustness and Computability

As artificial intelligence increasingly penetrates high-risk decision-making domains such as healthcare, financial sectors, and judicial processes, trust in algorithmic reasoning gains fundamental significance. This requirement is further underscored by the growing autonomy - meaning operation without human supervision - of such AI models. Consequently, transparent, comprehensible, and accountable computational systems are being demanded. A central question remains, however, whether and to what extent (strong) AI can actually fulfill these properties and how they can be integrated into AI systems through targeted design.

Research at our chair

Two papers tackling the trustworthiness property of AI:

  • Mathematical Algorithm Design for Deep Learning under Societal and Judicial Constraints: The Algorithmic Transparency Requirement (https://arxiv.org/abs/2401.10310)

General references

  • Classical paper introducing adversarial examples/non-robustness of neural networks
    Intriguing properties of neural networks (https://arxiv.org/pdf/1312.6199.pdf)
  • This paper describes limitations on the existence of algorithms for computing accurate neural networks
    The difficulty of computing stable and accurate neural networks: On the barriers of deep learning and Smale's 18th problem (https://www.pnas.org/doi/full/10.1073/pnas.2107151119)
  • A Paper analyzing the tension between the goal of adversarial robustness and that of standard generalization
    Robustness May Be at Odds with Accuracy (https://arxiv.org/abs/1805.12152)
  • Modern stealing attack from black-box language models
    Stealing Part of a Production Language Model (https://arxiv.org/abs/2403.06634)
  • Translation of broad regulations into technical requirements to ensure responsible and robust AI
    COMPL-AI Framework: A Technical Interpretation and LLM Benchmarking Suite for the EU Artificial Intelligence Act (https://arxiv.org/abs/2410.07959)

Contact

Do you have questions about our research in this area?

Please do not hesitate to contact us directly. Please send an e-mail to Adalbert Fono, one of our PhD students in the field of robustness and computability.

Inquiries from students are very welcome!