MADSlab University of Udine

Research area

Software and System Testing

Finding failures before users do: automated test generation and quality assurance for complex software, AI-based and self-adaptive systems.

Modern software no longer runs in isolation: it reasons with machine-learned components, adapts itself at runtime and acts in the physical world. Classic testing techniques struggle with these systems, whose input spaces are huge and whose correct behaviour is often hard to specify.

In the Software and System Testing area we develop techniques and tools to systematically exercise such systems and to expose their failures early. We combine search-based and model-based generation, simulation, and empirical evaluation on realistic case studies, often together with industrial partners.

Our results are released as open-source tools and artifacts, so that other researchers and practitioners can reproduce and build upon them.

Publications

  1. Automated Feature Extraction for Testing Deep Learning Systems through Illumination Search

    Tahereh Zohdinasab, Vincenzo Riccio, Paolo Tonella

    Software Testing, Verification and Reliability 36(1-2) · 2026

    Journal articleTestingDOI PDF
    BibTeX 
    @article{vr:zohdinasab2026automate,
  groups = {testing},
  title = {Automated Feature Extraction for Testing Deep Learning Systems through Illumination Search},
  volume = {36},
  doi = {10.1002/stvr.70019},
  number = {1-2},
  journal = {Software Testing, Verification and Reliability},
  publisher = {Wiley},
  author = {Zohdinasab, Tahereh and Riccio, Vincenzo and Tonella, Paolo},
  year = {2026},
  month = {January},
  pdf = {https://p1ndsvin.github.io/assets/pdf/riccio_STVR26.pdf}
}

  2. DeepNaqqal: Human-Aligned Automated Validation of Test Inputs for Deep Learning

    Maryam Maryam, Matteo Biagiola, Paolo Tonella, Vincenzo Riccio

    2026 IEEE International Conference on Software Testing, Verification and Validation (ICST), pp. 76–87 · IEEE · 2026

    Conference paperTestingDOI PDF
    BibTeX 
    @inproceedings{vr:maryam2026deepnaqq,
  groups = {testing},
  title = {DeepNaqqal: Human-Aligned Automated Validation of Test Inputs for Deep Learning},
  doi = {10.1109/icst69053.2026.00022},
  booktitle = {2026 IEEE International Conference on Software Testing, Verification and Validation (ICST)},
  publisher = {IEEE},
  author = {Maryam, Maryam and Biagiola, Matteo and Tonella, Paolo and Riccio, Vincenzo},
  year = {2026},
  month = {May},
  pages = {76–87},
  pdf = {https://p1ndsvin.github.io/assets/pdf/DeepNaqqal_ICST2026.pdf}
}

  3. DevOps Testing for Cyber-physical Systems

    Mattia Fazzini, Alessio Gambi, Vincenzo Riccio, Annibale Panichella, Stefan Klikovits

    Roadmap for DevOps in Cyber-Physical Systems, pp. 45–81 · Springer Nature Singapore · 2026

    Book chapterTestingDOI
    BibTeX 
    @inbook{vr:fazzini2026devops,
  groups = {testing},
  title = {DevOps Testing for Cyber-physical Systems},
  doi = {10.1007/978-981-95-1786-2_4},
  booktitle = {Roadmap for DevOps in Cyber-Physical Systems},
  publisher = {Springer Nature Singapore},
  author = {Fazzini, Mattia and Gambi, Alessio and Riccio, Vincenzo and Panichella, Annibale and Klikovits, Stefan},
  year = {2026},
  pages = {45–81}
}

  4. Does Road Diversity Really Matter in Testing Automated Driving Systems?

    Stefan Klikovits, Vincenzo Riccio, Ezequiel Castellano, Ahmet Cetinkaya, Alessio Gambi, Paolo Arcaini

    Empirical Software Engineering 32(1) · 2026

    Journal articleTestingDOI PDF
    BibTeX 
    @article{vr:klikovits2026does,
  groups = {testing},
  title = {Does Road Diversity Really Matter in Testing Automated Driving Systems?},
  volume = {32},
  doi = {10.1007/s10664-026-10920-5},
  number = {1},
  journal = {Empirical Software Engineering},
  publisher = {Springer Science and Business Media LLC},
  author = {Klikovits, Stefan and Riccio, Vincenzo and Castellano, Ezequiel and Cetinkaya, Ahmet and Gambi, Alessio and Arcaini, Paolo},
  year = {2026},
  month = {July},
  pdf = {https://p1ndsvin.github.io/assets/pdf/EMSE2026.pdf}
}

  5. EagleSemble at the ICST 2026 Tool Competition – Self-Driving Car Testing Track

    Cristian Aquilino, Ettore Ritacco, Vincenzo Riccio

    IEEE International Conference on Software Testing, Verification and Validation (ICST 2026), Tool Competition · 2026

    Conference paper 2nd place, ICST 2026 Tool CompetitionTesting PDF
    BibTeX 
    @inproceedings{vr:aquilino2026eaglesemble,
  groups = {testing},
  author = {Cristian Aquilino and Ettore Ritacco and Vincenzo Riccio},
  title = {EagleSemble at the ICST 2026 Tool Competition -- Self-Driving Car Testing Track},
  booktitle = {IEEE International Conference on Software Testing, Verification and Validation (ICST 2026), Tool Competition},
  year = {2026},
  pdf = {https://p1ndsvin.github.io/assets/pdf/ICST_SDC_EagleSemble.pdf},
  award = {2nd place, ICST 2026 Tool Competition}
}

  6. GIFTbench: Generative Image Fuzz Testing Benchmark

    Maryam Maryam, Matteo Biagiola, Andrea Stocco, Vincenzo Riccio

    Science of Computer Programming 253, pp. 103502 · 2026

    Journal articleTestingDOI PDF
    BibTeX 
    @article{vr:maryam2026giftbenc,
  groups = {testing},
  title = {GIFTbench: Generative Image Fuzz Testing Benchmark},
  volume = {253},
  doi = {10.1016/j.scico.2026.103502},
  journal = {Science of Computer Programming},
  publisher = {Elsevier BV},
  author = {Maryam, Maryam and Biagiola, Matteo and Stocco, Andrea and Riccio, Vincenzo},
  year = {2026},
  month = {August},
  pages = {103502},
  pdf = {https://p1ndsvin.github.io/assets/pdf/scp_giftbench.pdf}
}

  7. Targeted Deep Learning System Boundary Testing

    Oliver Weißl, Amr Abdellatif, Xingcheng Chen, Giorgi Merabishvili, Vincenzo Riccio, Severin Kacianka, Andrea Stocco

    ACM Transactions on Software Engineering and Methodology 35(7), pp. 1–31 · 2026

    Journal articleTestingDOI PDF
    BibTeX 
    @article{vr:weil2026targeted,
  groups = {testing},
  title = {Targeted Deep Learning System Boundary Testing},
  volume = {35},
  doi = {10.1145/3771557},
  number = {7},
  journal = {ACM Transactions on Software Engineering and Methodology},
  publisher = {Association for Computing Machinery (ACM)},
  author = {Weißl, Oliver and Abdellatif, Amr and Chen, Xingcheng and Merabishvili, Giorgi and Riccio, Vincenzo and Kacianka, Severin and Stocco, Andrea},
  year = {2026},
  month = {June},
  pages = {1–31},
  pdf = {https://p1ndsvin.github.io/assets/pdf/weissl_tosem2026_.pdf}
}

  8. XMutant: XAI-based Fuzzing for Deep Learning Systems

    Xingcheng Chen, Matteo Biagiola, Vincenzo Riccio, Marcelo d’Amorim, Andrea Stocco

    Empirical Software Engineering 31(4) · 2026

    Journal articleTestingDOI PDF
    BibTeX 
    @article{vr:chen2026xmutant,
  groups = {testing},
  title = {XMutant: XAI-based Fuzzing for Deep Learning Systems},
  volume = {31},
  doi = {10.1007/s10664-025-10792-1},
  number = {4},
  journal = {Empirical Software Engineering},
  publisher = {Springer Science and Business Media LLC},
  author = {Chen, Xingcheng and Biagiola, Matteo and Riccio, Vincenzo and d’Amorim, Marcelo and Stocco, Andrea},
  year = {2026},
  month = {March},
  pdf = {https://p1ndsvin.github.io/assets/pdf/xmutant.pdf}
}

  9. An Industrial Experience Report on Applying Search-based Boundary Input Generation to Cyber-Physical Systems

    Pablo Valle, Vincenzo Riccio, Aitor Arrieta, Paolo Tonella, Maite Arratibel

    Empirical Software Engineering 30(4) · 2025

    Journal articleTestingDOI PDF
    BibTeX 
    @article{vr:valle2025an,
  groups = {testing},
  title = {An Industrial Experience Report on Applying Search-based Boundary Input Generation to Cyber-Physical Systems},
  volume = {30},
  doi = {10.1007/s10664-025-10670-w},
  number = {4},
  journal = {Empirical Software Engineering},
  publisher = {Springer Science and Business Media LLC},
  author = {Valle, Pablo and Riccio, Vincenzo and Arrieta, Aitor and Tonella, Paolo and Arratibel, Maite},
  year = {2025},
  month = {May},
  pdf = {https://p1ndsvin.github.io/assets/pdf/EMSE_LIFTJANUS.pdf}
}

  10. Benchmarking Generative AI Models for Deep Learning Test Input Generation

    Maryam Maryam, Matteo Biagiola, Andrea Stocco, Vincenzo Riccio

    2025 IEEE Conference on Software Testing, Verification and Validation (ICST), pp. 174–185 · IEEE · 2025

    Conference paper Best Paper AwardTestingDOI PDF
    BibTeX 
    @inproceedings{vr:maryam2025benchmar,
  groups = {testing},
  title = {Benchmarking Generative AI Models for Deep Learning Test Input Generation},
  doi = {10.1109/icst62969.2025.10989043},
  booktitle = {2025 IEEE Conference on Software Testing, Verification and Validation (ICST)},
  publisher = {IEEE},
  author = {Maryam, Maryam and Biagiola, Matteo and Stocco, Andrea and Riccio, Vincenzo},
  year = {2025},
  month = {March},
  pages = {174–185},
  pdf = {https://p1ndsvin.github.io/assets/pdf/ICST2025.pdf},
  award = {Best Paper Award}
}

All publications in this area

People

Vincenzo Riccio

Associate Professor

Software testing, testing of deep-learning systems, self-adaptive systems

Other research areas

Cybersecurity

Breaking and fixing real systems: security of protocols, networks and industrial devices, from formal verification to hands-on vulnerability research.

Choreographic Autonomous Systems

Correct by construction: choreographic languages, types and formal models for coordinating fleets of autonomous, distributed agents.