Auto Appendix

The SC Conference Series, a leading forum on High Performance Computing (HPC), supports scientific rigor through an enhanced reproducibility of accepted papers. To that end, all manuscripts submitted to the SC Technical Papers program must contain an Artifact Description. Authors of accepted papers may request reproducibility badges, for which an Appendix describing the Artifact Evaluation is required.

In recent years, Chameleon has facilitated SC’s reproducibility initiative by enabling authors to develop and share computational, reproducible artifacts through the Chameleon cloud. The Chameleon platform helps authors and reviewers to easily share computational artifacts, which are included in the papers’ artifact appendices.

The proposed project aims to assess all AD/AE appendices submitted for reproducibility badge requests. This evaluation will focus on AD/AE appendices that utilized the Chameleon cloud as the execution platform, examining their potential for automation. Our aim is to evaluate the feasibility of fully automating various components of the appendices. Students will engage directly with the chairs of the SC24 Reproducibility Initiative in this effort.

Advancing SC Conference Artifact Reproducibility via Automation

• Topics: Reproducibility Reproducible Research Artifact Evaluation Open Science
• Skills: HPC, Cloud computing, Chameleon, MPI, OpenMP, CUDA
• Difficulty: Difficult
• Size: Large
• Mentors: Sascha Hunold
• Tasks:
  • Perform an analysis of the current limitations of AD/AE appendices submitted for Artifact Evaluation.
  • Re-run the computational artifacts to identify areas for enhancement, with a primary objective of achieving full automation of Artifact Evaluation using the Chameleon cloud.
  • Evaluate the existing automation capabilities of the Chameleon cloud.
  • Develop a set of recommendations for structuring Computational Artifacts, aimed at benefiting future SC conferences.