| Abstract: | We thank the discussants for sharing their unique perspectives on the problem of designing automatic algorithm change protocols (aACPs) for machine learning‐based software as a medical device. Both Pennello et al. and Rose highlighted a number of challenges that arise in real‐world settings, and we whole‐heartedly agree that substantial extensions of our work are needed to understand if and how aACPs can be safely deployed in practice. Our work demonstrated that aACPs that appear to be harmless may allow for biocreep, even when the data distribution is assumed to be representative and stationary over time. While we investigated two solutions that protect against this specific issue, many more statistical and practical challenges remain and we look forward to future research on this topic. |
