AI vs. AI: comparing artificial intelligence with actual intelligence for a gameplay task

Aycock, John; Biittner, Katie; Khaleel, Syeda Zainab; Therrien, Carl; Querengesser, Allie; Sikstrom, Hailey

AI vs. AI: comparing artificial intelligence with actual intelligence for a gameplay task

Author

Aycock, John

Biittner, Katie

Khaleel, Syeda Zainab

Therrien, Carl

Querengesser, Allie

Sikstrom, Hailey

Date

2025

Keywords

AI, reinforcement learning, user studies, code coverage, reverse engineering, Atari 2600

Abstract (summary)

Is AI always the best choice for every task? We conducted two user studies, one large-scale and one small-scale, to attempt a testing-related gameplay task, where the goal was to maximize the code and data coverage of a set of Atari 2600 games. This particular problem was previously addressed using an AI-based system published by Ganesh et al. in the 2023 IEEE Conference on Games. Our new, human-derived results not only replicate the coverage shown in the previous AI-based study, but more importantly, by using humans for gameplay we were able to get those results with much less time, effort, and resources.