FAQ: AIS AI Reviewer
What is this service?
ECIS 2027 is offering an optional AI-generated review as an opt-out feedback service for authors who submit Completed or Short Research Papes. The AI Review Service was developed by the AIS and piloted at ICIS in 2026 as an opt-in service where it was well received by authors.
Our goal in using the AIS AI review service is to provide authors with additional developmental feedback and to manage the reviewing burden on the IS community in light of the growing number of submissions conferences receive. We hope to lighten the load for conference reviewers while maintaining the quality and integrity of the review process.
The AI review does not replace human judgement but serves as an additional piece of information that human decision makers (associate editors and track chairs) can take into consideration. Given that this tool can serve as Reviewer 3, only two human reviewers will be recruited for each submission, and each human reviewer will be asked to ensure their review is not colored by any AI input.
The AIS AI reviewer is specially trained on ECIS submissions to adhere to AIS and ECIS standards. Thus, what we seek from human reviewers is purely human insight and evaluation rather than output from a non-AIS AI tool. It is our hope that freeing reviewers to focus more closely on fewer papers will enhance the ECIS review process. As with any innovation, we expect there to be a learning curve as AI is implemented in the review process. We ask for collaborative and constructive feedback throughout the process, with the goal of maintaining the standard of excellence ECIS is known for.
Will the AI reviewer make a decision on my paper?
No. The AI review is trained to provide supplemental feedback only. The AI review will also be weighted less heavily than human-generated reviews in paper acceptance decisions. The AI review may be used by track chairs and associate editors as they make decisions, but the AI tool will not give a recommendation about whether a submission should be accepted or rejected.
Will my paper be used to train OpenAI models?
No. AIS uses OpenAI through its API, and OpenAI states that data submitted through the API is not used to train or improve OpenAI models. OpenAI’s business data policy states the same for API customers.
Will my paper be stored by OpenAI?
AIS configures its OpenAI API calls with store=false, which disables storage of response objects for later retrieval. However, OpenAI states that API data may still be retained for abuse monitoring for up to 30 days, after which it is not retained unless longer retention is legally required. For that reason, the most accurate statement is that your paper is not used for model training and is not stored, but temporary retention may still occur for security and abuse-monitoring purposes.
How does the system generate the AI review?
The system retrieves consenting papers from the regular submission system and uses an AI review pipeline built on OpenAI’s API together with retrieval-augmented generation (RAG). The AIS AI review service is trained to assess five dimensions of scholarly contribution: theory, method, framing, phenomenon, and composition. To shape the style and content of the review, AIS developed and tested numerous prompts and review models using prior publications and selected the prompt that best aligned with desirable conference review traits, such as being informative, concise, fair, and constructive.
How reliable is the AI review?
The system is designed to generate feedback using a state-of-the-art OpenAI model, data from prior ECIS papers, and a prompt informed by an editorial on what reviewers should look for in papers. Even so, AI-generated feedback may sometimes be incomplete, inaccurate, or less nuanced than expert human review. Decision makers and authors will therefore treat the AI review as supplemental feedback, not as an authoritative judgment on paper quality.
Do I have to opt-in?
No. While the AIS AI review service is provided as a default setting, authors can opt-out during submissions. Authors who opt out will receive 2 human peer reviews.
Who can I contact if I have additional questions?
About the AIS AI review service, including technical questions, you can contact:
Dorothy E. Leidner, PhD
President, The Association for Information Systems (AIS)
Leslie H. Goldberg Jefferson Scholars Foundation Eminent Professor, University of Virginia
Reza Mousavi, PhD
Associate Vice President of AI, The Association for Information Systems (AIS)
Associate Professor, University of Virginia
About the ECIS 2027 submission and review process you can contact:
Leona Chandra Kruse, Jan Recker, Amber Grace Young
ECIS 2027 Programme Co-Chairs
About AIS AI Reviewer Policies, you can contact:
Amber Grace Young, PhD
Vice President of C.O.R.E., The Association for Information Systems (AIS)
Associate Professor, University of Oklahoma
