Consulting for the development and evaluation of a formative assessment and reporting system with AI for the strengthening of teaching practice in Peru

Description

The Inter-American Development Bank (IDB) is calling on specialized consulting firms for the co-design, implementation and psychometric evaluation of a pilot system of formative assessment and automated reporting with Artificial Intelligence.

The project will be executed on the existing EVAL-AI platform (whose source code and technical documentation will be provided by the Bank) with the aim of strengthening pedagogical feedback capacities in the area of mathematics within the Peruvian education system.

1. Design of the reporting system and AI metrics (Component 1)
Analytical visualization dashboards: Design and implement interactive and intuitive dashboards within EVAL-AI for teachers to consult in real time the results of the items answered by students.

Development of pedagogical metrics: Configure algorithms to extract key performance indicators, such as specific performance by mathematical ability, indexed difficulty levels, net percentage of correct answers, maps of common errors, and automatic detection of areas of academic opportunity.

Advanced data segmentation: Enable dynamic filters that allow faculty to isolate and interpret information by entire group, individual student, specific periods, curricular topics, or competencies assessed.

Portable Reporting and Usability: Develop a fully downloadable consolidated reporting system and run functional user experience (UX) tests with active faculty to validate usability and data loading.

2. Implementation Framework, Scope and Sample Design (Component 2)
Distribution of the pilot study: The consultancy will be deployed in 30 educational institutions in Metropolitan Lima under an experimental methodology structured in two blocks: a Treatment Group (15 schools that will use Eval-IA) and a Control Group (15 schools that will maintain their traditional methods).

Operational schedule and facilitators: The pilot will have an intensive duration of approximately 2 months, contemplating a minimum of 7 sessions of direct work with teachers and the deployment of at least 8 field facilitators for weekly technical supervision in schools.

Baselines and Exit (Impact Measurement): Collect metrics at the beginning and end of the intervention in both study groups. The methodological model will strictly differentiate direct impacts (pedagogical knowledge and teacher practices) from indirect impacts (learning achieved by students).

3. Psychometric validation, data analysis and scalability (Component 3)
Statistical reliability tests: To mathematically document the evidence of internal consistency of the evaluation instruments applied by performing advanced statistical tests, using Cronbach's alpha coefficient and McDonald's omega coefficient.

Structural construct validation: Apply exhaustive factor analysis on the collected data matrices to rigorously certify the technical validity of the constructs and psychometric instruments administered.

Iterative technology optimization: Evaluate the operational performance of the EVAL-AI application in real connectivity environments. The firm will need to adjust and refine the software architecture based on the usability findings and feedback collected.

Roadmap for scaling: Consolidate the technical report of the project into a final executive report that includes lessons learned, analytical conclusions and strategic recommendations aimed at an eventual scaling of the system at the national level.

Note: Interested companies should review the requirements on the BEO Bidder Portal.