REQUEST FOR PROPOSAL (RFP)
For Model Evaluation of AI-enabled Oral Reading Fluency solution in Hindi and Gujarati 

Closing Date for Submission: November 17, 2025, at 23:59 PM

I. INTRODUCTION

Lords Education and Health Society (LEHS), registered under Societies Registration Act XXI of 1860   is a charitable organisation with its head office in New Delhi, India working in the Health, Agriculture and Education Domain for the poor. LEHS in furtherance of charitable objectives through its flagship unit i.e. Wadhwani AI aims to build equitable and sustainable systems to the underserved population and to bring the benefits of modern AI technology to underserved populations by building and deploying AI solutions for social impact across domains such as healthcare, agriculture, governance and education in India.

LEHS aims to promote the integration of technologies, particularly in emerging domains like artificial intelligence and innovations into the Indian mainstream primary healthcare, education, and agriculture systems through a partnership with the State and National Government, apex institutions, international agencies, and private sector partners e.g. innovators, social enterprises and other ecosystem contributors in line with its stated objectives for the betterment of society particularly focusing on projects of national and social significance. In line with its mission to support projects of national and social significance, LEHS also undertakes rigorous monitoring, evaluation, and learning (MEL) activities to assess the AI model performance, impact, usability, and scalability of different programmatic interventions.

II. BACKGROUND

Aligned with the Ministry of Education's visionary NIPUN Bharat mission on Foundation Literacy and Numeracy (FLN) and the National Education Policy 2020, which emphasize improving Oral Reading Fluency (ORF), recognizes that 'Learning to Read' enables 'Reading to Learn'. Addressing the pressing challenge that about 60% of Grade 5 students cannot read Grade 2 level text remains a shared priority. While current reading fluency assessments offer value, they are often time-consuming, lack auditability and suffer from inconsistent administration. This presents opportunities for enhancement through standardization and automated systems that can provide more comprehensive reading proficiency insights.

Wadhwani AI, in collaboration with the Department of Education, Rajasthan, has successfully deployed an AI-powered Oral Reading Fluency (ORF) application in government schools since May 2025. Similarly, a partnership with the Department of Education, Gujarat, has enabled the deployment of the ORF solution in government schools since August 2023. This innovative tool assesses students by analyzing their audio, generating fluency metrics, and categorizing them into four cohorts ranging from non-decoders to fluent readers.

The AI-powered ORF tool transcribes the student’s reading audio using an Automatic Speech Recognition (ASR) model. This transcript is aligned with the target text to generate metrics, including correct words, incorrect words, missed words, and additional words read. It also generates composite metrics such as word accuracy, correct words per minute. A rule based system then classifies the students into four cohorts based on these scores.

While AI tools offer promising scalable solutions, we recognize the importance of validation to build credibility and inform evidence-based policy decisions. In our shared commitment to educational excellence, model evaluation study represents a crucial step to ensure the reliability and accuracy of the AI-based ORF tool. Through rigorous, independent validation, we also aim to identify any potential biases across different linguistic regions and student populations, ensuring the tool performs consistently with human expert judgments. We aim to validate both the Automatic Speech Recognition (ASR) component's transcription accuracy and the system's ability to correctly generate metrics and classify students into appropriate fluency cohorts.

This RFP invites qualified agencies to undertake this evaluation.

III. Available Data for Evaluation

Wadhwani will provide relevant anonymized 4000 audio recordings each from students in Grades 3–8 in Hindi and Gujarati. Sampling will be stratified to ensure representative coverage across all identified dialect zones with equal distribution across grades and gender.

A Gujarati audio file will contain approximately 121 words, while the Hindi audio files for Rajasthan will be about 54 words. 

The agency shall ensure that all data remain confidential and are used solely for the purpose of this evaluation, in accordance with data protection and privacy laws.

IV. SCOPE OF WORK

The selected agency is expected to conduct an independent, rigorous validation study of the ORF tool in Hindi and Gujarati. The evaluation will focus on determining the accuracy and reliability of the tool in transcribing, generating reading scores, and classifying student oral reading fluency against human expert assessments. Specifically, the agency will assess the Automatic Speech Recognition (ASR) component’s ability to generate accurate transcriptions and validate the reading scores and rule-based fluency classification system that categorizes students into four pre-defined cohorts. The evaluation will also examine potential variations in model performance across grades, dialect regions, and gender, to ensure that the system performs equitably across linguistic and demographic subgroups.

To achieve this, the agency will be responsible for finalizing execution workflow and analytical plan in consultation with Wadhwani AI. This includes preparing a detailed quality assurance protocol; training, and managing independent groups of trained Hindi and Gujarati language assessors; and ensuring adherence to standardized annotation guidelines provided by Wadhwani AI. 

At least three experts will independently annotate a set of 4,000 anonymized audio files each for Hindi and Gujarati, through a three-step process: (i) transcription validation (verbatim transcription of the student’s reading), (ii) metric assessment (identifying correct, incorrect, extra, and missed words), and (iii) fluency cohort assignment based on established ORF rubrics. 

All assessors will remain blinded to AI outputs to maintain objectivity, and inter-rater reliability will be established by adjudication by a third expert. 

Following the human annotation, the agency will conduct a comparative analysis between AI-generated outputs and human expert assessments. The analysis will include:

• Transcription accuracy using Word Error Rate (WER) as a key indicator;
• Reading fluency metric-level comparisons (e.g., Correct Words Per Minute, word accuracy, and error proportions) to evaluate agreement and bias; and
• Cohort classification accuracy, measured using accuracy, precision, recall, and Cohen’s kappa.

Results will be stratified by grade, dialect zone, and gender to identify any systematic differences in AI performance. The agency is expected to interpret findings critically, highlight strengths and limitations, and provide actionable recommendations to strengthen model robustness, reliability, and fairness.

The contract period will be ten (10) weeks from the date of signing of the agreement. 

V. Required Deliverables

Deliverables for language model include: 

• Human Annotation Transcription Completion: 4000 in Hindi and Gujarati each
• Human Metric Assessment:4000 in Hindi and Gujarati each
• Human Cohort classification: 4000 in Hindi and Gujarati each
• Comparative Analysis Report: Human vs. AI analysis
• Final Evaluation Report and PPT

VI. Proposal Submission Details

Your technical proposal must be separate for Gujarati and Hindi model evaluation. Each should include:Technical Proposal (Proposals should reflect the applicant’s understanding of this specific scope and context. Submissions that appear generic or not customized may not be scored favorably):

• Understanding of Scope: Brief description of the assignment objectives and key deliverables.
• Execution Plan: Approach, methodology, and weekwise timelines for completing the evaluation.
• Team Composition and Qualifications: Details of key personnel and annotators with relevant expertise in Hindi/Gujarati linguistics and FLN pedagogy. Preferably, annotators should be qualified language teachers who have taught Hindi or Gujarati to primary grades (1–5). Native speakers with a degree in the respective language or linguistics may be also considered. CVs of all experts responsible for executing the study are mandatory in the proposal.
• Detailed Analysis Plan: Proposed methods for comparing human and AI outputs, including transcription accuracy and cohort classification.
• Quality Assurance Mechanisms: Processes to ensure data quality, inter-rater reliability, and adherence to annotation standards.
• Ethical Safeguards: Measures for maintaining confidentiality, and data security.

Financial Proposal

• Detailed cost breakdown 
• Applicable taxes and contingencies
• Timeline-based payment schedule

Organizational Profile

• Relevant project experience
• Sample of similar work, previous evaluation reports

Applicants must provide a detailed, itemized budget covering all activities. The budget must clearly justify that costs are directly attributable to this project and not part of the applicant's business-as-usual operations.  

CSR Clause: All fund utilization must comply with the provisions of the Companies Act, 2013, and CSR Rules.

VII. Proposal Submission Guidelines

Who Can Apply

Non-Profit Organizations, Business firms/for Profit Organizations with:

• Experience in education, AI, or speech analytics evaluations
• Expertise in Hindi and Gujarati linguistics and FLN pedagogy is mandatory
• Strong quantitative analysis and validation capabilities
• Experience with government or donor-funded education projects
• Data security and privacy protocols in place.

Submission Details  

• All responses to this RFP must be received no later than November 17, 2025. 
• The technical and financial proposal should be submitted separately only through e-mail in PDF in the below-given e-mail id: rfp.lehs@wadhwaniai.org .  
• The email's subject line must contain the reference number and title of the RFP: Technical Proposal for Proposal for Model Evaluation (Hindi and Gujarati) and Financial Proposal for Proposal for Model Evaluation (Hindi and Gujarati). Only those who submit complete technical with password protected financial proposals and all mandatory enclosures (Section VII) will be considered.  
• Up to two agencies may be selected, one each for the Hindi and Gujarati model evaluations.

However, agencies demonstrating the expertise and capacity to conduct both evaluations will be preferred.

For detailed information, please check the complete version of the RFP attached below.

Job Email ID: rfp.lehs(at)wadhwaniai.org

Download Attachment: RFP- Model Evaluation of AI-enabled Oral Reading Fluency solution in Hindi and Guj.pdf