| Client | A national-level public administration entity |
| Goals | To simplify the formal verification process of funding applications using artificial intelligence |
| Scope of Work | |
| Duration | 2 months |
| Outcomes | A Proof of Concept (PoC) to assess the feasibility of AI implementation, focusing on analyzing four key aspects of the formal evaluation of funding applications |
| Technologies | GenAI ChatGPT 3.5/4.0, Langchain + Langsmith, Amazon Textract (OCR), Gradio, FastAPI, AWS infrastructure |
Client background
We conducted the project for a national-level public administration body through our technology partner. This entity distributes funds through various programs and competitions.
The procedure for submitting funding applications includes several stages, from the announcement of the call, through the preparation and submission of the application, to its evaluation and the selection of projects for funding. Each program has specific eligibility requirements for applicants and criteria for project evaluation. The formal assessment process is complex due to the numerous national and European regulations that must be met. Each application must comply with various eligibility rules and EU policies.
Evaluation criteria include technical, financial, and substantive aspects, requiring detailed analysis to ensure the feasibility and effectiveness of projects. This process must be transparent and fair, requiring thorough documentation and multi-stage verification.
Process
The business need was to develop a solution that would speed up the assessment of the eligibility of funding applications based on a checklist. The solution should initially evaluate which criteria were met and which were not, and also highlight doubtful cases. Based on this, the system user must manually verify the doubtful cases and ultimately sign off on the decision.
Accordingly, the application includes a set of documents related to the application, such as statements, VAT invoices, and payment confirmations. All of these must meet a range of formal criteria.
It was necessary to determine whether AI algorithms could assist humans in verifying their correctness and compliance with applicable requirements for a given support program.
Thus, we proposed a Proof of Concept (PoC) project. This allowed us to quickly determine whether artificial intelligence could handle the verification of applications against a few key questions from the extensive checklist.
Solution
Our solution analyzes a set of documents attached to the application for a few selected questions from the entire checklist.
One of the first challenges was selecting and preparing appropriate test cases to assess the effectiveness of generative AI models. We used the knowledge of employees from the administration body for which we implemented the project, allowing us to develop representative examples.
The process begins with using OCR (Optical Character Recognition) to read the content of each file. OCR was essential because most files are scans of documents of varying quality.
After obtaining text from the documents through OCR, it is classified by a generative AI model. This enables the application to “understand” what each document is.
Next, the content undergoes a more detailed analysis against selected formal criteria. It’s worth noting that the documents are not standardized, so AI and appropriate prompt engineering are necessary.
The output provides information on which criteria were met, which were not, and a list of doubtful cases. This allows the user to quickly decide on the further processing of the application.
Initial conclusions from the PoC are promising and suggest that artificial intelligence is applicable in such a project. ChatGPT 4.0 achieved an accuracy of over 90% in assessing criteria for the provided test data.
We thus positively evaluate the PoC and see significant developmental potential in it.
