.jpeg)
RootPerks is a tool developed by Rootstrap for internal use. It allows employees to upload photos of invoices and its corresponding data for refund requests, which could be for the lunch program, gym pass, travel expenses, and more.
We figured it could be a bit tedious to fill every time all the necessary fields, which are actually in the ticket itself. So, it came to mind ‘why not just upload the voucher photo and the form completes on its own?’
Here is where AI and Text recognition come to the rescue in order to identify all the corresponding items for the form and pre-fill it. This article will explain in general terms how it was implemented in the iOS App and the results are amazing!
Identify the text from an image
To accomplish this we employed Apple's Vision framework for text recognition in images, enabling RootPerks to extract text from uploaded invoice photos accurately. As is shown below the implementation was straightforward and its output very accurate:
Given the following ticket of a double expresso as an example, this is the result:
.png)
Ask OpenAI to identify the parameters
After extracting text from the uploaded invoice using the Vision framework, the next step in the process is to ask the OpenAI API to identify specific parameters required for the refund request. In this case, we used the GPT-3.5-turbo model and crafted a prompt specifying the parameters needed, including categories (from a specific list of options), currency, amount, and date.
The prompt also included the list of words extracted from the text recognizer to provide context to the AI. Additionally, we specified a specific format for the AI's response to make it easier to extract and pre-fill the refund request form programmatically.
This step is crucial in automating the refund request process, as it allows the AI to understand the content of the invoice and extract relevant information accurately. By leveraging the capabilities of the OpenAI API, RootPerks can ensure that the refund request forms are pre-filled with the correct information, saving time and reducing the likelihood of errors.
Once the AI identifies the items, pre-filling the forms becomes straightforward.
Pros and Cons
After extensive testing with a wide range of tickets, RootPerks has demonstrated several pros and cons.
Pros:
- Fast Response: RootPerks delivers rapid responses, enhancing efficiency for users.
- Adaptability: The tool can adjust to various scenarios, making it suitable for different types of refund requests.
Cons:
- Confusion with Total Amount: Occasionally, the AI struggles with recognizing the total amount on invoices. This issue can be mitigated by providing clearer prompts.
- Testing Required: To ensure accuracy, the tool requires testing with different types of invoices, such as travel or gym passes.
Conclusion
RootPerks represents a significant step forward in automating mundane tasks and improving efficiency in the workplace. By leveraging advanced technologies like Vision and the OpenAI API, Rootstrap has created a tool that not only saves time but also reduces the likelihood of errors in the refund request process. As technology continues to advance, tools like RootPerks will become increasingly valuable in streamlining business processes and empowering employees to focus on more meaningful work.
