In today’s fast paced utility management industry, efficiency and accuracy are paramount. The volume and complexity of data processed daily requires tools that streamline workflows and minimise the risk of human error. At udintel, we have always been committed to driving innovation that supports our clients.
“By embedding AI into our data parsing solutions, we are taking a bold step towards fully automated data management with a focus on reducing human intervention and errors in handling large data files..” Steven Xi, CTO
The challenge: complex data parsing across multiple formats
Historically, our platform has offered advanced data parsing capabilities, allowing our clients to process data in various formats, including EDI (Electronic Data Interchange), MM (Market Message), XML (Extensible Markup Language), and PDF (Portable Document Files). Parsing these data formats manually or even semi automatically required significant human input and brought a degree of risk.
While our solutions already provided a robust level of automation, the need for human verification could sometimes lead to errors and delays in processing, potentially causing delays in SLAs. Our new AI-driven system was developed to tackle these challenges by providing a more sophisticated, intelligent parsing solution. With machine learning algorithms trained on vast datasets, this technology can automatically recognize patterns, detect anomalies, and accurately parse data across all supported formats.
Key benefits of AI, driven data parsing
The introduction of AI brings two significant advantages: a reduction in manual data handling and a notable decrease in error rates. In detail:
How AI Enhances Our Data Parsing Capabilities
The integration of AI into our platform enables dynamic parsing, where the technology learns and improves with each dataset processed. Our AI algorithms adapt to new data patterns over time, refining their accuracy and reducing the need for human intervention. This adaptability makes our platform a truly future-proof solution, prepared to handle the ever-evolving landscape of data in the utility sector. Additionally, our AI technology supports real-time data parsing, meaning that information from EDI, MM, XML, and PDF files can be read, parsed, and integrated into client dashboards in near real-time. This feature is invaluable for clients looking to make quick, data-driven decisions and maintain a competitive edge.
Case Study: AI in Action
As part of our services, we are required to parse over 110 different invoice sources coming in various formats, EDI, MM, XML, and PDF. The complexity in processing them is related to the different technologies utilised to produce these files and frequent manual intervention from suppliers to customise output or deploy patches, as consuequence of contractual or sector changes. From SSE new XML format to Total Energies EDI or XLSX monthly changes, to EDF MM files, we applied our algorithms to current and historical data to around 1,250,000 invoices across 150,000 feeds with a team 4 people supporting dataset analysis and AI feedback across 3 months period. The data quality received and possible bias related to manual intervention from suppliers has been re-factured a number of times as well as we took in consideration the possible model drift, which seem to be a fundamental issue for this project.
when a supplier changes it's invoicing approach, the entire analysis becomes biased as it was based on a dataset not anymore accurate, and the accuracy of the results starts decreasing sensiblyThe results obtained have been compared to our historical machine learning (ML) approach and they showed some interesting results:
• AI driven activities are 15% more accurate than machine learning
• AI and ML are saving up to 95% of user time
This said, the maintenance cost for updating algorithms and identify model drifting issues, is justified only if there it a large database and a dedicated structure aligned to the data processing team.
OUR CONCLUSION
from our experience we believe the dataset required for a more robust model should be at least 10x what we analysed so far. A bigger business scale would justify economically also higher investments on AI modelling.