AI Predicts Software Delays Before They Happen

In software development, missed deadlines can derail projects and inflate costs, but a new system called CodeSight uses artificial intelligence to forecast delays early, helping teams stay on track. Developed by researchers at Inverbis Analytics, this approach analyzes GitHub activity logs to predict whether pull requests—key steps in code integration—will meet their deadlines, offering a proactive tool for managing complex workflows. By combining process mining with deep learning, CodeSight identifies inefficiencies and provides real-time insights, moving beyond traditional metrics to address the root causes of delays in software delivery.

The core finding is that CodeSight can accurately predict deadline compliance for pull requests, with an F1-score of 0.963 on test data, meaning it correctly identifies compliant and non-compliant cases with high precision. The system estimates the remaining time to complete a pull request, and when tested, it achieved a mean absolute error of about 8.8 hours—roughly one workday—in predicting resolution times. This allows teams to spot potential breaches early, such as when a task might exceed its allotted time, enabling interventions before deadlines are missed.

Methodologically, the researchers built CodeSight as an end-to-end system that collects data from GitHub repositories via its REST API, including pull requests, commits, and workflow runs like CI/CD executions. This raw data is transformed into structured event logs, where each pull request is treated as a case with a sequence of activities, such as opening, commits, and merging. Process mining techniques then extract patterns and bottlenecks from these logs, visualizing workflows to show variations and inefficiencies. For prediction, the system uses a Long Short-Term Memory (LSTM) neural network, which processes sequential activity data along with static features like branch types and file modifications to forecast remaining durations.

Results from the paper show that the LSTM model explains up to 93% of the variance in training data and 78% in test data for time predictions, indicating strong performance in capturing workflow dynamics. In classification tasks, the model's accuracy for deadline compliance reached 0.944 on the test set, with high precision and recall, meaning it rarely mislabels compliant pull requests. For instance, in tests, it correctly identified 91 true compliant cases and 28 true non-compliant ones, with only a few errors, demonstrating its reliability in real-world scenarios. The system also uncovered substantial variability in workflows, with 271 different process variants identified, highlighting the complexity of software development paths.

This innovation matters because it empowers organizations to move from reactive to proactive project management, reducing delays and improving efficiency in DevOps environments. By integrating with tools like Power BI dashboards, CodeSight provides teams with actionable insights, such as early alerts for deadline risks and workload rebalancing suggestions. In practical terms, this could mean faster software releases, lower costs, and enhanced collaboration, as developers gain a clearer view of their workflows without relying solely on high-level metrics like those from the DORA framework.

However, the paper notes limitations, including CodeSight's current reliance on GitHub data, which may not generalize to other platforms like GitLab or Bitbucket without adaptation. Additionally, workflows can vary significantly between teams due to differing branching strategies or labeling conventions, posing challenges for standardization. Future work aims to expand integration to multiple services, automate log mapping, and add explainability features to make the AI predictions more interpretable for industrial use.