Home A - Z
Page Index

Audit-matic

Project: AuditMatic

Tagline: Automating CMMI compliance by bridging the gap between business requirements and development artifacts.

Team: https://wiki.enovate-it.com/Teams/Team%20Fusion

Project Status: Development/Scaling (Actively implementing the RAG pipeline for Content Validation)

Overview

AuditMatic is an AI-powered system designed to automate the CMMI compliance auditing process. For many development teams, manually ensuring engineering work aligns with business requirements is slow, error-prone, and resource-intensive. AuditMatic solves this problem by intelligently bridging the gap between business documents and development tools such as Jira and GitLab.

The system ingests unstructured business requirement documents, uses NLP to extract key identifiers like ticket numbers and commit hashes, and cross-validates this data with live project data. This automated approach produces detailed compliance reports in minutes, saving time and reducing human error.

By combining off-the-shelf NLP models with a custom rule-based expert system, AuditMatic transforms a manual, high-effort audit process into a fast, reliable, and AI-driven pipeline.

The Problem

CMMI compliance auditing requires auditors to manually verify alignment and content across complex business documents and development tools. This process is:

  • Slow, costly, and resource-intensive (Phase 1 focus).
  • Prone to human error and subjective interpretation (Phase 2 focus).
  • Scales poorly across large organizations with diverse toolsets (Phase 3 focus).

Our AI Solution

What does it do?

AuditMatic transforms the audit process into a fast, reliable, and AI-driven pipeline by integrating:

  • Phase 1: Automated document existence and basic entity checks.
  • Phase 2: Deep Content Validation using the RAG Pipeline and the Gemini LLM.
  • Phase 3: Cross-Platform Ecosystem Coverage (Jira, GitLab, Bitbucket) for universal traceability.
How does it work?

The architecture has three main components:

  1. React Web App:

    • User interface for uploading documents.

  2. Express.js Backend:

    • Receives the file and triggers the AI pipeline.
What makes it innovative?

AuditMatic replaces a manual audit process with an automated AI-driven pipeline. It combines NLP, API integration, and rule-based logic to solve a high-value business problem efficiently.

Technology Stack

  • AI/ML Models: Gemini API (gemini-2.5-flash, embedding-001), OpenAI API (gpt-4o-mini, text-embedding-3-large), Xenova/Transformers (Local Embeddings), Custom Rule-Based Expert System, spaCy (Pre-processing)

  • LLM: Gemini (Google) and GPT (OpenAI), managed via a multi-LLM Strategy Pattern.

  • Frameworks & Libraries: jira, multer

  • Backend: Node.js v24.x with Express.js v5.1.0 (Built with TypeScript), ts-node, nodemon.

  • Frontend: React v18.2.0 (Built with TypeScript), Redux Toolkit (RTK) (State Management).

  • Databases: PostgreSQL 14+ (Primary DB), pgvector v0.2.1 (Vector Search Extension), Redis v5.9.0 (Caching, Session Mgmt).

  • Deployment & Tools: Git, GitHub, Docker, Jest (Testing), ESLint (Linting).

  • APIs Used: Jira API (Phase 3), GitLab API (Phase 3), Bitbucket API (Phase 3), Google Drive API, Google OAuth 2.0 API.

Project Architecture

Challenges & Learnings

Challenges:
  • Creating robust NLP patterns to handle document variations.
  • Ensuring smooth orchestration between React, Node.js.
  • Handling API rate limits and error management for Jira and GitLab.
Learnings:
  • Integrating NLP with rule-based expert systems.
  • Handling unstructured data efficiently.
  • Building a polyglot microservice architecture.

Future Roadmap

Phase 1: Foundation & Entity Extraction (Completed)

  • Goal: Prove the concept of automation by replacing manual document checks.

  • Key Achievement: Implemented basic checks for document existence using simple Regex and spaCy for entity extraction.

Phase 2: Content Validation & AI Pivot (Current Focus)

  • Goal: Move from "existence" to "content validation."

  • Key Achievement: Developed and implemented the RAG pipeline and Multi-LLM Strategy to semantically validate document content against live data.

Phase 3: Ecosystem Expansion & Automation (Future Roadmap)

  • Short-term Goal (P3): Integrate the Bitbucket API for unified code traceability.

  • Medium-term Goal (P3): Implement AI-driven Gap Filling (LLM suggests missing content or remediation steps).

  • Long-term Goal (P3): ML-based Document Categorization for dynamic rule application across different document types.

Repository & Live Demo

GitHub Repository: https://gitlab.enovate-it.com/enovate/hackathon/audit-matic

Live Demo: https://demo4.enovate-it.com/

Screenshots / Demo Video

Figma link: https://www.figma.com/make/hgPLDKHEFYH3ratyLlUx96/Create-AI-Design-Prompt?node-id=0-1&p=f&t=cC5O3FcQiEGFsTDr-0&fullscreen=1

Categories: #project #ai-hackathon-2025 #category-compliance #category-ai