Project: CommNet

“The AI-driven network for client communication.”

About Team : https://wiki.enovate-it.com/Teams/Team%20Spark

Project Description

The Client Communication Management Application is a multi-tenant platform designed to help businesses manage customer interactions through AI-powered communication channels.
It allows each client to onboard their projects, configure communication channels (Telegram, Email, etc.), integrate with preferred AI models (ChatGPT, Gemini, etc.), and leverage a knowledge base for context-aware responses.

The application provides:

• A dedicated client dashboard for project setup, knowledge base upload, and communication channel configuration.
• A customer-facing AI interface (chat page, Telegram bot, or email integration) where end-users can interact with the AI assistant.
• An admin dashboard to monitor, review, and override AI responses when necessary.

Uploaded project documents and external resources are indexed in a knowledge base powered by vector embeddings (Postgres pgvector or external vector DB). This enables retrieval-augmented generation (RAG), ensuring AI responses are accurate and project-specific.

The platform supports multiple AI connectors, enabling flexibility for businesses to choose the best model for their needs. Clients can seamlessly manage communication, while administrators maintain oversight and control over all conversations.

The system is built on a Python backend (FastAPI), PostgreSQL database, and integrates with external APIs for AI, messaging, email, and storage services.

System Architecture

Frontend

• UI: ShadcnUI (TailwindCSS + RadixUI)

• Build Tool: Vite

• Routing: TanStack Router

• Type Checking: TypeScript

• Linting/Formatting: Eslint & Prettier

• Icons: Lucide Icons, Tabler Icons (Brand icons only)

• Client dashboard: onboarding, project setup, knowledge base upload, channel configuration

• Customer-facing AI bot interface: chat page embedded in client’s site, Telegram, email, etc.

• Admin dashboard: monitor conversations, override responses

Backend (Go)

• Core framework: Go (Golang) using a modular, clean architecture pattern (e.g., layered or hexagonal).
• API layer: Standard Go HTTP server with chi or Echo for routing.
• Authentication & Accounts: JWT-based multi-tenant authentication.
• Project & Channel Management: Business logic for workspaces, channels, and configurations.
• Message Orchestration: Handles incoming messages, context injection, AI processing pipeline, and logging.
• Knowledge Base Integration: Document ingestion, indexing, and retrieval pipeline in Go.

Database

• PostgreSQL: for structured data (users, projects, configs, logs)
• Vector DB: pgvector

AI / NLP

• Connector: OpenAI API (Gemini), Perplexity API, and pluggable LLMs
• Pre-/Post-processing pipeline: inject context, system prompts, etc.
• Retrieval Augmented Generation (RAG): option to fine-tune or use RAG with knowledge base

Communication Integrations

• Telegram Bot API: customer chats
• Email Gateway: SMTP + IMAP, or services like SendGrid
• Optional: WhatsApp Business API, Slack, Web widget

File Handling & Knowledge Base

• File parser: PDF, DOCX, TXT
• Cloud storage integration: Google Drive/Dropbox API for linked knowledge sources
• Embeddings generator: OpenAI embeddings, HuggingFace models
• Indexing service: store embeddings into Vector DB

Infrastructure

• Cloud deployment: AWS
• Containerization: Docker
• Background task runner: Celery / RQ for async jobs (e.g., doc parsing)
• Caching: Redis for messages and sessions
• CI/CD: GITLAB

System Architecutre Diagram

1. Tech Stack

• Backend (Go backend)
  • The backend is implemented in Go, providing a performant and scalable foundation for multi-tenant communication workflows.
• Database
  • PostgreSQL (perfectly suited for multi-tenant, relational data storage).
• Frontend
  • ShadcnUI , vite
  • Admin dashboard + Client dashboard.
• AI/ML Integrations
  • OpenAI API / Perplexity API / Custom AI connectors.
  • Knowledge Base Indexing (e.g., LangChain + Vector DB like Postgres pgvector).
  • MCP Server (for tool orchestration + context passing).
• Communication Channels
  • Telegram Bot API integration.
  • Email integration (IMAP/SMTP or services like SendGrid/Mailgun).
• Infrastructure
  • Docker for containerization.
  • Nginx / Traefik for reverse proxy.
  • Optional: Kubernetes for scaling.

2. Core Modules & Features

🔹 Client Onboarding

• Form to capture client + project details.
• Generate dedicated project page for embedding AI bot.
• Multi-tenant user management (each client = tenant).

🔹 Communication Channels

• Configurable options for each client:
  • Telegram bot setup.
  • Email inbox setup.
  • Future: WhatsApp, Slack, etc.

🔹 AI Model Connector

• Ability to select which AI model:
  • ChatGPT (OpenAI).
  • Perplexity.
  • Others (pluggable architecture).
• Standardized response pipeline regardless of model.

🔹 Knowledge Base

• Upload project docs (PDF, DOCX, TXT).
• Import from Drive links / Notion / Confluence.
• Store embeddings in pgvector (Postgres extension).
• Retrieval-Augmented Generation (RAG) for query responses.

🔹 Query Processing

• Customer submits query → AI pipeline:
  1. Normalize input.
  2. Retrieve context from Knowledge Base.
  3. Feed context + query to chosen AI model.
  4. Return AI response.
• Option for admin override & monitoring.

🔹 Monitoring & Admin Portal

• View all conversations per client/project.
• Override/edit AI responses.
• Track usage per channel/model.
• Analytics & logs.

Dataflow Diagram

3. Database Design (PostgreSQL)

Tables

• accounts
  Stores client/organization information.
  Fields:

  • id (string, PK)
  • name (varchar(150))
  • domain (varchar(100))
  • description (text)

• workspaces
  Stores project metadata linked to an account.
  Fields:

  • id (string, PK)
  • account_id (string, FK → accounts.id)
  • name (varchar(150))
  • description (text)

• users
  Stores basic user profile information.
  Fields:

  • id (string, PK)
  • username (varchar(50))
  • full_name (varchar(150))
  • email (varchar(150))
  • phone_number (varchar(20))
  • password_hash (varchar(255))
  • auth_token (varchar(500))

• user_accounts
  Maps users to accounts and optionally workspaces; stores roles and status.
  Fields:

  • id (string, PK)
  • user_id (string, FK → users.id)
  • account_id (string, FK → accounts.id)
  • workspace_id (string, nullable, FK → workspaces.id)
  • role (varchar(50))
  • status (varchar(50))

• workspace_users
  Maps users to workspaces for workspace-specific roles/status.
  Fields:

  • id (string, PK)
  • user_id (string, FK → users.id)
  • workspace_id (string, FK → workspaces.id)
  • role (varchar(50))
  • status (varchar(50))

• groups
  Represents functional teams or user groups within a workspace.
  Fields:

  • id (string, PK)
  • workspace_id (string, FK → workspaces.id)
  • name (varchar(100))
  • description (text)

• group_users
  Maps users to groups.
  Fields:

  • id (string, PK)
  • group_id (string, FK → groups.id)
  • user_id (string, FK → users.id)
  • role (varchar(50))

• channels
  Configured communication channels within a workspace.
  Fields:

  • id (string, PK)
  • workspace_id (string, FK → workspaces.id)
  • type (varchar(50))
  • config (jsonb)
  • description (text)

• ai_models
  Available AI model connectors for accounts.
  Fields:

  • id (string, PK)
  • account_id (string, FK → accounts.id)
  • name (varchar(100))
  • ai_type (varchar(50))
  • version (varchar(50))
  • config (jsonb)
  • description (text)

• ai_agents
  Agents linked to a workspace and a single channel; connect to AI models.
  Fields:

  • id (string, PK)
  • workspace_id (string, FK → workspaces.id)
  • channel_id (string, unique, FK → channels.id)
  • ai_model_id (string, FK → ai_models.id)
  • name (varchar(100))
  • status (varchar(50))
  • description (text)

• mcp_connections
  Stores Message Coordination Processor connection configuration.
  Fields:

  • id (string, PK)
  • workspace_id (string, FK → workspaces.id)
  • type (varchar(50))
  • config (jsonb)
  • description (text)

• mcp_logs
  Stores request and response logs for MCP connections.
  Fields:

  • id (string, PK)
  • mcp_connection_id (string, FK → mcp_connections.id)
  • request_data (jsonb)
  • response_data (jsonb)
  • status (varchar(50))
  • log_timestamp (timestamp)

• knowledge_bases
  Stores knowledge base entries linked to workspaces.
  Fields:

  • id (string, PK)
  • workspace_id (string, FK → workspaces.id)
  • title (varchar(200))
  • description (text)

• kb_files
  Stores files uploaded to a knowledge base.
  Fields:

  • id (string, PK)
  • knowledge_base_id (string, FK → knowledge_bases.id)
  • workspace_id (string, FK → workspaces.id)
  • user_id (string, FK → users.id)
  • file_name (varchar(255))
  • file_path (text)
  • mime_type (varchar(100))
  • file_size (bigint)

• kb_file_chunks
  Stores content chunks for each knowledge base file, along with vector embeddings.
  Fields:

  • id (string, PK)
  • kb_file_id (string, FK → kb_files.id)
  • chunk_index (int)
  • content (text)
  • vector_embedding (text)

• tasks
  Tracks tasks created in a workspace (optionally linked to a conversation).
  Fields:

  • id (string, PK)
  • workspace_id (string, FK → workspaces.id)
  • agent_id (string, FK → ai_agents.id)
  • user_id (string, FK → users.id)
  • conversation_id (string, nullable, FK → conversations.id)
  • title (varchar(200))
  • description (text)
  • status (varchar(50))

• task_activity
  Logs actions and updates performed on tasks.
  Fields:

  • id (string, PK)
  • task_id (string, FK → tasks.id)
  • action (varchar(100))
  • performed_by (string)
  • details (jsonb)
  • activity_timestamp (timestamp)

• conversations
  Stores conversation metadata within a workspace.
  Fields:

  • id (string, PK)
  • workspace_id (string, FK → workspaces.id)
  • channel_id (string, FK → channels.id)
  • agent_id (string, FK → ai_agents.id)
  • title (varchar(200))
  • status (varchar(50))

• messages
  Stores messages in a conversation, including prompts and memory/archive info.
  Fields:

  • id (string, PK)
  • conversation_id (string, FK → conversations.id)
  • user_id (string, FK → users.id)
  • sender_type (varchar(50))
  • prompt_id (string, nullable)
  • content (text)
  • is_memory_archived (bool)
  • metadata (jsonb)

• memories
  Stores memory entries for agents within a conversation.
  Fields:

  • id (string, PK)
  • agent_id (string, FK → ai_agents.id)
  • conversation_id (string, FK → conversations.id)
  • memory_type (varchar(50))
  • content (text)

• invites
  Stores workspace invitation records for users.
  Fields:

  • id (string, PK)
  • workspace_id (string, FK → workspaces.id)
  • email (varchar(150))
  • role (varchar(50))
  • token (varchar(255))
  • status (varchar(50))
  • invited_by (string, FK → users.id)
  • expires_at (timestamp)

4. External Integrations

• Telegram Bot API → Webhook for messages.
• Email API → IMAP/SMTP or SendGrid.
• AI APIs → OpenAI, Perplexity, etc.
• Storage → AWS S3 / GCP Storage for uploaded files.
• Drive APIs → Google Drive API (for fetching docs).
• Authentication → JWT or OAuth2 (multi-tenant).

5. Deployment & Scaling

• Containerize with Docker.
• CI/CD (GitHub Actions / GitLab CI).
• Cloud hosting (AWS/GCP/Azure/DigitalOcean).
• Use Postgres + pgvector for persistence + semantic search.
• Caching layer (Redis) for session storage & rate limiting.

Sequence Diagram - Click to View

sequenceDiagram
    %% Participants
    participant ClientUser as Client (Registers on Comnet)
    participant Comnet as Comnet System
    participant Workspace as Workspace (Project)
    participant WorkspaceOwner as Workspace Owner (ClientUser Role)
    participant ChannelConfig as Channel Configuration
    participant Channel as Communication Channel (Email/Telegram/SMS)
    participant Agent as Channel Agent
    participant Customer as Customer (Client's customer)
    participant MCP as MCP Server (Message Coordination Processor)
    participant AI as AI Model (ChatGPT/Gemini)
    participant KB as Knowledge Base
    participant Task as Task Management Module
    participant Group as Group (Managers & Members)
    participant Member as Group Member

    %% Registration and Workspace Creation by Client (Workspace Owner)
    ClientUser->>Comnet: Register account
    Comnet-->>ClientUser: Account created
    ClientUser->>Workspace: Create new workspace (project)
    Workspace-->>ClientUser: Workspace created
    ClientUser->>WorkspaceOwner: Become workspace owner

    %% Workspace Setup & Channel Configuration (done by workspace owner)

    WorkspaceOwner->>Workspace: Create groups and assign roles (Manager / Member)
    WorkspaceOwner->>ChannelConfig: Add and configure channels (Email / Telegram / SMS)
    ChannelConfig-->>Workspace: Channels registered and linked to workspace
    ChannelConfig->>Agent: Provision Channel Agents for each channel
    Agent-->>Workspace: Channel Agents ready and bound to workspace

    %% Normal flow: Customer raises query on a configured channel
    Customer->>Channel: Send message/query (via configured channel)
    Channel->>Agent: Channel Agent receives and forwards message
    Agent->>MCP: Forward message + workspace context (client id, channel id, conversation history)

    %% AI processing using KB and workspace context
    MCP->>AI: Send message + workspace context + KB pointers
    AI->>KB: Request relevant KB entries and workspace-specific context
    KB-->>AI: Return KB results and context
    AI-->>MCP: Return generated response + intent detection (actionable? yes/no) + confidence

    %% Conditional: If AI detects actionable intent -> create task in same workspace and assign to group
    alt Actionable intent detected
        MCP->>Task: Create Task in Workspace (details, priority, assignee group)
        Task->>Group: Assign task to specific group or manager
        Group-->>Task: Acknowledgement of task assignment
        Task->>Workspace: Log task creation and link it to conversation
        Workspace-->>WorkspaceOwner: Notify workspace members (or assigned group) of new task
    end

    %% Response delivery back to customer via same channel
    MCP-->>Agent: Return AI-generated reply (and task info if created)
    Agent-->>Channel: Send reply via originating channel
    Channel-->>Customer: Deliver AI response to customer

    %% Group resolves task (if created)
    alt Task exists
        Member->>Task: View task and start work
        Member->>Group: Update task status / add comments / attach files
        Group-->>Task: Mark task progress or resolution
        Task-->>Workspace: Update task log and notify conversation thread
        Workspace-->>Customer: Optionally send status update (via Channel) if configured
    end

    %% Logging, KB updates and analytics (always)
    Agent->>Workspace: Log conversation and response metadata
    AI->>Workspace: (optional) Suggest KB update based on conversation
    Workspace->>KB: Update KB (optional manual review or auto-suggest)
    Workspace-->>WorkspaceOwner: Update analytics / reports / conversation history

    %% Ongoing conversation loop
    Customer-->>Channel: Follow-up messages (loop continues)
    Channel->>Agent: Forward follow-ups
    Agent->>MCP: Repeat AI processing with updated context

Future Scope

The platform is designed with scalability in mind, allowing integration of new features and channels to enhance client communication and management. Planned future improvements include:

1. Additional Communication Channels

• Meeting Integrations

  • Integration with platforms like Zoom, Google Meet, and Microsoft Teams.
  • AI-powered meeting notes, summaries, and action items.

• Phone Call Channels

  • Integration with telephony services (e.g., Twilio, Vonage).
  • AI-powered call transcription, sentiment analysis, and automated follow-up responses.

2. Ticketing & Workflow Management

• Jira / Trello / Asana Integrations

  • Automatic ticket creation from client queries.
  • Linking AI conversations to tasks and workflows.
  • Syncing client issues directly into project management tools.

• Internal Ticketing System

  • Native ticket creation and tracking within the platform.
  • Escalation workflows and admin assignment for unresolved queries.

3. Advanced AI Features

• Sentiment & Intent Analysis

  • Understanding customer tone and urgency for better prioritization.

• Personalized AI Responses

  • AI models trained on client-specific style guides and communication tone.

4. Analytics & Insights

• Advanced reporting on client queries, response times, and AI performance.
• Predictive analytics for identifying recurring issues or potential churn risks.
• Exportable insights for stakeholders.

5. Enterprise Readiness

• Multi-language support for global clients.
• Role-based access control with fine-grained permissions.
• Audit logging for compliance and security.

🔸 Scope of Demo 1

The first demo of CommNet establishes the core multi-tenant communication foundation and essential management modules.

✅ Completed in Demo 1

🧩 Client Onboarding & Authentication

• Registration, login, and onboarding workflows for businesses.
• Multi-tenant user and organization setup.

🤖 AI Model Management

• Create, update, and manage AI connectors.
• Support for pluggable AI model configurations.

🏢 Workspace Management

• Create and manage workspaces under business accounts.
• Link AI models to specific workspaces.

⚙️ MCP Server Management

• Configure and manage MCP connections.
• Store, validate, and monitor MCP configurations.

💬 Communication Channel (Telegram)

• Telegram bot setup and configuration as the first supported channel.
• Webhook-based message handling integrated with MCP and AI pipelines.

🧠 AI Agent Management

• Manage AI agents linked to workspaces and channels.
• Configure model bindings, activation states, and AI routing logic.

🔮 Future Scope

A. Functional Enhancements

📚 Knowledge Base (KB) Management

• Upload, parse, and index documents (PDF, DOCX, TXT).
• Store embeddings using pgvector for contextual RAG responses.
• Support integration with Google Drive and other data sources.

💬 Conversation & Message Flow

• End-to-end conversation orchestration between channels, MCP, and AI models.
• Store conversation history, enable admin override, and manage AI memory.

📋 Task & Group Management

• Auto-create tasks from actionable AI intents.
• Assign and track tasks within workspace groups.
• Provide activity logs and notifications.

🌐 Additional Communication Channels

• Extend support to Email (IMAP/SMTP, SendGrid) and WhatsApp Business API.
• Standardized message orchestration across all channels.

📊 Admin & Analytics Dashboard

• Comprehensive dashboard for monitoring usage, AI performance, and channel analytics.
• Reports for workspace activity, AI accuracy, and user engagement.

B. Non-Functional Enhancements

🚀 Scalability & Performance

• Optimize backend services for high concurrency using Go routines.
• Container orchestration via Docker/Kubernetes for horizontal scaling.

🔒 Security & Compliance

• Role-Based Access Control (RBAC).
• JWT/OAuth2 authentication with tenant isolation.
• Data encryption at rest and in transit.
• Audit logs for compliance (GDPR-ready).

⚙️ Reliability & Observability

• Centralized logging using Grafana + Loki + Prometheus.
• Real-time error tracking and health checks.
• Backup and disaster recovery mechanisms.

🎨 User Experience

• Improved UI using ShadcnUI components.
• Responsive layouts and faster navigation with Vite + TanStack Router.