# Project: CommNet

> **“The AI-driven network for client communication.”**

---


**About Team : **
[[https://wiki.enovate-it.com/Teams/Team%20Spark]]

# Project Description

The **Client Communication Management Application** is a multi-tenant platform designed to help businesses manage customer interactions through AI-powered communication channels.  
It allows each client to onboard their projects, configure communication channels (Telegram, Email, etc.), integrate with preferred AI models (ChatGPT, Gemini, etc.), and leverage a knowledge base for context-aware responses.

The application provides:
- A dedicated **client dashboard** for project setup, knowledge base upload, and communication channel configuration.
- A **customer-facing AI interface** (chat page, Telegram bot, or email integration) where end-users can interact with the AI assistant.
- An **admin dashboard** to monitor, review, and override AI responses when necessary.

Uploaded project documents and external resources are indexed in a knowledge base powered by vector embeddings (Postgres `pgvector`). This enables retrieval-augmented generation (RAG), ensuring AI responses are accurate and project-specific.  

The platform supports multiple AI connectors, enabling flexibility for businesses to choose the best model for their needs. Clients can seamlessly manage communication, while administrators maintain oversight and control over all conversations.

The system is built on a Go lang backend, PostgreSQL database, and integrates with external APIs for AI, messaging, email, and storage services.  


# System Architecture

## Frontend

- **UI:** React

- **Build Tool:** Vite

- **Routing:** TanStack Router

- **Type Checking:** TypeScript

- **Linting/Formatting:** Eslint & Prettier

- **Icons:** Lucide Icons, Tabler Icons (Brand icons only)

- **Client dashboard**: onboarding, project setup, knowledge base upload, channel configuration  
- **Customer-facing AI bot interface**: chat page embedded in client’s site, Telegram, email, etc.  
 

## Backend (Go)
- **Core framework**: Go (Golang) using a modular, clean architecture pattern.
- **API layer: Standard** Go HTTP server with Gin for routing.
- **Authentication & Accounts**: JWT-based multi-tenant authentication.
- **Project & Channel Management**: Business logic for workspaces, channels, and configurations.
- **Message Orchestration**: Handles incoming messages, context injection, AI processing pipeline, and logging.
- **Knowledge Base Integration**: Document ingestion, indexing, and retrieval pipeline in Go.


## Database
- **PostgreSQL**: for structured data (users, projects, configs, logs)  
- **Vector DB**: pgvector

## AI / NLP
- **Connector**: OpenAI API (Gemini)  
- **Pre-/Post-processing pipeline**: inject context, system prompts, etc.  
- **Retrieval Augmented Generation (RAG)**: option to fine-tune or use RAG with knowledge base  

## Communication Integrations
- **Telegram Bot API**: customer chats  
- **Email Gateway**: SMTP + IMAP, or services like SendGrid    

## File Handling & Knowledge Base
- **File parser**: PDF, DOCX, TXT  
- **Cloud storage integration**: Google Drive/Dropbox API for linked knowledge sources  
- **Embeddings generator**: OLLAMA_MODEL(bge-m3)  
- **Indexing service**: store embeddings into Vector DB  

## Infrastructure
- **Cloud deployment**: AWS 
- **Containerization**: Docker  
- **Caching**: Redis for messages and sessions
- **CI/CD**: GITLAB

## System Architecutre Diagram
![](./System_Architecture.png)

## 1. Tech Stack
- **Backend (Go backend)**
  - The backend is implemented in Go, providing a performant and scalable foundation for multi-tenant communication workflows.
- **Database**
  - **PostgreSQL** (perfectly suited for multi-tenant, relational data storage).
- **Frontend**
  - React , vite
  - Admin dashboard + Client dashboard.
- **AI/ML Integrations**
  - OpenAI API.
  - Knowledge Base Indexing (e.g., **LangChain + Vector DB** like **Postgres pgvector**).
  - MCP Server (for tool orchestration + context passing).
- **Communication Channels**
  - **Telegram Bot API** integration.
  - **Email integration** (IMAP/SMTP or services like SendGrid/Mailgun).
- **Infrastructure**
  - Docker for containerization.
  - Nginx / Traefik for reverse proxy.
  - Optional: Kubernetes for scaling.

---

## 2. Core Modules & Features

### 🔹 Client Onboarding

- Form to capture client + project details.
- Generate dedicated project page for embedding AI bot.
- Multi-tenant user management (each client = tenant).

### 🔹 Communication Channels
- Configurable options for each client:
  - Telegram bot setup.
  - Email inbox setup.
  - Future: Slack.

### 🔹 AI Model Connector
- Ability to select which AI model:
  - ChatGPT (OpenAI).
  - Gemini
  - Ollama.
- Standardized response pipeline regardless of model.

### 🔹 Knowledge Base
- Upload project docs (PDF, DOCX, TXT).
- Import from Drive links / Notion / Confluence.
- Store embeddings in **pgvector** (Postgres extension).
- Retrieval-Augmented Generation (RAG) for query responses.
- 
### 🔹 MCP (Model Context Protocol)
- Centralized system for managing **connections between AI agents and models**.  
- Handles authentication, routing, and configuration for both **local** and **remote** models.  
- Decouples application logic from model APIs, enabling **dynamic model switching**.  
- Standardizes inference requests and responses for consistent performance.  
- Forms the foundation for future **multi-model orchestration** and **context-aware AI pipelines**.
![](./TRD.PNG)
![](./MCP.PNG)
![](./JSONDB.PNG)

### 🔹 Query Processing
- Customer submits query → AI pipeline:
  1. Normalize input.
  2. Retrieve context from Knowledge Base.
  3. Feed context + query to chosen AI model.
  4. Return AI response.
- Option for admin override & monitoring.

### 🔹 Monitoring & Admin Portal
- View all conversations per client/project.
- Override/edit AI responses.
- Track usage per channel/model.
- Analytics & logs.

### RAG system- 
![](./KnowledgebSE.PNG)


## Dataflow Diagram
![Dataflow Diagram](./Comm_1.png)

---


## 3. Database Design (PostgreSQL)

### Tables

- **`accounts`**  
  Stores client/organization information.  
  **Fields:**  
  - `id` (string, PK)  
  - `name` (varchar(150))  
  - `domain` (varchar(100))  
  - `description` (text)  

- **`workspaces`**  
  Stores project metadata linked to an account.  
  **Fields:**  
  - `id` (string, PK)  
  - `account_id` (string, FK → accounts.id)  
  - `name` (varchar(150))  
  - `description` (text)  

- **`users`**  
  Stores basic user profile information.  
  **Fields:**  
  - `id` (string, PK)  
  - `username` (varchar(50))  
  - `full_name` (varchar(150))  
  - `email` (varchar(150))  
  - `phone_number` (varchar(20))  
  - `password_hash` (varchar(255))  
  - `auth_token` (varchar(500))  

- **`user_accounts`**  
  Maps users to accounts and optionally workspaces; stores roles and status.  
  **Fields:**  
  - `id` (string, PK)  
  - `user_id` (string, FK → users.id)  
  - `account_id` (string, FK → accounts.id)  
  - `workspace_id` (string, nullable, FK → workspaces.id)  
  - `role` (varchar(50))  
  - `status` (varchar(50))  

- **`workspace_users`**  
  Maps users to workspaces for workspace-specific roles/status.  
  **Fields:**  
  - `id` (string, PK)  
  - `user_id` (string, FK → users.id)  
  - `workspace_id` (string, FK → workspaces.id)  
  - `role` (varchar(50))  
  - `status` (varchar(50))  

- **`groups`**  
  Represents functional teams or user groups within a workspace.  
  **Fields:**  
  - `id` (string, PK)  
  - `workspace_id` (string, FK → workspaces.id)  
  - `name` (varchar(100))  
  - `description` (text)  

- **`group_users`**  
  Maps users to groups.  
  **Fields:**  
  - `id` (string, PK)  
  - `group_id` (string, FK → groups.id)  
  - `user_id` (string, FK → users.id)  
  - `role` (varchar(50))  

- **`channels`**  
  Configured communication channels within a workspace.  
  **Fields:**  
  - `id` (string, PK)  
  - `workspace_id` (string, FK → workspaces.id)  
  - `type` (varchar(50))  
  - `config` (jsonb)  
  - `description` (text)  

- **`ai_models`**  
  Available AI model connectors for accounts.  
  **Fields:**  
  - `id` (string, PK)  
  - `account_id` (string, FK → accounts.id)  
  - `name` (varchar(100))  
  - `ai_type` (varchar(50))  
  - `version` (varchar(50))  
  - `config` (jsonb)  
  - `description` (text)  

- **`ai_agents`**  
  Agents linked to a workspace and a single channel; connect to AI models.  
  **Fields:**  
  - `id` (string, PK)  
  - `workspace_id` (string, FK → workspaces.id)  
  - `channel_id` (string, unique, FK → channels.id)  
  - `ai_model_id` (string, FK → ai_models.id)  
  - `name` (varchar(100))  
  - `status` (varchar(50))  
  - `description` (text)  

- **`mcp_connections`**  
  Stores Message Coordination Processor connection configuration.  
  **Fields:**  
  - `id` (string, PK)  
  - `workspace_id` (string, FK → workspaces.id)  
  - `type` (varchar(50))  
  - `config` (jsonb)  
  - `description` (text)  

- **`mcp_logs`**  
  Stores request and response logs for MCP connections.  
  **Fields:**  
  - `id` (string, PK)  
  - `mcp_connection_id` (string, FK → mcp_connections.id)  
  - `request_data` (jsonb)  
  - `response_data` (jsonb)  
  - `status` (varchar(50))  
  - `log_timestamp` (timestamp)  

- **`knowledge_bases`**  
  Stores knowledge base entries linked to workspaces.  
  **Fields:**  
  - `id` (string, PK)  
  - `workspace_id` (string, FK → workspaces.id)  
  - `title` (varchar(200))  
  - `description` (text)  

- **`kb_files`**  
  Stores files uploaded to a knowledge base.  
  **Fields:**  
  - `id` (string, PK)  
  - `knowledge_base_id` (string, FK → knowledge_bases.id)  
  - `workspace_id` (string, FK → workspaces.id)  
  - `user_id` (string, FK → users.id)  
  - `file_name` (varchar(255))  
  - `file_path` (text)  
  - `mime_type` (varchar(100))  
  - `file_size` (bigint)  

- **`kb_file_chunks`**  
  Stores content chunks for each knowledge base file, along with vector embeddings.  
  **Fields:**  
  - `id` (string, PK)  
  - `kb_file_id` (string, FK → kb_files.id)  
  - `chunk_index` (int)  
  - `content` (text)  
  - `vector_embedding` (text)  

- **`tasks`**  
  Tracks tasks created in a workspace (optionally linked to a conversation).  
  **Fields:**  
  - `id` (string, PK)  
  - `workspace_id` (string, FK → workspaces.id)  
  - `agent_id` (string, FK → ai_agents.id)  
  - `user_id` (string, FK → users.id)  
  - `conversation_id` (string, nullable, FK → conversations.id)  
  - `title` (varchar(200))  
  - `description` (text)  
  - `status` (varchar(50))  

- **`task_activity`**  
  Logs actions and updates performed on tasks.  
  **Fields:**  
  - `id` (string, PK)  
  - `task_id` (string, FK → tasks.id)  
  - `action` (varchar(100))  
  - `performed_by` (string)  
  - `details` (jsonb)  
  - `activity_timestamp` (timestamp)  

- **`conversations`**  
  Stores conversation metadata within a workspace.  
  **Fields:**  
  - `id` (string, PK)  
  - `workspace_id` (string, FK → workspaces.id)  
  - `channel_id` (string, FK → channels.id)  
  - `agent_id` (string, FK → ai_agents.id)  
  - `title` (varchar(200))  
  - `status` (varchar(50))  

- **`messages`**  
  Stores messages in a conversation, including prompts and memory/archive info.  
  **Fields:**  
  - `id` (string, PK)  
  - `conversation_id` (string, FK → conversations.id)  
  - `user_id` (string, FK → users.id)  
  - `sender_type` (varchar(50))  
  - `prompt_id` (string, nullable)  
  - `content` (text)  
  - `is_memory_archived` (bool)  
  - `metadata` (jsonb)  

- **`memories`**  
  Stores memory entries for agents within a conversation.  
  **Fields:**  
  - `id` (string, PK)  
  - `agent_id` (string, FK → ai_agents.id)  
  - `conversation_id` (string, FK → conversations.id)  
  - `memory_type` (varchar(50))  
  - `content` (text)  

- **`invites`**  
  Stores workspace invitation records for users.  
  **Fields:**  
  - `id` (string, PK)  
  - `workspace_id` (string, FK → workspaces.id)  
  - `email` (varchar(150))  
  - `role` (varchar(50))  
  - `token` (varchar(255))  
  - `status` (varchar(50))  
  - `invited_by` (string, FK → users.id)  
  - `expires_at` (timestamp)


---

## 4. External Integrations
- **Telegram Bot API** → Webhook for messages.
- **Email API** → SendGrid.
- **AI APIs** → OpenAI, Gemini, etc.
- **Drive APIs** → Google Drive API (for fetching docs).
- **Authentication** → JWT or OAuth2 (multi-tenant).

---

## 5. Deployment & Scaling
- Containerize with Docker.
- CI/CD (GitHub Actions / GitLab CI).
- Cloud hosting (AWS/GCP/Azure/DigitalOcean).
- Use Postgres + pgvector for persistence + semantic search.
- Caching layer (Redis) for session storage & rate limiting.

---

# Sequence Diagram - [Click to View](https://getu.at/hWuEId)

```mermaid
sequenceDiagram
    %% Participants
    participant ClientUser as Client (Registers on Comnet)
    participant Comnet as Comnet System
    participant Workspace as Workspace (Project)
    participant WorkspaceOwner as Workspace Owner (ClientUser Role)
    participant ChannelConfig as Channel Configuration
    participant Channel as Communication Channel (Email/Telegram/SMS)
    participant Agent as Channel Agent
    participant Customer as Customer (Client's customer)
    participant MCP as MCP Server (Message Coordination Processor)
    participant AI as AI Model (ChatGPT/Gemini)
    participant KB as Knowledge Base
    participant Task as Task Management Module
    participant Group as Group (Managers & Members)
    participant Member as Group Member

    %% Registration and Workspace Creation by Client (Workspace Owner)
    ClientUser->>Comnet: Register account
    Comnet-->>ClientUser: Account created
    ClientUser->>Workspace: Create new workspace (project)
    Workspace-->>ClientUser: Workspace created
    ClientUser->>WorkspaceOwner: Become workspace owner

    %% Workspace Setup & Channel Configuration (done by workspace owner)
    
    WorkspaceOwner->>Workspace: Create groups and assign roles (Manager / Member)
    WorkspaceOwner->>ChannelConfig: Add and configure channels (Email / Telegram / SMS)
    ChannelConfig-->>Workspace: Channels registered and linked to workspace
    ChannelConfig->>Agent: Provision Channel Agents for each channel
    Agent-->>Workspace: Channel Agents ready and bound to workspace

    %% Normal flow: Customer raises query on a configured channel
    Customer->>Channel: Send message/query (via configured channel)
    Channel->>Agent: Channel Agent receives and forwards message
    Agent->>MCP: Forward message + workspace context (client id, channel id, conversation history)

    %% AI processing using KB and workspace context
    MCP->>AI: Send message + workspace context + KB pointers
    AI->>KB: Request relevant KB entries and workspace-specific context
    KB-->>AI: Return KB results and context
    AI-->>MCP: Return generated response + intent detection (actionable? yes/no) + confidence

    %% Conditional: If AI detects actionable intent -> create task in same workspace and assign to group
    alt Actionable intent detected
        MCP->>Task: Create Task in Workspace (details, priority, assignee group)
        Task->>Group: Assign task to specific group or manager
        Group-->>Task: Acknowledgement of task assignment
        Task->>Workspace: Log task creation and link it to conversation
        Workspace-->>WorkspaceOwner: Notify workspace members (or assigned group) of new task
    end

    %% Response delivery back to customer via same channel
    MCP-->>Agent: Return AI-generated reply (and task info if created)
    Agent-->>Channel: Send reply via originating channel
    Channel-->>Customer: Deliver AI response to customer

    %% Group resolves task (if created)
    alt Task exists
        Member->>Task: View task and start work
        Member->>Group: Update task status / add comments / attach files
        Group-->>Task: Mark task progress or resolution
        Task-->>Workspace: Update task log and notify conversation thread
        Workspace-->>Customer: Optionally send status update (via Channel) if configured
    end

    %% Logging, KB updates and analytics (always)
    Agent->>Workspace: Log conversation and response metadata
    AI->>Workspace: (optional) Suggest KB update based on conversation
    Workspace->>KB: Update KB (optional manual review or auto-suggest)
    Workspace-->>WorkspaceOwner: Update analytics / reports / conversation history

    %% Ongoing conversation loop
    Customer-->>Channel: Follow-up messages (loop continues)
    Channel->>Agent: Forward follow-ups
    Agent->>MCP: Repeat AI processing with updated context

```

# Future Scope

The platform is designed with scalability in mind, allowing integration of new features and channels to enhance client communication and management. Planned future improvements include:

## 1. Additional Communication Channels
- **Channels**  
  - Integration with  email
  - Integration with telephony services.
  - AI-powered call transcription, sentiment analysis, and automated follow-up responses.  

## 2. Ticketing & Workflow Management
- **Jira Integrations**  
  - Automatic ticket creation from client queries.  
  - Linking AI conversations to tasks and workflows.  
  - Syncing client issues directly into project management tools.  

- **Internal Ticketing System**  
  - Native ticket creation and tracking within the platform.  
  - Escalation workflows and admin assignment for unresolved queries.  

## 3. Advanced AI Features
- **Sentiment & Intent Analysis**  
  - Understanding customer tone and urgency for better prioritization.  

- **Personalized AI Responses**  
  - AI models trained on client-specific style guides and communication tone.   

## 4. Analytics & Insights
- Advanced reporting on client queries, response times, and AI performance.  
- Predictive analytics for identifying recurring issues or potential churn risks.  
- Exportable insights for stakeholders.  

## 🔸 Scope of Demo 1

The first demo of **CommNet** establishes the **core multi-tenant communication foundation** and **essential management modules**.

---

### ✅ Completed in Demo 1

#### 🧩 Client Onboarding & Authentication
- Registration, login, and onboarding workflows for businesses.  
- Multi-tenant user and organization setup.

#### 🤖 AI Model Management
- Create, update, and manage AI connectors.  
- Support for pluggable AI model configurations.

#### 🏢 Workspace Management
- Create and manage workspaces under business accounts.  
- Link AI models to specific workspaces.

#### ⚙️ MCP Server Management
- Configure and manage MCP connections.  
- Store, validate, and monitor MCP configurations.

#### 💬 Communication Channel (Telegram)
- Telegram bot setup and configuration as the first supported channel.  
- Webhook-based message handling integrated with MCP and AI pipelines.

#### 🧠 AI Agent Management
- Manage AI agents linked to workspaces and channels.  
- Configure model bindings, activation states, and AI routing logic.

---

### 🔮 Future Scope

### A. Functional Enhancements

#### 📚 Knowledge Base (KB) Management
- Upload, parse, and index documents (**PDF**, **DOCX**, **TXT**).  
- Store embeddings using **pgvector** for contextual **RAG** responses.  
- Support integration with **Google Drive** and other data sources.

#### 💬 Conversation & Message Flow
- End-to-end conversation orchestration between channels, MCP, and AI models.  
- Store conversation history, enable admin override, and manage AI memory.

#### 📋 Task & Group Management
- Auto-create tasks from actionable AI intents.  
- Assign and track tasks within workspace groups.  
- Provide activity logs and notifications.

#### 🌐 Additional Communication Channels
- Extend support to **Email (IMAP/SMTP, SendGrid)** and **WhatsApp Business API**.  
- Standardized message orchestration across all channels.

#### 📊 Admin & Analytics Dashboard
- Comprehensive dashboard for monitoring usage, AI performance, and matrix.  
- Reports for workspace activity, AI accuracy, and user engagement.

---

## B. Non-Functional Enhancements

#### 🚀 Scalability & Performance
- Optimize backend services for high concurrency using **Go routines**.  
- Container orchestration via **Docker/Kubernetes** for horizontal scaling.

#### 🔒 Security & Compliance
- Role-Based Access Control (**RBAC**).  
- **JWT/OAuth2** authentication with tenant isolation.  
- Data encryption at rest and in transit.  
- Audit logs for compliance (**GDPR-ready**).

#### ⚙️ Reliability & Observability
- Centralized logging using **Grafana + Loki + Prometheus**.  
- Real-time error tracking and health checks.  
- Backup and disaster recovery mechanisms.

#### 🎨 User Experience
- Improved UI using **ShadcnUI** components.  
- Responsive layouts and faster navigation with **Vite + TanStack Router**.