The project follows a **modular and layered architecture** designed for scalability, maintainability, and separation of concerns.

The **backend** is implemented using **FastAPI**, with distinct folders for models, schemas, services, routes, utilities, and background workers.

Each layer has a clear responsibility:

- **Models Layer:** Contains ORM classes mapping database tables.

- **Schemas Layer:** Defines Pydantic models for request validation and API responses.

- **Service Layer:** Implements business logic (NLP, chat assistant).

- **Routes Layer:** Defines API endpoints exposed via FastAPI.

- **Utility Layer:** Includes reusable components like embedding generation, configuration, logging, and security helpers.

- **Worker Layer:** Handles background jobs such as scheduled chat message fetching and NLP processing.

This modular layout ensures that each component can evolve independently — for example, NLP services or integrations can scale separately from the main API.

The **frontend** is built using **React** and consumes REST APIs exposed by the FastAPI backend. It serves as the user interface for screens, chatbot interactions, and employee views.

/repo

├─ /api # FastAPI app

│ ├─ main.py

│ ├─ routers/

│ │ ├─ auth.py

│ │ ├─ chat.py

│ │ ├─ employees.py

│ ├─ models/ # ORM classes

│ └─ services/ # Business logic (NLP invocations, DB helpers)

├─ /workers

│ ├─ telegram_worker.py

│ ├─ googlechat_worker.py

│ ├─ scheduler.py

│ └─ nlp_worker.py

├─ /nlp

│ └─ embeddings.py

├─ /ragsvc

│ └─ rag.py # Retrieval + prompt assembly + LLM call

├─ /frontend # React app

│ ├── src/

│ │ ├── components/ # Reusable UI components (cards, modals, charts)

│ │ ├── pages/ # Dashboard and chatbot screens

│ │ ├── services/ # API service calls

│ │ ├── hooks/ # Custom React hooks

│ │ ├── contexts/ # Auth and user management

│ │ ├── assets/ # Icons, images, and theme files

│ │ └── utils/ # Helper functions

│ ├── public/ # Static assets

│ ├── package.json

│ ├── vite.config.js or next.config.js

│ └── README.md

├─ /db

│ └─ migrations.sql

├─ docker-compose.yml

├─ Dockerfile.api

└─ README.md

The database schema defines all core entities and their relationships for the MVP.

It is implemented in **PostgreSQL** and **pgvector** for storing text embeddings used in semantic search.

The schema captures four main domains:

- **Tables:** `employees`, `employee_tag`

- Manages employee profiles, departmental details, and tag associations.

- Defines users of the system (employees and admins) and establishes relationships between employees and tags for access control and contextual filtering.

- **Tables:** `messages`

- Stores sanitized chat messages fetched from configured data sources.

- Each message is tied to an employee (sender) and a data source (Telegram or Google Chat), and includes an embedding vector for semantic search.

- **Tables:** `tags`

- Defines organizational tags or categories such as projects, teams, or topics.

- Tags serve as contextual identifiers used to categorize employees, data sources, and messages.

- **Tables:** `data_sources`

- Represents external chat integrations (Google Chat or Telegram groups).

- Each data source is linked to a specific tag and acts as an entry point for message ingestion.

- Foreign key relationships use `ON DELETE CASCADE` for integrity.

---

- `is_admin` defines access privileges (Employee/Admin).

- `is_active` allows temporary deactivation without deletion.

---

- `embedding` uses the **pgvector** extension for similarity-based AI search.

- `employee_id` may be nullable if sender info is anonymized or unavailable.

---

- Tags are assigned to employees and data sources to define context (e.g., “Project Alpha”).

- Uniqueness ensures no duplicate project or topic names.

---

- Defines a many-to-many relationship between employees and tags.

- `is_active` allows toggling tag assignment without deleting the record.

---

- Links chat groups/spaces to organizational tags.

- When a tag is deleted, `tag_id` becomes `NULL` but the data source remains for reference.

**Purpose:** Retrieve a specific employee’s profile (for dashboard view).

- `id` (integer) — Employee ID.

- `404 Not Found` — Employee ID not found.

- `400 Bad Request` — Invalid message structure.

---

**Purpose:** List all employees (Admin only).

- `is_active` (boolean, optional)

- `dept` (string, optional)

**Purpose:** Store messages fetched from integrations (used by ingestion worker).

- `400 Bad Request` — Invalid message structure.

**Purpose:** Create or update a tag (Admin only).

**Purpose:** Retrieve all active tags.

**Purpose:** Get tag details with linked employees and data sources.

**Purpose:** Assign or unassign a tag to an employee (Admin only).

**Purpose:** Fetch all tags assigned to a specific employee.

**Purpose:** Add or update a chat data source.

**Purpose:** List all configured data sources with their associated tags.

**Purpose:** Get details of a specific data source.

**Purpose:** Submit a natural language query to the AI assistant.

**Purpose:** Retrieve system overview for Admin dashboard.

1. **Models Layer** → ORM classes (DB mapping)

2. **Schemas Layer** → Pydantic models for request/response validation

3. **Service Layer** → Business logic (NLP, Chatbot, Handover)

4. **Routes Layer** → FastAPI endpoints

5. **Utility Layer** → Common helpers (security, embeddings, config)

6. **Scheduler Layer** → Cron jobs

---

Defines ORM entities using SQLAlchemy, each mapped to a database table.

- **Purpose:** Represents system users and their platform identifiers.

- **Key Fields:** `id`, `external_ids` (JSONB), `name`, `role`, `dept`, `tenure`, `is_admin`, `is_active`.

- **Purpose:** Stores cleaned messages fetched from data sources.

- **Key Fields:** `id`, `employee_id`, `data_source_id`, `text`, `embedding`, `created_at`.

- **Purpose:** Defines project or topic categories (e.g., “Project Alpha”).

- **Key Fields:** `id`, `tag_name`, `is_active`.

- **Purpose:** Junction table linking Employee and Tag.

- **Key Fields:** `id`, `employee_id`, `tag_id`, `is_active`.

- **Purpose:** Represents chat integrations like Telegram or Google Chat.

- **Key Fields:** `id`, `platform`, `external_id`, `name`, `tag_id`, `description`, `is_active`.

---

Defines data validation and serialization classes for API input/output.

- **EmployeeBase / EmployeeResponse**

Purpose: Serialize employee info for API responses.

Fields: `id`, `name`, `role`, `dept`, `tenure`, `is_admin`, `is_active`, `external_ids`, `tags`.

- **MessageIngestRequest**

Purpose: Request body for bulk message ingestion.

Fields: `data_source_id`, `messages` (list of `{employee_id, text, embedding}`).

- **TagCreate / TagResponse**

Purpose: Used for tag creation and retrieval.

Fields: `id`, `tag_name`, `is_active`.

- **EmployeeTagRequest**

Purpose: Assign or unassign tag to employee.

Fields: `employee_id`, `tag_id`, `is_active`.

- **DataSourceRequest / DataSourceResponse**

Purpose: Manage chat data sources.

Fields: `id`, `platform`, `external_id`, `name`, `tag_id`, `description`, `is_active`.

- **ChatbotQueryRequest / ChatbotResponse**

Purpose: Handles chatbot input/output payloads.

Fields (Request): `query`, `employee_id`

Fields (Response): `answer`, `related_messages`

---

Implements all application logic — CRUD operations, AI processing, and integrations.

- **Purpose:** Manage employee CRUD and tag relationships.

- **Key Methods:**

- `get_employee(id)` – Fetch employee with tags.

- `list_employees(is_active)` – List all employees.

- `create_or_update(data)` – Upsert employee.

- **Purpose:** Handle tag creation, lookup, and linking to employees or sources.

- **Key Methods:**

- `create_or_update(tag_data)`

- `get_tag(id)`

- `get_all_tags()`

- **Purpose:** Manage mappings between employees and tags.

- **Key Methods:**

- `assign_tag(employee_id, tag_id)`

- `get_tags_for_employee(employee_id)`

- **Purpose:** Ingest, store, and fetch messages.

- **Key Methods:**

- `bulk_ingest(messages)` – Save messages after processing.

- `get_by_tag(tag_id)` – Retrieve all messages under a tag.

- **Purpose:** Manage chat integration sources.

- **Key Methods:**

- `create_or_update(source_data)`

- `get_active_sources()`

- `get_source_details(id)`

- **Purpose:** Provide AI-based Q&A by semantic search.

- **Key Methods:**

- `query(text, employee_id)` – Generate embeddings, find similar messages.

---

Defines REST API endpoints; connects incoming requests to service methods.

- **EmployeeRouter**

- `GET /employee/{id}` – Fetch employee details

- `GET /employee` – List employees

- **MessageRouter**

- `POST /message/bulk_ingest` – Store processed chat messages

- **TagRouter**

- `POST /tag` – Create or update a tag

- `GET /tag` – Retrieve all tags

- `GET /tag/{id}` – Tag details with employees and data sources

- **EmployeeTagRouter**

- `POST /employee_tag` – Assign/unassign tag to employee

- `GET /employee_tag/{employee_id}` – Get employee’s tags

- **DataSourceRouter**

- `POST /data_source` – Add or update chat source

- `GET /data_source` – List all data sources

- `GET /data_source/{id}` – Fetch specific data source

- **ChatbotRouter**

- `POST /chatbot/query` – Handle AI question-answer queries

- **AdminRouter**

- `GET /admin/overview` – Returns admin dashboard summary

---

Provides shared functionality for embeddings, sanitization, security, and configuration.

- **Purpose:** Convert text to embeddings using SentenceTransformer

- **Key Methods:** `generate(text)` – Returns 768-dim vector embedding

- **Purpose:** Clean raw chat messages

- **Key Methods:** `sanitize(text)` – Remove PII, URLs, emojis, and normalize case

- **Purpose:** Handle authentication and access control

- **Key Methods:**

- `validate_token(token)`

- `is_admin(user)`

- **Purpose:** Centralized configuration management

- **Sources:** `.env file`, environment variables

- **Purpose:** Unified logging across services

- **Key Methods:** `info()`, `error()`, `debug()`

---

Handles periodic and background operations such as message ingestion and cleanup.

- **Purpose:** Periodically fetch new messages from Telegram/Google Chat sources

- **Key Methods:** `run()` – Fetch active data sources and ingest new messages

- **Purpose:** Recompute embeddings for all stored messages (if model changes)

- **Key Methods:** `run()` – Iterate through messages, regenerate embeddings

- **Purpose:** Archive or delete inactive data sources and outdated messages

- **Key Methods:** `run()` – Perform cleanup based on retention policies

1. **Scheduler (ChatIngestionJob)** runs periodically.

2. **DataSourceService** fetches all active sources from the `data_sources` table.

3. For each source:

- The system fetches messages using **Telegram/Google Chat APIs**.

- **SanitizationUtils** cleans the text.

- **EmbeddingService** converts it into a semantic vector.

- **MessageService** stores the text + embedding in the `messages` table.

4. The job logs completion status.

1. The **Frontend Chat UI** sends the query to `/chatbot/query` along with `employee_id`.

2. **ChatbotService** generates a vector embedding for the query.

3. **EmployeeTagService** fetches which tags this employee has access to.

4. **MessageService** performs a vector similarity search within messages linked to those tags.

5. The **LLM** (e.g., OpenAI GPT / local model) generates a summarized, context-aware response.

6. The system returns both:

- `answer` → generated explanation or summary.

- `related_messages` → list of original message snippets.

7. The Frontend displays both to the user.