# HLD ## 1. Introduction **SkillShift** is a system designed to help employees and HR/Admins manage, explore, and retrieve organizational knowledge through **AI-assisted interactions**. Organizations often face operational and productivity risks when employees leave or move across teams without structured handovers or visibility into their unique knowledge, decisions, and expertise. This tool leverages **NLP-based knowledge mining** to automatically build a dynamic knowledge base and organizational decisions — all derived from communication channels like **Google Chat** and **Telegram**. The platform centralizes internal communication data, maintains tags for context classification, and provides employees and HR/Admins with an AI assistant to query messages and insights related to them or their assigned tags. In modern organizations, large volumes of valuable information — discussions, decisions, technical clarifications, and project context — are scattered across internal messaging tools. This tool provides a unified way to collect, organize, and retrieve such communication efficiently using AI-assisted search and tag-based categorization. Unlike traditional HR or knowledge base systems, this assistant focuses on **conversation intelligence** rather than static documentation. Instead of manually searching through thousands of chat messages, users can simply ask **natural language questions** through the AI assistant, which semantically understands the context and retrieves relevant messages tagged to their area, project, or topic. ### Platform Users - **Employees** - View their profiles, assigned tags, and interact with the AI assistant to find relevant discussions or information. - **HR/Admins** - Manage and assign tags, configure data source integrations, and use the AI assistant for organization-wide insights. --- ## 2. Objectives The system aims to achieve the following core objectives: - Consolidate organizational conversations from multiple data sources such as **Google Chat** and **Telegram** into a single, searchable knowledge base. - Simplify knowledge discovery through a **conversational AI assistant**. - Enable structured organization of communication via **tags** for projects, discussions, or themes. - Empower employees with **role-based access** to only relevant messages and topics. - Provide HR/Admins tools for efficient management of users, tags, and data sources. - Enhance organizational transparency and reduce information silos through AI-assisted knowledge retrieval. --- ## 3. Initial Scope (MVP Deliverables) - **Integrations:** Google Chat & Telegram - **Screens:** - Employee Profile - AI Assistant - Tag Management - Data Source Management --- ## 4. Users & Roles - **Employee:** View their profile and access the AI assistant. - **HR/Manager/System Admin:** Manage integrations and tag management. --- ## 5. Architecture Overview  --- ## 6. Module Breakdown ### 6.1 Integration Layer - **Telegram Bot API:** Captures chat messages. - **Google Chat API:** Webhook/polling for spaces & groups. --- ### 6.2 Processing & Embedding Layer - **Text Preprocessing:** Removes stopwords, links, emojis, and Personally Identifiable Information (PII). - **Embeddings:** Creates message text embeddings for semantic search. --- ### 6.3 Knowledge Store (PostgreSQL + pgvector) #### Schema (MVP) | Table | Description | |-------|--------------| | `employees` | id, external_id, name, role, dept, tenure, is_active, is_admin | | `messages` | id, employee_id, data_source_id, text, embedding | | `tags` | id, tag_name, is_active | | `employee_tag` | id, tag_id, employee_id, is_active | | `data_sources` | id, platform, external_id, name, tag_id, description, is_active | #### Logical Data Model - **Employees** - Represents all individuals using or referenced by the platform. - Attributes: `id`, `name`, `role`, `department`, `tenure`, `is_active`. - Linked to multiple tags and messages. - **Messages** - Stores chat data from Google Chat or Telegram. - Each message is tied to one employee and one data source. - Stores cleaned text and a semantic embedding vector. - **Tags** - Represents thematic or organizational labels (e.g., *Project Alpha*, *DevOps*). - Used to group employees and messages under meaningful categories. - **Data Sources** - Represents external chat systems (Telegram group, Google Chat space). - One data source → one tag; one tag → many data sources. #### Relationships (Conceptually) | Relationship | Type | Description | |---------------|------|-------------| | Employee ↔ Message | 1 → N | One employee can send multiple messages | | Employee ↔ Tag (via employee_tag) | M ↔ N | Employees can belong to multiple tags | | Message ↔ Data Source | 1 → N | Each message belongs to one data source | | Data Source ↔ Tag | 1 → N | One tag can link to multiple data sources | | Tag ↔ Employee_Tag | 1 → N | Each tag can appear in multiple employee_tag records | | Employee ↔ Employee_Tag | 1 → N | Each employee can have multiple tag associations | --- ### 6.4 AI Workflow Automation Layer - **Retriever:** Uses pgvector to fetch top-K relevant messages. - **LLM (OpenAI GPT-4 / LLaMA):** - Performs Q&A generation. - **LangChain:** Used for the RAG (Retrieval-Augmented Generation) pipeline. --- ### 6.5 Backend (API Layer) **Framework:** FastAPI #### Key Endpoints | Method | Endpoint | Description | |--------|-----------|-------------| | GET | `/employee/{id}` | Retrieve a specific employee’s profile | | GET | `/employee` | List all employees (Admin only) | | POST | `/message/bulk_ingest` | Store messages fetched from integrations | | POST | `/tag` | Create or update a tag (Admin only) | | GET | `/tag` | Retrieve all active tags | | GET | `/tag/{id}` | Get tag details with linked employees & sources | | POST | `/employee_tag` | Assign or unassign a tag to an employee | | GET | `/employee_tag/{employee_id}` | Fetch tags assigned to a specific employee | | POST | `/data_source` | Add or update a data source | | GET | `/data_source` | List all configured data sources | | GET | `/data_source/{id}` | Get details of a specific data source | | POST | `/chatbot/query` | Ask a natural language question | | GET | `/admin/overview` | Dashboard summary for Admins | --- ### 6.6 Frontend (React) #### Employee View - **Profile View** - Displays employee details, department, tenure, and assigned tags. - Uses `/employee/{id}` and `/employee_tag/{employee_id}` APIs. - **AI Assistant Chat** - Interactive chat that calls `/chatbot/query`. - Allows employees to query messages related to assigned tags. #### HR/Manager/Admin View - **Profile Page** - Similar to employee view, with added admin controls. - **Tag Management Panel** - Create/edit tags via `/tag` API. - Assign tags to employees via `/employee_tag`. - **Data Source Management** - Manage Telegram/Google Chat integrations. - Create or deactivate sources using `/data_source`. --- ## 7. Data Flow Example **Example Query:** “Did we use Kubernetes in any project?” 1. Chatbot API receives query. 2. NLP extracts keyword → *Kubernetes*. 3. pgvector performs semantic similarity search. 4. Relevant messages + metadata passed into LLM. 5. LLM responds → “Yes, Kubernetes was used in Project Alpha (2024).” 6. Frontend chatbot displays the answer with source link. --- ## 8. Tech Stack | Component | Technology | |------------|-------------| | **Frontend** | React, Tailwind | | **Backend** | FastAPI (Python) | | **Database** | PostgreSQL + pgvector | | **AI/NLP** | OpenAI Embeddings, GPT-4 | | **Integrations** | Google Chat API, Telegram Bot API | | **Infra** | Docker Compose (MVP), AWS/DigitalOcean | --- ## 9. Security & Compliance (Outside MVP) ### 9.1 Data Protection - Data processed locally or within client’s environment. - No raw chat data stored post-extraction. - Encryption: HTTPS (in transit), AES-256 (at rest). ### 9.2 Access Control & Authentication - Role-based access: Admin, Manager, Employee. - OAuth 2.0 for integrations (Google, Telegram). - Logs and audit trails for sensitive actions. ### 9.3 SOC 2 & GDPR Alignment - Privacy-by-design principles. - Data minimization: only relevant skill/project data extracted. - Right to erasure and consent-based processing. - Future goal: SOC 2 Type 2 & GDPR certification. ### 9.4 Data Retention & Anonymization - Temporary data auto-deletion post-processing. - PII replaced with internal IDs before storage. - Configurable retention policies per client. --- ## 10. Future Enhancements - Integration with Slack, Microsoft Teams, Jira, Confluence, etc. - **Employee Handover Package Generator** during role transitions. ---