Home A - Z
Page Index

HLD

1. Introduction

SkillShift is a system designed to help employees and HR/Admins manage, explore, and retrieve organizational knowledge through AI-assisted interactions.

Organizations often face operational and productivity risks when employees leave or move across teams without structured handovers or visibility into their unique knowledge, decisions, and expertise.

This tool leverages NLP-based knowledge mining to automatically build a dynamic knowledge base and organizational decisions — all derived from communication channels like Google Chat and Telegram. The platform centralizes internal communication data, maintains tags for context classification, and provides employees and HR/Admins with an AI assistant to query messages and insights related to them or their assigned tags.

In modern organizations, large volumes of valuable information — discussions, decisions, technical clarifications, and project context — are scattered across internal messaging tools. This tool provides a unified way to collect, organize, and retrieve such communication efficiently using AI-assisted search and tag-based categorization.

Unlike traditional HR or knowledge base systems, this assistant focuses on conversation intelligence rather than static documentation. Instead of manually searching through thousands of chat messages, users can simply ask natural language questions through the AI assistant, which semantically understands the context and retrieves relevant messages tagged to their area, project, or topic.

Platform Users

  • Employees
    • View their profiles, assigned tags, and interact with the AI assistant to find relevant discussions or information.
  • HR/Admins
    • Manage and assign tags, configure data source integrations, and use the AI assistant for organization-wide insights.

2. Objectives

The system aims to achieve the following core objectives:

  • Consolidate organizational conversations from multiple data sources such as Google Chat and Telegram into a single, searchable knowledge base.
  • Simplify knowledge discovery through a conversational AI assistant.
  • Enable structured organization of communication via tags for projects, discussions, or themes.
  • Empower employees with role-based access to only relevant messages and topics.
  • Provide HR/Admins tools for efficient management of users, tags, and data sources.
  • Enhance organizational transparency and reduce information silos through AI-assisted knowledge retrieval.

3. Initial Scope (MVP Deliverables)

  • Integrations: Google Chat & Telegram
  • Screens:
    • Employee Profile
    • AI Assistant
    • Tag Management
    • Data Source Management

4. Users & Roles

  • Employee: View their profile and access the AI assistant.
  • HR/Manager/System Admin: Manage integrations and tag management.

5. Architecture Overview

6. Module Breakdown

6.1 Integration Layer

  • Telegram Bot API: Captures chat messages.
  • Google Chat API: Webhook/polling for spaces & groups.

6.2 Processing & Embedding Layer

  • Text Preprocessing: Removes stopwords, links, emojis, and Personally Identifiable Information (PII).
  • Embeddings: Creates message text embeddings for semantic search.

6.3 Knowledge Store (PostgreSQL + pgvector)

Schema (MVP)

Table Description
employees id, external_id, name, role, dept, tenure, is_active, is_admin
messages id, employee_id, data_source_id, text, embedding
tags id, tag_name, is_active
employee_tag id, tag_id, employee_id, is_active
data_sources id, platform, external_id, name, tag_id, description, is_active

Logical Data Model

  • Employees

    • Represents all individuals using or referenced by the platform.
    • Attributes: id, name, role, department, tenure, is_active.
    • Linked to multiple tags and messages.

  • Messages

    • Stores chat data from Google Chat or Telegram.
    • Each message is tied to one employee and one data source.
    • Stores cleaned text and a semantic embedding vector.

  • Tags

    • Represents thematic or organizational labels (e.g., Project Alpha, DevOps).
    • Used to group employees and messages under meaningful categories.

  • Data Sources

    • Represents external chat systems (Telegram group, Google Chat space).
    • One data source → one tag; one tag → many data sources.

Relationships (Conceptually)

Relationship Type Description
Employee ↔ Message 1 → N One employee can send multiple messages
Employee ↔ Tag (via employee_tag) M ↔ N Employees can belong to multiple tags
Message ↔ Data Source 1 → N Each message belongs to one data source
Data Source ↔ Tag 1 → N One tag can link to multiple data sources
Tag ↔ Employee_Tag 1 → N Each tag can appear in multiple employee_tag records
Employee ↔ Employee_Tag 1 → N Each employee can have multiple tag associations

6.4 AI Workflow Automation Layer

  • Retriever: Uses pgvector to fetch top-K relevant messages.
  • LLM (OpenAI GPT-4 / LLaMA):
    • Performs Q&A generation.
  • LangChain: Used for the RAG (Retrieval-Augmented Generation) pipeline.

6.5 Backend (API Layer)

Framework: FastAPI

Key Endpoints

Method Endpoint Description
GET /employee/{id} Retrieve a specific employee’s profile
GET /employee List all employees (Admin only)
POST /message/bulk_ingest Store messages fetched from integrations
POST /tag Create or update a tag (Admin only)
GET /tag Retrieve all active tags
GET /tag/{id} Get tag details with linked employees & sources
POST /employee_tag Assign or unassign a tag to an employee
GET /employee_tag/{employee_id} Fetch tags assigned to a specific employee
POST /data_source Add or update a data source
GET /data_source List all configured data sources
GET /data_source/{id} Get details of a specific data source
POST /chatbot/query Ask a natural language question
GET /admin/overview Dashboard summary for Admins

6.6 Frontend (React)

Employee View

  • Profile View

    • Displays employee details, department, tenure, and assigned tags.
    • Uses /employee/{id} and /employee_tag/{employee_id} APIs.

  • AI Assistant Chat

    • Interactive chat that calls /chatbot/query.
    • Allows employees to query messages related to assigned tags.

HR/Manager/Admin View

  • Profile Page

    • Similar to employee view, with added admin controls.

  • Tag Management Panel

    • Create/edit tags via /tag API.
    • Assign tags to employees via /employee_tag.

  • Data Source Management

    • Manage Telegram/Google Chat integrations.
    • Create or deactivate sources using /data_source.

7. Data Flow Example

Example Query: “Did we use Kubernetes in any project?”

  1. Chatbot API receives query.
  2. NLP extracts keyword → Kubernetes.
  3. pgvector performs semantic similarity search.
  4. Relevant messages + metadata passed into LLM.
  5. LLM responds → “Yes, Kubernetes was used in Project Alpha (2024).”
  6. Frontend chatbot displays the answer with source link.

8. Tech Stack

Component Technology
Frontend React, Tailwind
Backend FastAPI (Python)
Database PostgreSQL + pgvector
AI/NLP OpenAI Embeddings, GPT-4
Integrations Google Chat API, Telegram Bot API
Infra Docker Compose (MVP), AWS/DigitalOcean

9. Security & Compliance (Outside MVP)

9.1 Data Protection

  • Data processed locally or within client’s environment.
  • No raw chat data stored post-extraction.
  • Encryption: HTTPS (in transit), AES-256 (at rest).

9.2 Access Control & Authentication

  • Role-based access: Admin, Manager, Employee.
  • OAuth 2.0 for integrations (Google, Telegram).
  • Logs and audit trails for sensitive actions.

9.3 SOC 2 & GDPR Alignment

  • Privacy-by-design principles.
  • Data minimization: only relevant skill/project data extracted.
  • Right to erasure and consent-based processing.
  • Future goal: SOC 2 Type 2 & GDPR certification.

9.4 Data Retention & Anonymization

  • Temporary data auto-deletion post-processing.
  • PII replaced with internal IDs before storage.
  • Configurable retention policies per client.

10. Future Enhancements

  • Integration with Slack, Microsoft Teams, Jira, Confluence, etc.
  • Employee Handover Package Generator during role transitions.