* **Source Fetching:** A scheduled task (running every few minutes) initiates the pipeline, fetching new articles from a predefined list of news APIs, blogs, and financial websites.
-
* **Preprocessing:** Raw HTML is cleaned, and the core article text, title, and metadata are extracted.
-
* **Vectorization:** The cleaned text is converted into a numerical vector embedding using a Sentence-Transformers model.
-
* **Deduplication:** A similarity search is performed against the PG Vector database. If an article with a highly similar vector already exists, it is discarded to avoid redundant analysis and alerts.
-
* **AI Sentiment Analysis:** For new, unique articles, a structured prompt is sent to the Gemini API. The API returns a detailed analysis, including:
-
* **A sentiment score** (e.g., from -1.0 to 1.0).
-
* **An impact level** (e.g., High, Medium, Low).
-
* **A concise summary of the news.**
-
* **Persistence:** The processed article—including its text, metadata, vector embedding, and the full sentiment analysis result—is saved as a new record in the news_articles table in the PostgreSQL database.
+
* Fetch new articles from trusted sources.
+
+
* Preprocess text, remove duplicates, and normalize data.
+
+
* Run sentiment & credibility analysis via Gemini API.
+
+
* Store processed articles in PostgreSQL with vector embeddings.
**Module 2: The Alerting Engine**<br>
-
**Objective:**<br> To monitor the analyzed news in real-time and deliver personalized, high-impact alerts to the correct users instantly.<br>
+
**Objective:** TDeliver real-time, personalized alerts for high-impact news.<br>
**Technology:** Python, Redis, PostgreSQL.
**Process Flow:**
-
* **Event Detection:** A dedicated worker service continuously monitors the news_articles table for newly inserted records.
-
* **Audience Matching:** For each new article, the engine performs a critical matching query to identify the
-
* **target audience:** It retrieves the coin_id (e.g., 'ethereum') and the sentiment_score (e.g., -0.92) from the new article.It then queries the users and user_watchlist tables to find all users who meet both of the following criteria:<br>
-
**a.** The user is following that specific coin (user_watchlist.coin_id = 'ethereum').<br>
-
**b.** The article's sentiment score meets the user's personal alert_sensitivity threshold (e.g., abs(-0.92) >= user.alert_sensitivity).
-
* **Payload Construction:** For the matched list of users, the engine constructs the push notification payload (title, body, and any data needed to deep-link into the app).
-
* **Dispatch:** The engine makes a single API call to Firebase Cloud Messaging (FCM), providing the payload and the list of unique device tokens for the matched users. FCM then handles the reliable delivery of the notification to each user's mobile device.
+
* Detect new sentiment-analyzed articles.
+
+
* Match articles to users based on watchlist and alert thresholds.
+
+
* Build payload and send notifications via FCM.
+
+
* Track delivery and engagement for analytics.
**Module 3: Backend API**<br>
@@ 122,21 120,13 @@
**Technology:** NodeJS, PostgreSQL.<br>
**Key Responsibilities:**
-
* **User Management:** Handles user registration, login (authentication), and profile management.
-
* **Watchlist Management:** Provides CRUD (Create, Read, Update, Delete) endpoints for users to manage their list of followed cryptocurrencies.
-
* **Settings Management:** Allows users to update their personal settings, most importantly their alert_sensitivity level.
-
* **Data Serving:** Provides data for the main dashboard, including prices (fetched and cached from the CoinGecko API) and a feed of recent high-impact news from the PostgreSQL database. Serves historical news and sentiment data for specific coin detail pages.
-
* **Device Registration:** Securely receives and stores the user's FCM device token required for push notifications.
-
* **End-to-End Workflow Example**
-
* **A news site publishes an article:** "Major Exchange Reports Security Audit Failure."
-
* The Ingestion Pipeline fetches it, creates an embedding, sees it's unique, and sends it to Gemini. Gemini returns a sentiment score of -0.95. The data is saved to PostgreSQL.
-
* **The Alerting Engine detects the new article.**
-
* **The engine queries the database:** "Find all users following this exchange's coin whose sensitivity is set to 95% or less."
-
* **It finds two users:** <br>
-
User A (sensitivity 80%) and User B (sensitivity 90%).<br>
-
It discards User C (sensitivity 98%).
-
* The engine sends the device tokens for User A and B to FCM with the alert message.
-
* Users A and B receive a push notification within minutes of the news being published. User C does not.
+
* Manage user accounts, authentication, and profiles.
+
+
* Provide CRUD for watchlists and notification settings.
+
+
* Serve live market data, news feeds, and historical sentiment.
+
+
* Ensure security, caching, and performance optimization.
Later, all three users open the Sentify App. The Backend API serves them the dashboard, where they can all see the negative news item in their feed.