RSS Implementation and Web Consumption

What is RSS? (Really Simple Syndication)

RSS is a data format based on XML (eXtensible Markup Language) designed for the automated distribution of content. In network architecture, we consider it a method of content syndication.

Strictly speaking, RSS allows a server to publish a list of updates (such as blog articles, news, or podcast episodes) in a standard format that any client (reader) can interpret without needing to manually navigate the visual interface of the web.

Structure of an RSS File

An RSS document is, in essence, a text file with a hierarchical structure:

<rss>: The root element that defines the protocol version.
<channel>: Contains the channel's metadata (title, description, language).
<item>: Each of the individual entries or news items.

Relationship with Web Architecture (Backend and Frontend)

RSS acts as a pure data bridge between the server and the client:

Backend (The Generator): The database server contains the information. The backend (programmed in Python, PHP, Node.js, etc.) queries the database and, instead of rendering complex HTML, generates a strict XML file.
Communication Protocol: Generally transmitted via HTTP/HTTPS. The client makes a GET request to a specific URL (e.g., domain.com/feed.xml).
Frontend / Client (The Consumer): The "reader" (which can be a mobile app, another website, or a browser plugin) receives the XML, parses it (analyzes the syntax), and presents the information to the end user.

Implementation: Generation from the Backend

Imagine we are in the backend of a magical newspaper. We will use Python to generate this XML file rigorously.

# We import the elementtree library to build XML structures
import xml.etree.ElementTree as ET

# We define a function to generate the feed
def generate_rss():
    # We create the root element <rss> and specify version 2.0
    # ET.Element creates an object representing an XML tag
    root = ET.Element("rss", version="2.0")

    # We create the main channel <channel> inside the root
    # ET.SubElement adds a child to the indicated element
    channel = ET.SubElement(root, "channel")

    # We add the website title
    # .text assigns the content that will go inside the tag
    web_title = ET.SubElement(channel, "title")
    web_title.text = "Ravencloud Library"

    # We add a description of the channel
    web_desc = ET.SubElement(channel, "description")
    web_desc.text = "Updates on network protocols"

    # We simulate an article (item) from the database
    item = ET.SubElement(channel, "item")

    # Title of the specific article
    art_title = ET.SubElement(item, "title")
    art_title.text = "Introduction to HTTP/3"

    # Direct link to the article
    art_link = ET.SubElement(item, "link")
    art_link.text = "https://ravencloud.edu/http3"

    # We generate the representation in string format
    # encoding='unicode' ensures the text is treated correctly
    xml_data = ET.tostring(root, encoding='unicode')

    # We return the final result to be sent by the server
    return xml_data

# We call the function and print it (as the server would to the client)
# print() is the standard function to show data via console
print(generate_rss())

Consumption: How is RSS read?

To "consume" the RSS, the client must make an asynchronous call (AJAX or Fetch) and interpret the text. Here is an example of how a modern Frontend (using JavaScript) could process this data:

// We use the fetch function to request the XML file from the server
// 'feed_url' would be the address where the RSS file resides
fetch('https://library.ravencloud.edu/feed.xml')
  // The .then method handles the response when it arrives
  .then(response => response.text()) // We convert the response to plain text
  .then(str => {
    // DOMParser is a native browser tool to parse XML/HTML
    const parser = new DOMParser();

    // We convert the text string into a usable XML Document object
    // 'application/xml' indicates the content type we are parsing
    const xml = parser.parseFromString(str, "application/xml");

    // We search for all elements tagged as <item>
    const items = xml.querySelectorAll("item");

    // We iterate over each found item
    items.forEach(el => {
      // For each element, we extract the text from the <title> tag
      const title = el.querySelector("title").innerHTML;

      // We print the title to the browser console
      // console.log() sends information to the debugging tool
      console.log("New article found: " + title);
    });
  })
  // .catch handles possible network or syntax errors
  .catch(err => console.error("Error in communication:", err));

Ravencloud Student's Rigorous Summary

RSS is nothing more than a data interface that separates content from form. While a website's Frontend worries about typography and colors, RSS cares only about semantics and availability. In the era of closed algorithms on social networks, RSS is an act of resistance for information decentralization.

Internal Question: I understand that the backend, in the case of a blog for example, depends on the technology used. Does this blog query the database every time there is an insertion, modification, or deletion action and update the XML file (which must be hosted somewhere on the server), maintaining it automatically every time the database content is updated?

Two Backend Strategies for RSS

Static Generation (My hypothesis)

Every time an article is published (Insert/Update), the backend overwrites a physical file named feed.xml on the server's hard drive.

Advantage: It is incredibly fast to serve (the web server only has to deliver a text file).
Disadvantage: If there are thousands of articles or many changes, constantly writing to disk is inefficient and can generate concurrency problems (file locks).

Dynamic Generation (The Ravencloud Standard)

In this architecture, the feed.xml file does not exist physically. When a user requests the URL myblog.com/rss, the backend intercepts the request, queries the database at that instant, assembles the XML in RAM, and "fires" it into the network.

Comparison of Data Flow

Feature	Static Generation	Dynamic Generation (On-the-fly)
Calculation Moment	When content is updated.	When someone requests the feed.
CPU Load	Low in reading, high in editing.	High in reading, null in editing.
Freshness	Depends on the last write.	Always 100% updated.
Storage	Occupies disk space (HDD/SSD).	Only occupies temporary memory (RAM).

Rigorous Implementation: The Dynamic Backend

Imagine we are using Node.js with the Express framework (very common in PaaS architectures). Here we will see how the backend generates the RSS without the file being saved anywhere.

// We import the 'express' module to manage network requests
// require is the function to load external libraries in Node.js
const express = require('express');

// We initialize the server application
const app = express();

// We define the route that will listen for RSS requests
// app.get defines a response for the HTTP GET method
app.get('/rss', (req, res) => {

    // We simulate a query to the Database (DB)
    // In a real case, here we would do a: SELECT * FROM articles
    const dbArticles = [
        { title: "The Art of Networks", desc: "How Ravencloud connects the world." },
        { title: "HTTP/3 Protocol", desc: "The future of low latency." }
    ];

    // We start building the XML string
    // We use 'let' to declare a variable that will change
    let xml = '<?xml version="1.0" encoding="UTF-8" ?>';

    // We add the root tag of the RSS protocol
    xml += '<rss version="2.0">';

    // We open the channel
    xml += '<channel>';

    // We add the mandatory metadata
    xml += '<title>Ravencloud Blog</title>';

    // We iterate over each article retrieved from the database
    // .forEach is a method that executes a function for each element in the list
    dbArticles.forEach(article => {
        // For each article, we create an <item> entry in the XML
        xml += '<item>';
        // We add the article title inside the <title> tag
        xml += `<title>${article.title}</title>`;
        // We add the description inside the <description> tag
        xml += `<description>${article.desc}</description>`;
        // We close the article entry
        xml += '</item>';
    });

    // We close the main tags
    xml += '</channel>';
    xml += '</rss>';

    // IMPORTANT: We indicate to the browser/client that what we are sending is XML, not HTML
    // res.set configures the 'Content-Type' header of the HTTP response
    res.set('Content-Type', 'text/xml');

    // We send the final text string to the client via the network
    // res.send finalizes the request and sends the data
    res.send(xml);
});

// The server starts listening on port 3000
// app.listen activates the server process on the local network
app.listen(3000, () => {
    // console.log shows a message to confirm the service is active
    console.log("Ravencloud server broadcasting RSS on port 3000");
});

The Role of Frontend and Caching

In high-performance systems, we do not query the database every time someone requests the RSS (this could saturate the backend if there are thousands of simultaneous requests).

Frontend/Consumer: The RSS reader (like Feedly or a widget on another site) makes the request.
Intermediate Layer (Proxy/Cache): A server like Nginx or a tool like Redis keeps a copy of the XML in RAM for, say, 5 minutes.
Result: The backend only works once every 5 minutes, but thousands of users receive the information at light speed.

This is the true magic of network architecture: optimizing the data path.

Internal Question: There was a time on the Internet when RSS was very common, but lately I have the feeling it is in disuse. Why do I think it is no longer used? Why have they been replaced?

The Decline of RSS and the Rise of Walled Gardens

RSS was born in an era where the web aspired to be decentralized and open. Today, we are in the era of "Walled Gardens". From our technical and networking perspective, let's analyze why this protocol has taken a back seat and why it has been replaced.

The Conflict of Interests: The Attention Economy

RSS is a "Pull" protocol: the client decides when and what to read. Current major platforms prefer the "Push & Engagement" model.

Loss of Monetization: If you read content through an RSS reader, the website owner cannot show you ads, cannot analyze your browsing behavior with cookies, and cannot retain you with an algorithm.
Algorithm Control: RSS is purely chronological. Platforms like Facebook, X (Twitter), or Instagram want to decide what you see and in what order to maximize the time you spend inside their network.

The Technical Substitute: From "Pull" to "Push"

RSS requires your client to ask the server: "Is there anything new?". This is inefficient on mobile devices with limited battery. The world has moved towards Push Notifications and WebSockets.

Feature	RSS (Legacy)	Notifications/Algorithms (Modern)
Model	Pull (Client asks)	Push (Server pushes)
Protocol	HTTP (Static XML)	WebSockets / HTTP/2 (Binary/Full-duplex)
Privacy	High (Anonymous consumption)	Low (Monitored consumption)
Interaction	One-way (Read)	Bidirectional (Like, Share, Comment)

Current Substitutes by Usage

RSS hasn't died completely, but it has been fragmented into other technologies:

Social Networks: Replace RSS as a source of news discovery.
Newsletters (Substack, etc.): Have moved syndication towards email (SMTP protocol), which allows a direct and monetizable relationship with the user.
Platform APIs: Many services have closed their RSS feeds to force developers to use their JSON APIs (with access keys and request limits).

Technical Example: The Paradigm Shift (Push vs. Pull)

While RSS expected you to make a request, modern applications use technologies like WebSockets for real-time communication. Here is an example of how information is "consumed" today in a modern network environment:

// Instead of downloading an XML every X minutes (RSS),
// we open a constant communication tunnel with the server.

// We create a WebSocket connection with the news server
// WebSocket is a network protocol that allows bidirectional communication
const socket = new WebSocket('wss://news.ravencloud.edu');

// We listen for the 'open' event, which indicates the tunnel is ready
socket.onopen = function(event) {
    // console.log sends a debug message to the console
    console.log("Connection established with real-time data flow.");
};

// This event triggers automatically when the SERVER has news
// Unlike RSS, here we don't have to ask for anything; data "arrives" on its own
socket.onmessage = function(event) {
    // Data usually arrives in JSON format, lighter than RSS XML
    // JSON.parse converts a string into a JavaScript object
    const newsItem = JSON.parse(event.data);

    // We show the new news item that the server has "pushed" (Push)
    // the + operator concatenates (joins) strings
    console.log("Breaking news: " + newsItem.title);
};

// We manage possible errors in the network infrastructure
socket.onerror = function(error) {
    // console.error shows the error message in red in the console
    console.error("Data flow error: ", error);
};

Ravencloud House Conclusion

RSS was the open library of the web; current systems are private clubs. Technically, RSS is superior in simplicity and privacy, but it has lost the battle against the convenience of mobile apps and the data business model.

However, RSS remains the backbone of Podcasting (without RSS, podcasts as we know them wouldn't exist) and many automated workflows between servers.

Internal Question: I think there are not many ways to convert that pull-type information into push-type. There is the option mentioned before of websockets, or there is that of a bot (in this case on Telegram).

House-elf: RSS on Telegram BOT

In distributed network architecture, this pattern is called a Polling-to-Push Bridge. The intermediary server does the "heavy lifting" of constantly asking (Pull) to save work for the end client, who only receives information when it is relevant (Push).

The System Architecture

To perform this transformation, we need three components connected through application protocols (Layer 7 of the OSI model):

The Source (RSS): A static web server offering an XML file.
The Bridge (Our Bot): A script running on a server (IaaS) or container.
The Destination (Telegram API): A service that receives HTTPS requests and transforms them into instant notifications to the user's device.

Technical Implementation (Python)

We will use Python for its efficiency in handling network requests. This script will act as our "watchman" in the tower.

# We import the 'requests' library to make HTTP requests to the network
# 'import' is the keyword to load external modules
import requests

# We import 'time' to manage time intervals between queries
import time

# We define configuration constants (Network Metadata)
# The TOKEN is provided by BotFather on Telegram
TOKEN_BOT = "YOUR_TOKEN_HERE"
# The Chat ID where we want to send the notification
CHAT_ID = "YOUR_CHAT_ID"
# The URL of the RSS we want to monitor
RSS_URL = "https://library.ravencloud.edu/feed.xml"

# Variable to store the last read news item and avoid duplicates
# Initialized as None (empty)
last_news = None

def send_telegram_notification(message):
    """
    Function that transforms internal data into an HTTPS request to Telegram.
    """
    # We build the Telegram API URL using 'f-strings' to insert variables
    # The 'sendMessage' method is part of the Telegram protocol
    api_url = f"https://api.telegram.org/bot{TOKEN_BOT}/sendMessage"

    # We create a dictionary with the request parameters
    # 'chat_id' indicates the recipient and 'text' the content
    payload = {
        "chat_id": CHAT_ID,
        "text": message
    }

    # We send a POST request across the Internet
    # requests.post() performs a data write to the Telegram server
    requests.post(api_url, data=payload)

def monitor_rss():
    """
    Main loop that executes the 'Polling' logic.
    """
    # We reference the global variable to be able to modify it
    global last_news

    # 'while True' creates an infinite loop for constant execution
    while True:
        try:
            # We make a GET request to "pull" the RSS XML
            response = requests.get(RSS_URL)

            # The content of the response is found in 'response.text'
            content = response.text

            # (Simplification) We search for the title of the first <item> inside the XML
            # In a production environment, we would use a rigorous XML parser

            # First we find the start of the first item to skip the channel title
            item_start = content.find("<item>")

            # If there are no items, we skip this iteration
            if item_start == -1:
                continue

            # We search for the title tag starting from the item position
            start = content.find("<title>", item_start) + 7
            end = content.find("</title>", start)
            current_title = content[start:end]    # We extract the text fragment

            # We compare if the title has changed compared to the last time
            if current_title != last_news:
                # If it is new, we update the variable
                last_news = current_title

                # We call the function that does the "Push" to Telegram
                send_telegram_notification(f"New entry at Ravencloud: {current_title}")

        except Exception as e:
            # If there is a network error, we show it on screen
            # print() writes to the standard output stream
            print(f"Connection error: {e}")

        # We stop execution for 60 seconds to not saturate the network
        # time.sleep() suspends the current execution thread
        time.sleep(60)

# Program entry point
# Checks if the script is running directly
if __name__ == "__main__":
    monitor_rss()

Data Flow Analysis (Network Flow)

This script radically changes the communication topology:

From RSS to Bot (Pull): The bot makes an HTTP GET request every 60 seconds. This is known as Active Polling.
From Bot to Telegram (Webhook/API): When a change is detected, the bot makes an HTTP POST request to Telegram servers.
From Telegram to Mobile (Push): Telegram maintains a persistent connection (often via a protocol called MTProto over TCP) with your mobile. When they receive your POST, they "push" the notification to your device instantly.

Where should this run? (Infrastructure)

Since the bot must be "alive" 24 hours a day, we cannot run it on our personal computer (which turns off or loses Wi-Fi). We need a solution from our specialty at Ravencloud:

VPS (Virtual Private Server): A minimal virtual machine (like EC2 on AWS or a Droplet on DigitalOcean).
Serverless Functions: You could program a function (AWS Lambda) that runs every 5 minutes automatically.
Containers (Docker): You can package the script and deploy it on a Kubernetes cluster to ensure that, if it fails, it restarts itself.

Security Note: Never leave the TOKEN_BOT written directly in the code if you upload it to a public place like GitHub. Use Environment Variables.

Search

IT Spellbook