How to Stream Selenium Web Automation using Headless Chrome and Flask in Python

May 25, 2024 • Python, Programming • By

Selenium makes it easy to automate your browser and take control of web pages for testing and automation. But have […]

Selenium makes it easy to automate your browser and take control of web pages for testing and automation. But have you ever thought about streaming your Selenium browser sessions in real time? In this post, I’ll guide you through setting up a headless Chrome browser automated by Selenium, and streaming its output through a simple Flask server. This can be particularly useful for remote monitoring of web automation, showcasing automated tasks, or for educational purposes.

If you’re an advanced programmer, just go here to see the full code 🙂

Setting Up the Environment

Before diving into the code, ensure you have the following prerequisites installed:

  • Python (obviously!)
  • Flask (pip install flask)
  • Selenium (pip install selenium)
  • ChromeDriver (compatible with your Chrome version)

The Core Idea

In this example, I’m simply going to go on https://iamrizwan.me and perform actions like scrolling up and down every 3 seconds. Meanwhile, we’ll capture the browser’s output as a series of screenshots and stream these images live using a Flask server.

Key Components of the Code

1. Selenium WebDriver Setup

Setting up the Selenium WebDriver for headless Chrome browsing involves a few critical steps that enable our script to automate a browser session without a graphical user interface. This is particularly important for running the script on servers or systems without a display. It allows the script to run in the background while performing tasks as a regular browser would.

We begin by importing WebDriver from Selenium and setting up Chrome options. The Chrome options are crucial for configuring the browser for headless operation. 

from selenium import webdriver

options = webdriver.ChromeOptions()

options.add_argument('--headless')
options.add_argument('--disable-gpu')

driver = webdriver.Chrome(options=options)

2. Web Automation

We’ll navigate to https://iamrizwan.me and perform a smooth scrolling action using JavaScript injected through Selenium continuously in a loop. This mimics user interaction with the webpage. You can change this part for your own automation.

def scroll_page(driver):
    try:
        driver.get("https://iamrizwan.me/")

        while True:
            time.sleep(3)
            driver.execute_script("window.scrollTo({top: document.body.scrollHeight, behavior: 'smooth'});")
            time.sleep(3)
            driver.execute_script("window.scrollTo({top: 0, behavior: 'smooth'});")

    except Exception as e:
        print(f"Error in selenium_automation: {e}")

3. Flask Streaming Server

Now we’ll setup Flask application to stream the images captured from the Selenium browser. We use a generator function to capture screenshots and yield them in a format that Flask can stream. But before that’ we’ll define a function to generate current screenshot of the webdriver:

def generate_frames(driver):
    while True:
        # Capture screenshot
        screenshot = driver.get_screenshot_as_png()
        img = Image.open(io.BytesIO(screenshot))

        # Convert RGBA to RGB
        if img.mode == 'RGBA':
            img = img.convert('RGB')

        # Convert image to JPEG format
        img_byte_arr = io.BytesIO()
        img.save(img_byte_arr, format='JPEG')
        img_byte_arr = img_byte_arr.getvalue()

        # Yield a frame for streaming
        yield (b'--frame\r\n'
               b'Content-Type: image/jpeg\r\n\r\n' + img_byte_arr + b'\r\n')

And here’s the Flask route to send this screenshot as a stream:

@app.route('/video_feed')
def video_feed():
    return Response(generate_frames(driver),
                    mimetype='multipart/x-mixed-replace; boundary=frame')

4. Thread Management

We’ll use Python’s threading to run the web automation and the Flask server simultaneously. This allows continuous interaction with the webpage while streaming the output:

if __name__ == '__main__':
    # Set up headless Chrome
    options = webdriver.ChromeOptions()
    options.add_argument('--headless')
    options.add_argument('--window-size=1728,903')
    driver = webdriver.Chrome(options=options)

    # Navigate to the page
    driver.get('https://iamrizwan.me')

    # Start a thread for scrolling
    scroll_thread = Thread(target=scroll_page, args=(driver,))
    scroll_thread.start()

Finally, serve the server on a specific port:

    # Start the Flask app
    app.run(host='0.0.0.0', port=8000, threaded=True)

5. Running the Code and Viewing the Stream

Once you run the script, the headless browser starts, and the Flask server begins streaming. This can be accessible on http://localhost:8000/video_feed. But since you’re running it on a remote machine, you’ll need to make sure you have access to this port, for that we’ll use SSH tunnelling to forward that port to you.

6. SSH Tunnel

You need to connect to your server using a ssh but with -L flag to specify the port. Here’s a sample command:

ssh -L 8000:localhost:8000 user@your-server-ip-address

Or if you use Public Key authentication then:

ssh -L 8000:localhost:8000 -i /path/to/your/private/key user@your-server-ip-address

Now you can view the live stream by navigating to http://localhost:8000/video_feed in any web browser.

Applications and Considerations

This innovative integration of Selenium and Flask for streaming a headless browser session has a broad spectrum of applications, each with its unique value and considerations.

  1. Web Scraping and Data Collection: For those involved in data collection from websites, this setup can visually monitor scraping processes in real-time, ensuring that the scripts are functioning as intended.
  2. Automated Testing: It’s a boon for QA engineers and developers who need to observe how their websites behave under different automated interactions, making debugging and verification more intuitive.
  3. Educational and Demonstrative Purposes: Educators and trainers can use this setup to teach web automation and browser dynamics. It’s an excellent tool for live demonstrations, making complex concepts more accessible.
  4. Remote Monitoring: For instances where remote monitoring of web interactions is necessary, such as in security audits or compliance checks, this streaming capability allows real-time oversight.

Conclusion

Combining Selenium and Flask to stream a browser session is a clever and useful trick. It shows how powerful Python and its tools can be when used together. This setup is great for anyone who needs to see what’s happening in a browser automatically, whether for checking websites, teaching, or just for fun. It’s a straightforward yet effective way to blend web browsing with live streaming. As we keep finding new ways to use technology, solutions like this one open doors to exciting and practical possibilities.

Here’s the Full Code

Related Posts