Schedule execution every day at 8:00 AM

In the previous tutorial, we discussed how to automate batch data processing using Python. Today, we’ll explore another practical use case: setting up scheduled automation tasks. These tasks involve running specific Python operations at regular intervals—such as fetching data, generating reports, or sending emails—without manual intervention. Scheduled automation is highly valuable in daily workflows, significantly improving efficiency and reducing repetitive manual effort.

Use Case Background

Suppose we need to regularly fetch weather data from an API and log it into a CSV file. We want this task to run automatically every day, with no manual involvement required. To achieve this, we’ll use the schedule library.

Implementation Steps

1. Install Required Libraries

First, install the requests, schedule, and pandas libraries. Open your terminal or command prompt and run:

pip install requests schedule pandas

2. Write a Python Script to Fetch and Save Weather Data

Below is a simple Python script that retrieves weather data from an API and saves it to a CSV file. It uses requests to fetch the data and pandas to process and persist it.

import requests
import pandas as pd
from datetime import datetime

def fetch_weather_data(city):
    # Replace with your actual API key
    api_key = 'YOUR_API_KEY'
    url = f'http://api.openweathermap.org/data/2.5/weather?q={city}&appid={api_key}&units=metric'
    response = requests.get(url)
    data = response.json()
    
    if response.status_code == 200:
        weather_data = {
            'date': datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
            'city': data['name'],
            'temperature': data['main']['temp'],
            'humidity': data['main']['humidity'],
            'weather': data['weather'][0]['description']
        }
        return weather_data
    else:
        print("Error fetching data:", data)
        return None

def save_to_csv(data, filename='weather_data.csv'):
    df = pd.DataFrame(data)
    df.to_csv(filename, mode='a', header=not pd.io.common.file_exists(filename), index=False)

if __name__ == "__main__":
    city = 'Beijing'  # The city for which you want weather data
    weather_data = fetch_weather_data(city)
    
    if weather_data:
        save_to_csv([weather_data])

3. Schedule the Task

Next, we’ll use the schedule library to run the above logic daily. We’ll integrate the scheduling logic directly into the same script.

import schedule
import time

def job():
    city = 'Beijing'
    weather_data = fetch_weather_data(city)
    
    if weather_data:
        save_to_csv([weather_data])
        print(f"Saved weather data for {city}.")

# Schedule execution every day at 8:00 AM
schedule.every().day.at("08:00").do(job)

if __name__ == "__main__":
    while True:
        schedule.run_pending()
        time.sleep(60)  # Check for pending jobs every minute

4. Run the Script

Save the complete script (including both data-fetching and scheduling logic) as weather_scheduler.py, then execute it from the command line:

python weather_scheduler.py

This will keep the script running in the background, automatically fetching and saving weather data for Beijing to weather_data.csv every morning at 8:00 AM.

Summary

In this section, we built a scheduled automation task that periodically fetches weather data via an API and stores it in a CSV file. This kind of automated workflow eliminates manual effort and ensures consistent, timely data updates—providing reliable input for downstream analysis or reporting.

In the next (and final) tutorial, we’ll summarize the entire series, review key concepts covered, and discuss how to apply these automation techniques in real-world scenarios. We encourage you to continue practicing and experimenting with Python-based automation—building fluency through hands-on experience and steadily enhancing your technical capabilities.