Read the CSV file

In previous tutorials, we explored how to implement automated login functionality using Python. Today, we continue our journey by diving deeper into batch data processing with Python—a highly practical skill, especially when handling large volumes of data, significantly boosting work efficiency.

What Is Batch Data Processing?

As the name suggests, batch data processing refers to performing operations on multiple data entries simultaneously. Such operations commonly include data cleaning, data transformation, or data analysis. With Python, these tasks can be implemented effortlessly. Next, we’ll illustrate how to achieve this through a concrete example.

Case Background

Suppose we have a CSV file (customers.csv) containing customer information, structured as follows:

id,name,email,age
1,John Doe,john@example.com,28
2,Jane Smith,jane@example.com,34
3,Bob Johnson,bob@example.com,45

Our goal is to increment the age field for all customers by 1 year and save the updated records into a new CSV file (updated_customers.csv).

Environment Setup

First, ensure the required library is installed on your machine. We’ll use the pandas library to handle CSV files. Install it via the following command:

pip install pandas

Implementation Steps

Step 1: Read the CSV File

We begin by loading the original CSV file using pandas.

import pandas as pd

# Read the CSV file
df = pd.read_csv('customers.csv')
print("Original customer data:")
print(df)

Step 2: Update the Age Column

Next, we modify the age column by adding 1 to each value.

# Increment age by 1
df['age'] = df['age'] + 1
print("Updated customer data:")
print(df)

Step 3: Save the Updated Data

Finally, we write the modified DataFrame to a new CSV file.

# Save updated data to a new file
df.to_csv('updated_customers.csv', index=False)
print("Updated data saved to updated_customers.csv")

Complete Code

Combining all steps above yields the full script for batch data processing:

import pandas as pd

# Read the CSV file
df = pd.read_csv('customers.csv')
print("Original customer data:")
print(df)

# Increment age by 1
df['age'] = df['age'] + 1
print("Updated customer data:")
print(df)

# Save updated data to a new file
df.to_csv('updated_customers.csv', index=False)
print("Updated data saved to updated_customers.csv")

Execution Output

Upon running the code, the console will display both the original and updated customer data. Additionally, a new file named updated_customers.csv will appear in the current directory, containing the following content:

id,name,email,age
1,John Doe,john@example.com,29
2,Jane Smith,jane@example.com,35
3,Bob Johnson,bob@example.com,46

Summary

Through this simple example, we demonstrated how to perform batch data processing in Python—including reading and writing CSV files and updating data fields. This represents only one basic application scenario; real-world requirements often involve more complex operations such as data cleaning, merging, pivoting, or reshaping.

In the next tutorial, we’ll explore scheduled automation tasks, further expanding our capabilities in the domain of automation. Through these case studies, we aim to help you master fundamental rules and techniques of automation—enabling you to handle diverse tasks more efficiently in your daily work.