自动化爬虫进阶：巧用Python + Selenium轻松抓取表格数据

2023-12-16 07:49:07

一、准备工作

安装必要的库

pip install selenium
pip install openpyxl

导入必要的库

import selenium
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from openpyxl import Workbook

二、代码实现

打开浏览器并访问目标网页

driver = webdriver.Chrome()
driver.get("https://www.example.com/table.html")

等待表格元素加载完成

WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.ID, "table-id"))
)

获取表格数据

table = driver.find_element_by_id("table-id")
rows = table.find_elements_by_tag_name("tr")

将表格数据保存到Excel文件中

workbook = Workbook()
sheet = workbook.active
sheet.title = "表格数据"

for row in rows:
    cells = row.find_elements_by_tag_name("td")
    for cell in cells:
        sheet.cell(row=row.get_attribute("rowIndex"), column=cell.get_attribute("cellIndex")).value = cell.text

workbook.save("表格数据.xlsx")