使用 pymysql 插入 table

Question

我正在解析一个网站并想插入到 MySQL。特定字段下有6个字段需要解析每个url。任何人都知道如何使用 pymysql 将 MySQL table 转换为 Python 中的字典对象？

from bs4 import BeautifulSoup
import requests
import urllib.request
import pymysql

con = pymysql.connect(host = 'localhost',user = 'root',passwd = 'root',db = 'm_db')
cursor = con.cursor(pymysql.cursors.DictCursor)

with open(r"C:\New folder\urllist.txt") as f:
    urls = [u.strip('\n') for u in f.readlines()]
    page = 0
    while page < 5:
        try:
            soup = BeautifulSoup(requests.get(urls[page]).content, "html.parser")
            text = soup.select("head script[type=text/javascript]")[-1].text
            start = text.find('dataLayer = [') + len('dataLayer = [')
            end = text.rfind('];')
            rows = text[start:end].strip().split('\n')
        except:
            pass
        for d in rows:
            cursor.execute("INSERT INTO micro (d['brand'], d['productPrice'], d['SKU'], d['productID'], d['mpn'], d['ean'], ) VALUES (%s,%s,%s,%s,%s,%s)",(d['brand'], d['productPrice'], d['SKU'], d['productID'], d['mpn'], d['ean']))
            d = cursor.fetchall()
        con.commit()
        page = page + 1

如何用我拥有的数据填充 SQL 插入语句。这是我的数据

{
     'brand':'Fluke',
     'productPrice':'2199.99',
     'SKU':'34353362',
     'productID':'443329',
     'mpn':'RT-AC3200',
     'ean':'886290480914',
}

它产生这个错误

Traceback (most recent call last): File "C:/PycharmProjects/untitled4/m_new.py", line 22, 
in <module> cursor.execute("INSERT INTO micro (d['brand'], d['productPrice'], d['SKU'], d['productID'], d['mpn'], d['ean'], ) VALUES (%s,%s,%s,%s,%s,%s)",(d['brand'], d['productPrice'], d['SKU'], d['productID'], d['mpn'], d['ean'])) 
TypeError: string indices must be integers

Answer 1

您的SQL查询不正确。应该是

cursor.execute("INSERT INTO micro (brand, productPrice, SKU, productID, mpn, ean) VALUES (%s,%s,%s,%s,%s,%s)",(d['brand'], d['productPrice'], d['SKU'], d['productID'],d['mpn'], d['ean']))

使用 pymysql 插入 table

use pymysql to insert table

python

mysql

web-scraping

pymysql