PDF 文本提取并将它们存储为键值对
PDF Text extraction and storing them as key-value pair
我想从 PDF 中提取文本。我从文本提取中获得的输出没有那么有条理。
PDF Link（仅第一页）：https://microprecision.com/wp-content/uploads/2020/08/Sample-Cert_rev-7-1.pdf
我想提取 MPC Control No #、Serial No.、Model Number 等参数，并将它们作为键值对存储在字典中。
我正在尝试使用以下代码，但未获得所需的输出。
import io
from pdfminer.layout import LAParams, LTTextBox
from pdfminer3.pdfpage import PDFPage
from pdfminer3.pdfinterp import PDFResourceManager,PDFPageInterpreter
from pdfminer3.converter import PDFPageAggregator,TextConverter
def pdftotext(path):
resource_manager = PDFResourceManager()
file_handle = io.StringIO()
laprams = LAParams(word_margin=1.0,boxes_flow=0.5,char_margin=2.0,line_overlap=0.5,line_margin=0.5)
converter = TextConverter(resource_manager,file_handle, laparams=laprams)
page_interpreter = PDFPageInterpreter(resource_manager,converter)
i = 1
with open(path,'rb') as fh:
for page in PDFPage.get_pages(fh,caching=False,check_extractable=True):
page_interpreter.process_page(page)
text = file_handle.getvalue()
converter.close()
file_handle.close()
return text
raw = pdftotext('Sample-Certificate.pdf')
print(raw)
确实，您没有得到键值对，而且 pdfminer 根本无法提供给您。它只会从 PDF 中提取文本（+ 可能还有一些附加信息）。
要获得良好的逻辑标签值对，您需要先使用 Information Extraction approach and / or Named Entity Recognition on top of the extracted text. There are plenty of options here. You may want to have a look at SpaCy or NLTK。
通常，从文档中提取有意义的数据及其关系有一个新的性感名称 Document Intelligence。
在处理 PDF 文件时，我更喜欢使用 PyMuPDF 库 https://pypi.org/project/PyMuPDF/
import fitz
txt = []
doc = fitz.open("Sample-Cert_rev-7-1.pdf") # some existing PDF
page = doc[0]
text = page.getText("text")
txt = list(text)
print(text)
text = text.split('\n')
txt = list(text)
print(txt)
ix = text.index('MPC Control #:')
print(ix)
print(text[ix+18])
注意如何正确安装库
这是输出：
"C:\Program Files\Python38\python.exe" C:/Python/Whosebug extract_pdf_text1.py
MICRO PRECISION CALIBRATION, INC.
22835 INDUSTRIAL PLACE
GRASS VALLEY CA 95949
530-268-1860
Cert No.
551220083746791
Date: Aug 3, 2020
Certificate of Calibration
AC-1969.00
N/A
July 01, 2021
N/A
Customer:
MPC Control #:
Asset ID:
Gage Type:
Manufacturer:
Model Number:
Size:
Temp/RH:
Serial Number:
Department:
Performed By:
Received Condition:
Returned Condition:
Cal. Date:
Cal. Interval:
Cal. Due Date:
Work Order #:
DIGITAL MULTIMETER
DANNY BOY B. BUTIAL
0258964
0258964
NONE
AGILENT
34401A
10MHZ
SAMPLE
N/A
IN TOLERANCE
IN TOLERANCE
July 01, 2020
N/A
12 MONTHS
Calibration Notes:
SAMPLE COMPANY
23.0°C / 40.0%
Location:
Calibration performed at MPC facility
Standards Used to Calibrate Equipment
I.D.
Description.
Model
Serial
Manufacturer
Cal. Due Date
Traceability #
PH1405
MULTI-PRODUCT CALIBRATOR
5520A
7575006
FLUKE
Sep 10, 2020
551220083204793
AL4394
DIGITAL MULTIMETER
3458A
2823A09832
AGILENT
Aug 1, 2020
551220083719099
Procedures Used in this Event
Procedure Name
Description
MPC Automated Procedure
MPCCAL Rev. 00
STATEMENTS OF PASS OR FAIL CONFORMANCE: The uncertainty of measurement has been taken into account when determining compliance with specification. All measurements and test results guard banded to ensure the
probability of false-accept does not exceed 2% in compliance with ANSI/NCSL Z540.3-2006 and in case without guard banded the probability of false-accept depending on test uncertainty ratio.
THE CALIBRATION REPORT STATUS:
PASS- Term used when compliance statement is given, and the measurement result is PASS.
PASSz- Term used when compliance statement is given, and the measurement result is conditional passed or PASSz.
FAIL- Term used when compliance statement is given, and the measurement result is FAIL.
FAILz- Term used when compliance statement is given, and the measurement result is conditional failed or FAILz.
REPORT OF VALUE - Term used when reported measurement is not requiring compliance statement in report.
ADJUSTED- When adjustments are made to an instrument which changes the value of measurement from what was measured as found to new value as left.
LIMITED - When an instrument fails calibration but is still functional in a limited manner.
The expanded uncertainty of measurement is stated as the standard uncertainty of measurement multiplied by the coverage factor k=2, which for a normal distribution corresponds to a coverage probability of approximately 95%, unless otherwise stated. This
calibration report complies with ISO/IEC 17025:2017 and ANSI/NCSL Z540.3. Calibration cycles and resulting due dates were submitted/approved by the customer. Any number of factors may cause an instrument to drift out of tolerance before the next
scheduled calibration. Recalibration cycles should be based on frequency of use, environmental conditions and customer's established systematic accuracy. All standards are traceable to SI through the National Institute of Standards and Technology (NIST)
and/or recognized national or international standards laboratories. Services rendered include proper manufacturer’s service instruction and are warranted for no less than thirty (30) days. The information on this report pertains only to the instrument identified,
this may not be reproduced in part or in a whole without the prior written approval of the issuing MP Calibration Laboratory.
Rick Hernandez
Calibrating Technician:
QC Approval:
DANNY BOY B. BUTIAL
(CERT, Rev 7)
Page 1 of 1
['MICRO PRECISION CALIBRATION, INC.', '22835 INDUSTRIAL PLACE', 'GRASS VALLEY CA 95949', '530-268-1860', 'Cert No.', '551220083746791', 'Date: Aug 3, 2020', 'Certificate of Calibration', 'AC-1969.00', 'N/A', 'July 01, 2021', 'N/A', 'Customer:', 'MPC Control #:', 'Asset ID:', 'Gage Type:', 'Manufacturer:', 'Model Number:', 'Size:', 'Temp/RH:', 'Serial Number:', 'Department:', 'Performed By:', 'Received Condition:', 'Returned Condition:', 'Cal. Date:', 'Cal. Interval:', 'Cal. Due Date:', 'Work Order #:', 'DIGITAL MULTIMETER', 'DANNY BOY B. BUTIAL', '0258964', '0258964', 'NONE', 'AGILENT', '34401A', '10MHZ', 'SAMPLE', 'N/A', 'IN TOLERANCE', 'IN TOLERANCE', ' July 01, 2020', 'N/A', '12 MONTHS', 'Calibration Notes:', 'SAMPLE COMPANY', '23.0°C / 40.0%', 'Location:', 'Calibration performed at MPC facility', 'Standards Used to Calibrate Equipment', 'I.D.', 'Description.', 'Model', 'Serial', 'Manufacturer', 'Cal. Due Date', 'Traceability #', 'PH1405', 'MULTI-PRODUCT CALIBRATOR', '5520A', '7575006', 'FLUKE', 'Sep 10, 2020', '551220083204793', 'AL4394', 'DIGITAL MULTIMETER', '3458A', '2823A09832', 'AGILENT', 'Aug 1, 2020', '551220083719099', 'Procedures Used in this Event', 'Procedure Name', 'Description', 'MPC Automated Procedure', 'MPCCAL Rev. 00', 'STATEMENTS OF PASS OR FAIL CONFORMANCE: The uncertainty of measurement has been taken into account when determining compliance with specification. All measurements and test results guard banded to ensure the', 'probability of false-accept does not exceed 2% in compliance with ANSI/NCSL Z540.3-2006 and in case without guard banded the probability of false-accept depending on test uncertainty ratio.', 'THE CALIBRATION REPORT STATUS:', 'PASS- Term used when compliance statement is given, and the measurement result is PASS.', 'PASSz- Term used when compliance statement is given, and the measurement result is conditional passed or PASSz.', 'FAIL- Term used when compliance statement is given, and the measurement result is FAIL.', 'FAILz- Term used when compliance statement is given, and the measurement result is conditional failed or FAILz.', 'REPORT OF VALUE - Term used when reported measurement is not requiring compliance statement in report.', 'ADJUSTED- When adjustments are made to an instrument which changes the value of measurement from what was measured as found to new value as left.', 'LIMITED - When an instrument fails calibration but is still functional in a limited manner.', 'The expanded uncertainty of measurement is stated as the standard uncertainty of measurement multiplied by the coverage factor k=2, which for a normal distribution corresponds to a coverage probability of approximately 95%, unless otherwise stated. This', 'calibration report complies with ISO/IEC 17025:2017 and ANSI/NCSL Z540.3. Calibration cycles and resulting due dates were submitted/approved by the customer. Any number of factors may cause an instrument to drift out of tolerance before the next', "scheduled calibration. Recalibration cycles should be based on frequency of use, environmental conditions and customer's established systematic accuracy. All standards are traceable to SI through the National Institute of Standards and Technology (NIST)", 'and/or recognized national or international standards laboratories. Services rendered include proper manufacturer’s service instruction and are warranted for no less than thirty (30) days. The information on this report pertains only to the instrument identified,', 'this may not be reproduced in part or in a whole without the prior written approval of the issuing MP Calibration Laboratory.', 'Rick Hernandez', 'Calibrating Technician:', 'QC Approval:', 'DANNY BOY B. BUTIAL', '(CERT, Rev 7)', 'Page 1 of 1', '']
13
0258964
Process finished with exit code 0
我想从 PDF 中提取文本。我从文本提取中获得的输出没有那么有条理。
PDF Link（仅第一页）：https://microprecision.com/wp-content/uploads/2020/08/Sample-Cert_rev-7-1.pdf
我想提取 MPC Control No #、Serial No.、Model Number 等参数，并将它们作为键值对存储在字典中。
我正在尝试使用以下代码，但未获得所需的输出。
import io
from pdfminer.layout import LAParams, LTTextBox
from pdfminer3.pdfpage import PDFPage
from pdfminer3.pdfinterp import PDFResourceManager,PDFPageInterpreter
from pdfminer3.converter import PDFPageAggregator,TextConverter
def pdftotext(path):
resource_manager = PDFResourceManager()
file_handle = io.StringIO()
laprams = LAParams(word_margin=1.0,boxes_flow=0.5,char_margin=2.0,line_overlap=0.5,line_margin=0.5)
converter = TextConverter(resource_manager,file_handle, laparams=laprams)
page_interpreter = PDFPageInterpreter(resource_manager,converter)
i = 1
with open(path,'rb') as fh:
for page in PDFPage.get_pages(fh,caching=False,check_extractable=True):
page_interpreter.process_page(page)
text = file_handle.getvalue()
converter.close()
file_handle.close()
return text
raw = pdftotext('Sample-Certificate.pdf')
print(raw)
确实，您没有得到键值对，而且 pdfminer 根本无法提供给您。它只会从 PDF 中提取文本（+ 可能还有一些附加信息）。
要获得良好的逻辑标签值对，您需要先使用 Information Extraction approach and / or Named Entity Recognition on top of the extracted text. There are plenty of options here. You may want to have a look at SpaCy or NLTK。
通常，从文档中提取有意义的数据及其关系有一个新的性感名称 Document Intelligence。
在处理 PDF 文件时，我更喜欢使用 PyMuPDF 库 https://pypi.org/project/PyMuPDF/
import fitz
txt = []
doc = fitz.open("Sample-Cert_rev-7-1.pdf") # some existing PDF
page = doc[0]
text = page.getText("text")
txt = list(text)
print(text)
text = text.split('\n')
txt = list(text)
print(txt)
ix = text.index('MPC Control #:')
print(ix)
print(text[ix+18])
注意如何正确安装库
这是输出：
"C:\Program Files\Python38\python.exe" C:/Python/Whosebug extract_pdf_text1.py
MICRO PRECISION CALIBRATION, INC.
22835 INDUSTRIAL PLACE
GRASS VALLEY CA 95949
530-268-1860
Cert No.
551220083746791
Date: Aug 3, 2020
Certificate of Calibration
AC-1969.00
N/A
July 01, 2021
N/A
Customer:
MPC Control #:
Asset ID:
Gage Type:
Manufacturer:
Model Number:
Size:
Temp/RH:
Serial Number:
Department:
Performed By:
Received Condition:
Returned Condition:
Cal. Date:
Cal. Interval:
Cal. Due Date:
Work Order #:
DIGITAL MULTIMETER
DANNY BOY B. BUTIAL
0258964
0258964
NONE
AGILENT
34401A
10MHZ
SAMPLE
N/A
IN TOLERANCE
IN TOLERANCE
July 01, 2020
N/A
12 MONTHS
Calibration Notes:
SAMPLE COMPANY
23.0°C / 40.0%
Location:
Calibration performed at MPC facility
Standards Used to Calibrate Equipment
I.D.
Description.
Model
Serial
Manufacturer
Cal. Due Date
Traceability #
PH1405
MULTI-PRODUCT CALIBRATOR
5520A
7575006
FLUKE
Sep 10, 2020
551220083204793
AL4394
DIGITAL MULTIMETER
3458A
2823A09832
AGILENT
Aug 1, 2020
551220083719099
Procedures Used in this Event
Procedure Name
Description
MPC Automated Procedure
MPCCAL Rev. 00
STATEMENTS OF PASS OR FAIL CONFORMANCE: The uncertainty of measurement has been taken into account when determining compliance with specification. All measurements and test results guard banded to ensure the
probability of false-accept does not exceed 2% in compliance with ANSI/NCSL Z540.3-2006 and in case without guard banded the probability of false-accept depending on test uncertainty ratio.
THE CALIBRATION REPORT STATUS:
PASS- Term used when compliance statement is given, and the measurement result is PASS.
PASSz- Term used when compliance statement is given, and the measurement result is conditional passed or PASSz.
FAIL- Term used when compliance statement is given, and the measurement result is FAIL.
FAILz- Term used when compliance statement is given, and the measurement result is conditional failed or FAILz.
REPORT OF VALUE - Term used when reported measurement is not requiring compliance statement in report.
ADJUSTED- When adjustments are made to an instrument which changes the value of measurement from what was measured as found to new value as left.
LIMITED - When an instrument fails calibration but is still functional in a limited manner.
The expanded uncertainty of measurement is stated as the standard uncertainty of measurement multiplied by the coverage factor k=2, which for a normal distribution corresponds to a coverage probability of approximately 95%, unless otherwise stated. This
calibration report complies with ISO/IEC 17025:2017 and ANSI/NCSL Z540.3. Calibration cycles and resulting due dates were submitted/approved by the customer. Any number of factors may cause an instrument to drift out of tolerance before the next
scheduled calibration. Recalibration cycles should be based on frequency of use, environmental conditions and customer's established systematic accuracy. All standards are traceable to SI through the National Institute of Standards and Technology (NIST)
and/or recognized national or international standards laboratories. Services rendered include proper manufacturer’s service instruction and are warranted for no less than thirty (30) days. The information on this report pertains only to the instrument identified,
this may not be reproduced in part or in a whole without the prior written approval of the issuing MP Calibration Laboratory.
Rick Hernandez
Calibrating Technician:
QC Approval:
DANNY BOY B. BUTIAL
(CERT, Rev 7)
Page 1 of 1
['MICRO PRECISION CALIBRATION, INC.', '22835 INDUSTRIAL PLACE', 'GRASS VALLEY CA 95949', '530-268-1860', 'Cert No.', '551220083746791', 'Date: Aug 3, 2020', 'Certificate of Calibration', 'AC-1969.00', 'N/A', 'July 01, 2021', 'N/A', 'Customer:', 'MPC Control #:', 'Asset ID:', 'Gage Type:', 'Manufacturer:', 'Model Number:', 'Size:', 'Temp/RH:', 'Serial Number:', 'Department:', 'Performed By:', 'Received Condition:', 'Returned Condition:', 'Cal. Date:', 'Cal. Interval:', 'Cal. Due Date:', 'Work Order #:', 'DIGITAL MULTIMETER', 'DANNY BOY B. BUTIAL', '0258964', '0258964', 'NONE', 'AGILENT', '34401A', '10MHZ', 'SAMPLE', 'N/A', 'IN TOLERANCE', 'IN TOLERANCE', ' July 01, 2020', 'N/A', '12 MONTHS', 'Calibration Notes:', 'SAMPLE COMPANY', '23.0°C / 40.0%', 'Location:', 'Calibration performed at MPC facility', 'Standards Used to Calibrate Equipment', 'I.D.', 'Description.', 'Model', 'Serial', 'Manufacturer', 'Cal. Due Date', 'Traceability #', 'PH1405', 'MULTI-PRODUCT CALIBRATOR', '5520A', '7575006', 'FLUKE', 'Sep 10, 2020', '551220083204793', 'AL4394', 'DIGITAL MULTIMETER', '3458A', '2823A09832', 'AGILENT', 'Aug 1, 2020', '551220083719099', 'Procedures Used in this Event', 'Procedure Name', 'Description', 'MPC Automated Procedure', 'MPCCAL Rev. 00', 'STATEMENTS OF PASS OR FAIL CONFORMANCE: The uncertainty of measurement has been taken into account when determining compliance with specification. All measurements and test results guard banded to ensure the', 'probability of false-accept does not exceed 2% in compliance with ANSI/NCSL Z540.3-2006 and in case without guard banded the probability of false-accept depending on test uncertainty ratio.', 'THE CALIBRATION REPORT STATUS:', 'PASS- Term used when compliance statement is given, and the measurement result is PASS.', 'PASSz- Term used when compliance statement is given, and the measurement result is conditional passed or PASSz.', 'FAIL- Term used when compliance statement is given, and the measurement result is FAIL.', 'FAILz- Term used when compliance statement is given, and the measurement result is conditional failed or FAILz.', 'REPORT OF VALUE - Term used when reported measurement is not requiring compliance statement in report.', 'ADJUSTED- When adjustments are made to an instrument which changes the value of measurement from what was measured as found to new value as left.', 'LIMITED - When an instrument fails calibration but is still functional in a limited manner.', 'The expanded uncertainty of measurement is stated as the standard uncertainty of measurement multiplied by the coverage factor k=2, which for a normal distribution corresponds to a coverage probability of approximately 95%, unless otherwise stated. This', 'calibration report complies with ISO/IEC 17025:2017 and ANSI/NCSL Z540.3. Calibration cycles and resulting due dates were submitted/approved by the customer. Any number of factors may cause an instrument to drift out of tolerance before the next', "scheduled calibration. Recalibration cycles should be based on frequency of use, environmental conditions and customer's established systematic accuracy. All standards are traceable to SI through the National Institute of Standards and Technology (NIST)", 'and/or recognized national or international standards laboratories. Services rendered include proper manufacturer’s service instruction and are warranted for no less than thirty (30) days. The information on this report pertains only to the instrument identified,', 'this may not be reproduced in part or in a whole without the prior written approval of the issuing MP Calibration Laboratory.', 'Rick Hernandez', 'Calibrating Technician:', 'QC Approval:', 'DANNY BOY B. BUTIAL', '(CERT, Rev 7)', 'Page 1 of 1', '']
13
0258964
Process finished with exit code 0