Python Mongoengine：如何绕过无法通过验证的文档以避免崩溃

Question

我正在使用 Mongoengine 将 .csv 文件中的数据保存到 MongoDB。

我将文档class定义为：

class Marc(Document):
    BibID = IntField(required=True)
    ISBN = StringField(required=True, max_length=13)
    Author = StringField(max_length=50)
    Title = StringField(required=True, max_length=200)
    Summary = StringField(max_length=2000)
    Genre = StringField(max_length=30)
    TopicalMain = ListField(StringField(max_length=80))
    TopicalGeographic = ListField(StringField(max_length=50))
    meta = {'allow_inheritance': True, 'strict': False}

并一一创建文档记录：

def __createMarcRecord(self, rec):
    return NewDoc(
        BibID=rec['BibID'],
        ISBN=rec['ISBN'],
        Title=rec['Title'],
        Author=rec['Author'] if "Author" in rec else None,
        Summary=rec['Summary'] if "Summary" in rec else None,
        Genre=rec['Genre'] if "Genre" in rec else None,
        TopicalMain=rec['Topical_Main'] if "Topical_Main" in rec else None,
        TopicalGeographic=rec['Topical_Geographic'] if "Topical_Geographic" in rec else None,
        Stored=datetime.datetime.now()
    )

然后我尝试保存文档：

def storeToMongoDB(self):
    count = 0
    with open(self.errorLog, 'w') as ef:
        for i in self.bookList:
            count += 1
            print(count)
            marcRec = self.__createMarcRecord(i)
            try:
                marcRec.save()
            except:
                ef.write("{0}-{1}\n".format(count, sys.exc_info()[0]))
    ef.close()

.csv 文件中存在无法通过验证的条目（没有内容“Title”），因此代码会崩溃：

Traceback (most recent call last):
  File "/.../mongo.py", line 132, in storeToMongoDB
    marcRec = self.__createMarcRecord(i)
  File "/.../mongo.py", line 112, in __createMarcRecord
    Title=rec['Title'],
KeyError: 'Title'

请问有没有办法绕过这些文件来避免死机？

Answer 1

乍一看，您似乎可以使用 try and except KeyError around line 112

try:
    marcRec = self.__createMarcRecord(i)
except KeyError:
    continue  # move onto next book

Python Mongoengine：如何绕过无法通过验证的文档以避免崩溃

Python Mongoengine: How to bypass documents which cannot pass validation to avoid crash

python

mongodb

pymongo

mongoengine