MongoDB 日志机制:私有视图的目的是什么?
MongoDB journaling mechanics: what is the purpose of private view?
我正在学习 MongoDB 在线课程。我刚刚观看了描述 MongoDB 如何处理内存中的数据以及如何将数据保存到磁盘的视频 "Journaling's impact on resident memory"。我还阅读了有关 MongoDB 日记的这些文章:
我似乎无法理解私有视图概念背后的理由。
据我了解,当 MMAPv1 用作存储引擎并启用日记功能时,MongoDB 中的数据会发生以下情况:
- 当MongoDB启动时,其现有数据文件使用内存映射映射到虚拟内存中(参见mmap)。虚拟内存的这个区域称为共享视图。
- 共享视图随后被重新映射到虚拟内存的另一部分,称为私有视图。逻辑上(不是物理上),看起来所有数据在两个视图之间都是重复的。
- 在写入操作中,更改应用于私有视图。
- 更改也会(增量地)保存到日志文件中。
- 确认更改存储在日志中后,它们也会应用于共享视图。
在那之后的某个时候,共享视图将刷新到磁盘上的数据文件(这称为同步)。
在某些时候,共享视图也会重新映射到私人视图。
问题
为什么需要私有视图?看起来其中的数据没有用于共享视图不能用于的任何事情。为什么更改在写入日志之前不直接应用于共享视图?这将使私有视图变得不必要,并且 MongoDB 在虚拟内存中的占用空间将减少一半。
这里引用 William Cross 在 MongoDB 大学论坛中提供的答案。
Imagine if there were only a shared view, and no private view. You
perform a write, and you want to first capture these writes in a
write-ahead log. You need the write-ahead log first because otherwise,
you would end up with your volatile memory evaporating on system
failure, with part of a write committed to disk and no way to roll it
back. The problem with a merged public/private view is that if you put
data in the public view, it could start getting flushed to disk at an
arbitrarily short interval, and this could happen before the journal
is done, if you're unlucky.
If this happens, and the server goes down, then all you have is
corrupt data; there's no way to roll it back. Keep in mind, with
MMAPv1, most of the storage engine's work is pushed off onto the OS.
MMAPv1 was created to avoid the sort of fine-grained control over the
data going to disk that we would need here in order for the flow to be
what we want. So, in order to prevent this from happening, the public
view of the data stays the same until the write-ahead log entry is
completed on disk, and only then does the public view get updated. We
have the private view of the data so that it can exist in a totally
separate place, and get flushed to disk before we touch the public
view and risk having it partially committed to disk. In the mean time,
though, we've received a write, so we also have a private view to
answer requests for reads.
In terms of exactly how it's written, I know that we write bytes, and
we start at the first change, so it's an offset + bytes. This keeps
things relatively minimal.
简而言之,当您触发写入操作时,mongod post 首先将写入操作写入私有视图。私有视图是一个内存映射文件,在 200 毫秒后它将数据复制到 journal,它是您 data/db.
的子目录
在日志提交后,它将数据复制到共享视图,该视图在 60 秒后刷新到 data/db。
原因是如果数据库crash或者dirty shutdown,在journal文件中仍然会有写操作。一旦 db 再次出现,它将重播从日志到共享视图的所有操作,并将其刷新回 main data/db.
私有视图和共享视图在 RAM 中分配 space。 (内存映射文件)。了解更多关于内存映射文件的信息。例如,如果您的文件在磁盘上有 2000 字节,那么它会将其映射到内存地址 10-10010,因此您可以直接从内存地址读取文件。内存地址中文件的任何更改都将刷新回磁盘中的文件。
我正在学习 MongoDB 在线课程。我刚刚观看了描述 MongoDB 如何处理内存中的数据以及如何将数据保存到磁盘的视频 "Journaling's impact on resident memory"。我还阅读了有关 MongoDB 日记的这些文章:
我似乎无法理解私有视图概念背后的理由。
据我了解,当 MMAPv1 用作存储引擎并启用日记功能时,MongoDB 中的数据会发生以下情况:
- 当MongoDB启动时,其现有数据文件使用内存映射映射到虚拟内存中(参见mmap)。虚拟内存的这个区域称为共享视图。
- 共享视图随后被重新映射到虚拟内存的另一部分,称为私有视图。逻辑上(不是物理上),看起来所有数据在两个视图之间都是重复的。
- 在写入操作中,更改应用于私有视图。
- 更改也会(增量地)保存到日志文件中。
- 确认更改存储在日志中后,它们也会应用于共享视图。
在那之后的某个时候,共享视图将刷新到磁盘上的数据文件(这称为同步)。
在某些时候,共享视图也会重新映射到私人视图。
问题
为什么需要私有视图?看起来其中的数据没有用于共享视图不能用于的任何事情。为什么更改在写入日志之前不直接应用于共享视图?这将使私有视图变得不必要,并且 MongoDB 在虚拟内存中的占用空间将减少一半。
这里引用 William Cross 在 MongoDB 大学论坛中提供的答案。
Imagine if there were only a shared view, and no private view. You perform a write, and you want to first capture these writes in a write-ahead log. You need the write-ahead log first because otherwise, you would end up with your volatile memory evaporating on system failure, with part of a write committed to disk and no way to roll it back. The problem with a merged public/private view is that if you put data in the public view, it could start getting flushed to disk at an arbitrarily short interval, and this could happen before the journal is done, if you're unlucky.
If this happens, and the server goes down, then all you have is corrupt data; there's no way to roll it back. Keep in mind, with MMAPv1, most of the storage engine's work is pushed off onto the OS. MMAPv1 was created to avoid the sort of fine-grained control over the data going to disk that we would need here in order for the flow to be what we want. So, in order to prevent this from happening, the public view of the data stays the same until the write-ahead log entry is completed on disk, and only then does the public view get updated. We have the private view of the data so that it can exist in a totally separate place, and get flushed to disk before we touch the public view and risk having it partially committed to disk. In the mean time, though, we've received a write, so we also have a private view to answer requests for reads.
In terms of exactly how it's written, I know that we write bytes, and we start at the first change, so it's an offset + bytes. This keeps things relatively minimal.
简而言之,当您触发写入操作时,mongod post 首先将写入操作写入私有视图。私有视图是一个内存映射文件,在 200 毫秒后它将数据复制到 journal,它是您 data/db.
的子目录在日志提交后,它将数据复制到共享视图,该视图在 60 秒后刷新到 data/db。
原因是如果数据库crash或者dirty shutdown,在journal文件中仍然会有写操作。一旦 db 再次出现,它将重播从日志到共享视图的所有操作,并将其刷新回 main data/db.
私有视图和共享视图在 RAM 中分配 space。 (内存映射文件)。了解更多关于内存映射文件的信息。例如,如果您的文件在磁盘上有 2000 字节,那么它会将其映射到内存地址 10-10010,因此您可以直接从内存地址读取文件。内存地址中文件的任何更改都将刷新回磁盘中的文件。