并行添加到数组
Adding to array in parallel
我正在使用 Grand Central Dispatch 将一个数组的元素转换为另一个数组。我在源数组上调用 dispatch_apply
,将其转换为零个或多个项目,然后将它们添加到目标数组。这是一个简化的例子:
let src = Array(0..<1000)
var dst = [UInt32]()
let queue = dispatch_queue_create("myqueue", DISPATCH_QUEUE_CONCURRENT)
dispatch_apply(src.count, queue) { i in
dst.append(arc4random_uniform(UInt32(i))) // <-- potential error here
}
print(dst)
我 有时 在 append
行上遇到错误。错误始终是以下之一:
1. malloc: *** error for object 0x107508f00: pointer being freed was not allocated
2. fatal error: UnsafeMutablePointer.destroy with negative count
3. fatal error: Can't form Range with end < start
我猜这是因为 append
不是线程安全的。我做错了什么以及如何解决?
如您所见,Swift 可能会重新定位数组以提高内存效率。当您 运行 多个 append
时,它必然会发生。我的猜测是,与您的转换相比,附加到数组是一种成本非常低的操作。您可以创建一个串行队列来扩展 dst
数组:
let src = Array(0..<1000)
var dst = [UInt32]()
let queue = dispatch_queue_create("myqueue", DISPATCH_QUEUE_CONCURRENT)
let serialQueue = dispatch_queue_create("mySerialQueue", DISPATCH_QUEUE_SERIAL)
let serialGroup = dispatch_group_create()
dispatch_apply(src.count, queue) { i in
let result = arc4random_uniform(UInt32(i)) // Your long-running transformation here
dispatch_group_async(serialGroup, serialQueue) {
dst.append(result)
}
}
// Wait until all append operations are complete
dispatch_group_wait(serialGroup, DISPATCH_TIME_FOREVER)
如果有人对 Swift 4(我想是 Swift 3 之后)解决方案感兴趣(方法略有不同):
let source = Array(0..<1000)
// If inside class/struct, make this lazy or execute the block inside init
var destination: [UInt32] = {
var copy = Array<UInt32>.init(repeating: 0, count: source.count)
DispatchQueue.concurrentPerform(iterations: source.count) { i in
// Do stuff to get the i'th element
copy[i] = UInt32.random(in: 0..<i)
}
return copy
}()
如果您选择避免数组大小与并发操作数之间的联系:
(例如,您可以选择 0..<500
与 500..<1000
并行进行两次迭代)
let iterations = 2
DispatchQueue.concurrentPerform(iterations: iterations) { index in
// First iteration would start at 0 whereas second (index = 1) starts at 500
let start = index * source.count / iterations
// First and second iterations will have 500 and 1000 upper bounds respectively
let end = (index + 1) * source.count / iterations
for i in start..<end {
// Do stuff to get the i'th element
copy[i] = UInt32.random(in: 0..<source.count)
}
}
我正在使用 Grand Central Dispatch 将一个数组的元素转换为另一个数组。我在源数组上调用 dispatch_apply
,将其转换为零个或多个项目,然后将它们添加到目标数组。这是一个简化的例子:
let src = Array(0..<1000)
var dst = [UInt32]()
let queue = dispatch_queue_create("myqueue", DISPATCH_QUEUE_CONCURRENT)
dispatch_apply(src.count, queue) { i in
dst.append(arc4random_uniform(UInt32(i))) // <-- potential error here
}
print(dst)
我 有时 在 append
行上遇到错误。错误始终是以下之一:
1. malloc: *** error for object 0x107508f00: pointer being freed was not allocated
2. fatal error: UnsafeMutablePointer.destroy with negative count
3. fatal error: Can't form Range with end < start
我猜这是因为 append
不是线程安全的。我做错了什么以及如何解决?
如您所见,Swift 可能会重新定位数组以提高内存效率。当您 运行 多个 append
时,它必然会发生。我的猜测是,与您的转换相比,附加到数组是一种成本非常低的操作。您可以创建一个串行队列来扩展 dst
数组:
let src = Array(0..<1000)
var dst = [UInt32]()
let queue = dispatch_queue_create("myqueue", DISPATCH_QUEUE_CONCURRENT)
let serialQueue = dispatch_queue_create("mySerialQueue", DISPATCH_QUEUE_SERIAL)
let serialGroup = dispatch_group_create()
dispatch_apply(src.count, queue) { i in
let result = arc4random_uniform(UInt32(i)) // Your long-running transformation here
dispatch_group_async(serialGroup, serialQueue) {
dst.append(result)
}
}
// Wait until all append operations are complete
dispatch_group_wait(serialGroup, DISPATCH_TIME_FOREVER)
如果有人对 Swift 4(我想是 Swift 3 之后)解决方案感兴趣(方法略有不同):
let source = Array(0..<1000)
// If inside class/struct, make this lazy or execute the block inside init
var destination: [UInt32] = {
var copy = Array<UInt32>.init(repeating: 0, count: source.count)
DispatchQueue.concurrentPerform(iterations: source.count) { i in
// Do stuff to get the i'th element
copy[i] = UInt32.random(in: 0..<i)
}
return copy
}()
如果您选择避免数组大小与并发操作数之间的联系:
(例如,您可以选择 0..<500
与 500..<1000
并行进行两次迭代)
let iterations = 2
DispatchQueue.concurrentPerform(iterations: iterations) { index in
// First iteration would start at 0 whereas second (index = 1) starts at 500
let start = index * source.count / iterations
// First and second iterations will have 500 and 1000 upper bounds respectively
let end = (index + 1) * source.count / iterations
for i in start..<end {
// Do stuff to get the i'th element
copy[i] = UInt32.random(in: 0..<source.count)
}
}