人口模拟中的确定性动作碰撞处理

Deterministic action collision handling in population simlulations

我正在为我自己的模拟环境研究理论框架。我想模拟种群的进化算法,但是我不知道如何处理多个个体之间的冲突行为。

模拟具有离散的时间步长,并且在具有不同个体的随机群体的棋盘(或拼贴网格)上进行。每个人都有一些输入、内部状态和每一步可能采取的行动列表。例如,一个人可以读取其在网格上的位置(输入)并朝特定方向移动一个方块(动作)。现在,假设我有两个人 A 和 B。他们都在同一个模拟步骤中执行某个动作,这将导致两个人最终出现在同一个板块上。然而,这是环境规则所禁止的。

更抽象地说:我的模拟处于有效状态 S1。由于多个个体采取的独立行动,下一个状态S2(在一个模拟步骤之后)将是无效状态。

如何解决这些冲突/冲突,使我永远不会处于无效状态?

我希望模拟可以重播,因此行为应该是确定性的。

另一个问题是公平。可以说我通过先到先得的人来解决冲突。因为理论上所有的动作都是同时发生的(离散时间步长),所以“先到者”不是时间的度量,而是数据布局。较早处理的个体现在具有优势,因为它们恰好位于内部数据结构中的有利位置(即数组中较低的索引)。

有没有办法保证公平?如果不是,我怎样才能减少不公平?


我知道这些是非常宽泛的问题,但由于我还没有计算出模拟的所有约束和规则,所以我想大致了解一下这些系统中哪些是可能的,或者是常见的做法。对于进一步研究的任何指示,我都很高兴。

由于 "独立 多人采取的行动" 我想没有办法避免潜在的碰撞,因此你需要一些解决这些问题的机制。

您的“先到先得” 方法的公平版本可能涉及在每个时间步开始时随机洗牌,例如在每个时间步为您个人选择一个新的随机处理顺序。 如果您修复随机种子,模拟结果仍然是确定性的。

如果个人在模拟过程中获得某种类型的分数/适应度,这也可以用来解决冲突。例如。冲突总是由最适合的人获胜(那时你需要一个额外的平局规则)。
或者选择一个获胜概率与适应度成正比的随机获胜者:如果个体 1 和 2 的适应度为 f1 和 f2,则 1 获胜的概率为 f1/(f1+f2),而 2 获胜的概率为 f2/(f1 +f2).关系 (f1 = f2) 也将自动解决。
我想那些基于适应度的规则可以被称为公平,只要

  1. 每个人都有相同的起始适应度(或者起始适应度也是随机设置的)
  2. 每个人都有相同的机会获得高适应性,例如所有起始位置相同或起始位置随机设置

这个问题非常广泛,但这是我对以下情况的回答:

  1. 智能体在具有循环边界条件的方形(网格)板上移动
  2. 可能的移动是:a) 留在原地,b) 移动到 9 个相邻位置之一 可能的移动被分配了一个概率
  3. 两个代理针对的每个单元都会发生冲突,因为我们假设没有两个代理可以占用同一个单元(排除)。我们将通过重新掷骰子来解决冲突,直到没有冲突为止。

总体思路:

  1. 为每个代理 i 掷骰子 --> 代理 i 的目标单元格

  2. 是否有两个代理针对同一个单元格?

    如果是:

       for every pair of conflicting agents:
    
           re-roll the dice for BOTH agents
    

    备注:

    a) 当代理丢失时检测到冲突,因为它已被另一个代理“粉碎”(由于代理列表中的相对位置)。

    b) 在这里,我假设对两者重新掷骰子是一种“公平”处理,因为不必做出任意决定。

  3. 解决问题了吗?如果不是,返回 2)

  4. 将代理移动到新位置并返回到 1)

我提供了一个python程序。没有花哨的图形。在终端中运行。 默认参数为:

Board_size = 4

Nb_of_agents=8(50%职业)

如果你想看看它是如何随着问题的大小而变化的,那么输入 VERBOSE=False 否则你会被输出淹没。注意:-1表示一个空单元格。

EXAMPLES OF OUTPUT:
Note: I used a pretty high occupancies (50% and 25%) for 
examples 1 and 2. Much lower occupancies will result in 
no conflict most of the time.

##################################################################
EXAMPLE 1: 
VERBOSE=True
Board = 4 x 4
Nb. of agents: 8 (occupation 50%)
==============================================================
Turn:  0
Old board:
[[-1.  7.  3.  0.]
 [ 6. -1.  4.  2.]
 [-1. -1.  5. -1.]
 [-1.  1. -1. -1.]]
Proposed new board:
[[ 1. -1. -1. -1.]
 [-1.  4. -1. -1.]
 [-1.  6. -1.  2.]
 [-1.  7.  5. -1.]]
# of conflicts to solve:  2
Conflicts to solve: [agent_a, agent_b, targeted cell]: 
[0, 1, array([0, 0])]
[3, 5, array([2, 3])]
Proposed new board:
[[-1. -1.  1.  3.]
 [-1.  4. -1.  5.]
 [-1.  6. -1.  2.]
 [-1.  7. -1.  0.]]
No conflicts
<<< OUTPUT >>>
Old board:
[[-1.  7.  3.  0.]
 [ 6. -1.  4.  2.]
 [-1. -1.  5. -1.]
 [-1.  1. -1. -1.]]

Definitive new board:
[[-1. -1.  1.  3.]
 [-1.  4. -1.  5.]
 [-1.  6. -1.  2.]
 [-1.  7. -1.  0.]]
==============================================================
Turn:  1
Old board:
[[-1. -1.  1.  3.]
 [-1.  4. -1.  5.]
 [-1.  6. -1.  2.]
 [-1.  7. -1.  0.]]
Proposed new board:
[[ 3. -1. -1. -1.]
 [ 5. -1.  4. -1.]
 [ 7. -1. -1. -1.]
 [ 6.  1. -1.  2.]]
# of conflicts to solve:  1
Conflicts to solve: [agent_a, agent_b, targeted cell]: 
[0, 6, array([0, 3])]
Proposed new board:
[[ 3. -1. -1. -1.]
 [ 5. -1.  4. -1.]
 [ 7. -1. -1. -1.]
 [-1.  6. -1.  2.]]
# of conflicts to solve:  2
Conflicts to solve: [agent_a, agent_b, targeted cell]: 
[0, 7, array([0, 2])]
[1, 6, array([1, 3])]
Proposed new board:
[[ 3.  1. -1. -1.]
 [ 5. -1.  4. -1.]
 [ 0.  7. -1. -1.]
 [ 6. -1. -1.  2.]]
No conflicts
<<< OUTPUT >>>
Old board:
[[-1. -1.  1.  3.]
 [-1.  4. -1.  5.]
 [-1.  6. -1.  2.]
 [-1.  7. -1.  0.]]

Definitive new board:
[[ 3.  1. -1. -1.]
 [ 5. -1.  4. -1.]
 [ 0.  7. -1. -1.]
 [ 6. -1. -1.  2.]]
==============================================================

##################################################################
EXAMPLE 2:
VERBOSE=False
Board = 200 x 200
Nb. of agents: 10000 (occupation 25%)
==============================================================
Turn:  0
# of conflicts to solve:  994
# of conflicts to solve:  347
# of conflicts to solve:  137
# of conflicts to solve:  63
# of conflicts to solve:  24
# of conflicts to solve:  10
# of conflicts to solve:  6
# of conflicts to solve:  4
# of conflicts to solve:  2
No conflicts
==============================================================
Turn:  1
# of conflicts to solve:  1002
# of conflicts to solve:  379
# of conflicts to solve:  150
# of conflicts to solve:  62
# of conflicts to solve:  27
# of conflicts to solve:  9
# of conflicts to solve:  2
No conflicts
==============================================================

程序(在python):

#!/usr/bin/env python
# coding: utf-8

import numpy
import numpy as np

np.random.seed(1) # will reproduce the examples
# Verbose: if True: show the boards (ok for small boards)
Verbose=True 
# max nb of turns
MaxTurns=2

Board_size= 4
Nb_of_cells=Board_size**2
Nb_of_agents=8 # should be < Board_size**2

agent_health=np.ones(Nb_of_agents)     # Example 1:  All agents move (see function choose_move)
#agent_health=np.random.rand(Nb_of_agents) # With this: the probability of moving is given by health
#agent_health=0.8*np.ones(Nb_of_agents) # With this: 80% of the time they move, 20% the stay in place

possible_moves=np.array([[0,0], [-1,-1],[-1, 0],[-1,+1], [ 0,-1], [ 0,+1], [+1,-1],[+1, 0],[+1,+1]])
Nb_of_possible_moves=len(possible_moves)

def choose_move(agent, health):
    # Each agent chooses randomly a move among the possible moves 
    # with a mobility proportional to health. 
    
    prob[0]=1-agent_health[agent] # low health --> low mobility
    prob[1:9]=(1-prob[0])/8
    move=np.random.choice(Nb_of_possible_moves,1,p=prob)
    return move

def identify_conflicts_to_solve(missing_agents, Nb_of_agents, Old_X, Old_Y):
    # 1) Identify conflicts to solve:
    target_A=[]
    target_B=[]
    [target_A.append([a,(Old_X[a]+possible_moves[move[a]][0])%Board_size, (Old_Y[a]+possible_moves[move[a]][1])%Board_size]) for a in missing_agents];
    [target_B.append([a,(Old_X[a]+possible_moves[move[a]][0])%Board_size, (Old_Y[a]+possible_moves[move[a]][1])%Board_size]) for a in range(Nb_of_agents) if not a in missing_agents];
    target_A=np.array(target_A)
    target_B=np.array(target_B)
    conflicts_to_solve=[]
    for m in range(len(target_A)):
        for opponent in range(len(target_B[:,0])): 
            if all(target_A[m,1:3] == target_B[opponent,1:3]): # they target the same cell
                conflicts_to_solve.append([target_A[m,0], target_B[opponent,0], target_A[m,1:3]])

    return conflicts_to_solve

# Fill the board with -1 (-1 meaning: empty cell)
Old_Board=-np.ones(len(np.arange(0,Board_size**2))) 

# Choose a cell on the board for each agent:
# position = index of the occupied cell
Old_indices = np.random.choice(Nb_of_cells, size=Nb_of_agents, replace=False) 
# We populate the board
for i in range(Nb_of_agents):
    Old_Board[Old_indices[i]]=i
New_Board=Old_Board 


# Coordinates: We assume a cyclic board
Old_X=np.array([Old_indices[i] % Board_size for i in range(len(Old_indices))]) # X position of cell i 
Old_Y=np.array([Old_indices[i] // Board_size for i in range(len(Old_indices))])# Y position of cell i

# Define other properties

move=np.zeros(Nb_of_agents,dtype=int)
prob=np.zeros(Nb_of_possible_moves)

print('==============================================================')
for turn in range(MaxTurns):
    print("Turn: ",turn)
    if Verbose:
        print('Old board:')
        print(New_Board.reshape(Board_size,Board_size))
    Nb_of_occupied_cells_before_the_move=len(Old_Board[Old_Board>-1])
    Legal_move=False
    while not Legal_move:

        for i in range(0,Nb_of_agents):
            move[i]=choose_move(agent=i, health=agent_health[i])
                   
        conflicts_to_solve=-1
        while conflicts_to_solve!=[]:
            # New coordinates (with cyclic boundary conditions):
            New_X=np.array([(Old_X[i]+possible_moves[move[i]][0]) % Board_size for i in range(Nb_of_agents)])
            New_Y=np.array([(Old_Y[i]+possible_moves[move[i]][1]) % Board_size for i in range(Nb_of_agents)])

            # New board
            New_indices=New_Y*Board_size+New_X
            New_Board=-np.ones(Board_size**2) # fill the board with -1 (-1 meaning: empty cell)       
            for i in range(Nb_of_agents): # Populate new board
                New_Board[New_indices[i]]=i        

            # Look for missing agents: an agent is missing if it has been "overwritten" by another agent,
            # indicating conflicts in reaching a particular cell
            missing_agents=[agent for agent in range(Nb_of_agents) if not agent in New_Board]            
            # 1) identify conflicts to solve:
            conflicts_to_solve = identify_conflicts_to_solve(missing_agents, Nb_of_agents, Old_X, Old_Y)
            if Verbose:
                print('Proposed new board:')
                print(New_Board.reshape(Board_size,Board_size))
            if len(conflicts_to_solve)>0:
                print("# of conflicts to solve: ", len(conflicts_to_solve))
                if Verbose:
                    print('Conflicts to solve: [agent_a, agent_b, targeted cell]: ')
                    for c in conflicts_to_solve:
                        print(c)
            else:
                print("No conflicts")
                
            # 2) Solve conflicts
            # The way we solve conflicting agents is "fair" since we re-roll the dice for all of them
            # Without making arbitrary decisions
            for c in conflicts_to_solve:
                # re-choose a move for "a"
                move[c[0]]=choose_move(c[0], agent_health[c[0]])
                # re-choose a move for "b"
                move[c[1]]=choose_move(c[1], agent_health[c[1]])
                 
            

            
        Nb_of_occupied_cells_after_the_move=len(New_Board[New_Board>-1])

        Legal_move = Nb_of_occupied_cells_before_the_move == Nb_of_occupied_cells_after_the_move
        if not Legal_move:
            # Note: in principle, it should never happen but,
            # better to check than being sorry...
            print("Problem: Illegal move")
            Turn=MaxTurns
            # We stop there
    if Verbose:
        print("<<< OUTPUT >>>")
        print("Old board:")
        print(Old_Board.reshape(Board_size,Board_size))
        print()
        print("Definitive new board:")
        print(New_Board.reshape(Board_size,Board_size))
    print('==============================================================')

    Old_X=New_X
    Old_Y=New_Y
    Old_indices=New_indices
    Old_Board=New_Board