如何根据与上一行的差异对行进行分组?
How to group rows based on difference with previous row?
我有以下数据框:
| start_time | end_time | id |
|---------------------|---------------------|-----|
| 2017-03-30 01:00:00 | 2017-03-30 01:15:30 |1 |
| 2017-03-30 02:02:00 | 2017-03-30 03:30:00 |4 |
| 2017-03-30 03:37:00 | 2017-03-30 03:39:00 |7 |
| 2017-03-30 03:41:30 | 2017-03-30 04:50:00 |8 |
| 2017-03-30 07:10:00 | 2017-03-30 07:10:30 |10 |
| 2017-03-30 07:11:00 | 2017-03-30 07:20:00 |13 |
| 2017-03-30 07:22:00 | 2017-03-30 08:00:00 |15 |
| 2017-03-30 10:00:00 | 2017-03-30 10:03:00 |20 |
当行 "i-1" 的 time_finish 在行 "i" 的 time_start 之前至多 900 秒时,我想将行分组在同一 ID 下。
基本上,上面示例的输出将是:
结果将是:
| start_time | end_time | id |
|---------------------|---------------------|-----|
| 2017-03-30 01:00:00 | 2017-03-30 01:15:30 |1 |
| 2017-03-30 02:02:00 | 2017-03-30 03:30:00 |4 |
| 2017-03-30 03:37:00 | 2017-03-30 03:39:00 |4 |
| 2017-03-30 03:41:30 | 2017-03-30 04:50:00 |4 |
| 2017-03-30 07:10:00 | 2017-03-30 07:10:30 |10 |
| 2017-03-30 07:11:00 | 2017-03-30 07:20:00 |10 |
| 2017-03-30 07:22:00 | 2017-03-30 08:00:00 |10 |
| 2017-03-30 10:00:00 | 2017-03-30 10:03:00 |20 |
我通过以下代码实现了它,但我确信有一种更优雅(和高效)的方法可以做到这一点:
df['endTime_delayed'] = df.end_time.shift(1)
df['id_delayed'] = df['id'].shift(1)
for (i,row) in df.iterrows():
if (row.start_time-row.endTime_delayed).seconds <= 900 :
df.id.iloc[i] = df.id_delayed.iloc[i]
try :
df.id_delayed.iloc[i+1] = df.id.iloc[i]
except :
break
mask
和 ffill
diff = df.start_time.sub(df.end_time.shift())
mask = diff < pd.Timedelta(900, unit='s')
df.id.mask(mask).ffill().astype(df.id.dtype)
0 1
1 4
2 4
3 4
4 10
5 10
6 10
7 20
Name: id, dtype: int64
我有以下数据框:
| start_time | end_time | id |
|---------------------|---------------------|-----|
| 2017-03-30 01:00:00 | 2017-03-30 01:15:30 |1 |
| 2017-03-30 02:02:00 | 2017-03-30 03:30:00 |4 |
| 2017-03-30 03:37:00 | 2017-03-30 03:39:00 |7 |
| 2017-03-30 03:41:30 | 2017-03-30 04:50:00 |8 |
| 2017-03-30 07:10:00 | 2017-03-30 07:10:30 |10 |
| 2017-03-30 07:11:00 | 2017-03-30 07:20:00 |13 |
| 2017-03-30 07:22:00 | 2017-03-30 08:00:00 |15 |
| 2017-03-30 10:00:00 | 2017-03-30 10:03:00 |20 |
当行 "i-1" 的 time_finish 在行 "i" 的 time_start 之前至多 900 秒时,我想将行分组在同一 ID 下。
基本上,上面示例的输出将是:
结果将是:
| start_time | end_time | id |
|---------------------|---------------------|-----|
| 2017-03-30 01:00:00 | 2017-03-30 01:15:30 |1 |
| 2017-03-30 02:02:00 | 2017-03-30 03:30:00 |4 |
| 2017-03-30 03:37:00 | 2017-03-30 03:39:00 |4 |
| 2017-03-30 03:41:30 | 2017-03-30 04:50:00 |4 |
| 2017-03-30 07:10:00 | 2017-03-30 07:10:30 |10 |
| 2017-03-30 07:11:00 | 2017-03-30 07:20:00 |10 |
| 2017-03-30 07:22:00 | 2017-03-30 08:00:00 |10 |
| 2017-03-30 10:00:00 | 2017-03-30 10:03:00 |20 |
我通过以下代码实现了它,但我确信有一种更优雅(和高效)的方法可以做到这一点:
df['endTime_delayed'] = df.end_time.shift(1)
df['id_delayed'] = df['id'].shift(1)
for (i,row) in df.iterrows():
if (row.start_time-row.endTime_delayed).seconds <= 900 :
df.id.iloc[i] = df.id_delayed.iloc[i]
try :
df.id_delayed.iloc[i+1] = df.id.iloc[i]
except :
break
mask
和 ffill
diff = df.start_time.sub(df.end_time.shift())
mask = diff < pd.Timedelta(900, unit='s')
df.id.mask(mask).ffill().astype(df.id.dtype)
0 1
1 4
2 4
3 4
4 10
5 10
6 10
7 20
Name: id, dtype: int64