如何根据另一个列值对数据进行透视

Question

我正在尝试 Pivot/Transpose 我的列值并尝试获取相应的日期时间。

table 我有：

User  Status     LogTime
----------------------------------------
Tom   Active     2019-09-06 17:36:08.233
Tom   Active     2019-09-06 18:37:08.244
Tom   Active     2019-09-06 20:46:08.133
Tom   InActive   2019-09-06 23:46:08.133
Tom   Active     2019-09-07 12:37:08.244
Tom   Active     2019-09-08 10:46:08.133
Tom   InActive   2019-09-08 11:46:08.133

正在尝试获取如下数据。

User  Active                     InActive
------------------------------------------------------
Tom  2019-09-06 20:46:08.133   2019-09-06 23:46:08.133
Tom  2019-09-08 10:46:08.133   2019-09-08 11:46:08.133

我正在尝试将“状态”列与上次活动日志时间和上次活动之后的非活动日志时间进行调换

Answer 1

你可以试试 shift() to grab the previous row of InActive , then segregate every 2 rows as a group and unstack():

m=df[df.Status.eq('InActive')|df.Status.eq('InActive').shift(-1)].reset_index(drop=True)
m.assign(k=m.groupby(m.index//2).ngroup()).set_index(['User','Status','k']).unstack(1)

                        LogTime                         
Status                   Active                 InActive
User k                                                  
Tom  0  2019-09-06 20:46:08.133  2019-09-06 23:46:08.133
     1  2019-09-08 10:46:08.133  2019-09-08 11:46:08.133

或使用与 pivot_table 相同的 m:

m.assign(k=m.groupby(m.index//2).ngroup()).pivot_table(index=['User','k']
          ,columns='Status',values='LogTime',aggfunc='first').rename_axis(None,axis=1)

                      Active                 InActive
User k                                                  
Tom  0  2019-09-06 20:46:08.133  2019-09-06 23:46:08.133
     1  2019-09-08 10:46:08.133  2019-09-08 11:46:08.133

Answer 2

在 'User'、'Status'、'date' 的 'LogTime' 部分尝试 groupby，并在 'LogTime' 上调用 'last'。接下来，'unstack'，将索引放入列并删除不需要的列，然后 'dropna'

df1 = (df.groupby(['User','Status', df.LogTime.dt.date]).LogTime.last()
        .unstack(1).reset_index().drop('LogTime',1).dropna())

Out[890]:
Status User                  Active                InActive
0       Tom 2019-09-06 20:46:08.133 2019-09-06 23:46:08.133
2       Tom 2019-09-08 10:46:08.133 2019-09-08 11:46:08.133

Answer 3

此查询在 Hive 中适用于您的数据集。当用户的日志中没有 InActive 或 Active 状态时，我尝试考虑可能的边界条件，当然应该在真实数据集上验证和调整逻辑。

演示：

with data as (
select stack(7,
'Tom','Active',   '2019-09-06 17:36:08.233',
'Tom','Active',   '2019-09-06 18:37:08.244',
'Tom','Active',   '2019-09-06 20:46:08.133',
'Tom','InActive', '2019-09-06 23:46:08.133',
'Tom','Active',   '2019-09-07 12:37:08.244',
'Tom','Active',   '2019-09-08 10:46:08.133',
'Tom','InActive', '2019-09-08 11:46:08.133'
) as(User,Status,LogTime)
) --use your_table instead of this


select User, Active, InActive
from
(
select User,MaxInActive,MaxActive,--Status,LogTime,nextStatus,
       case when (prevStatus='Active' and Status='InActive')  --the last Active LogTime
                 then prevLogTime
            when (Status='Active' and nextStatus is NULL) --boundary condition, Active is the last status, take current
                 OR (LogTime=MaxActive  and MaxInActive is NULL) --No InActive, take current
                 then LogTime             
       end as Active,

       case when (prevStatus='Active' and Status='InActive') --InActive LogTime after the last Active
                 OR (LogTime=MaxInActive and MaxActive is NULL) --No Active exists, take current
                 then LogTime
       end as InActive

from       
(
select User,Status,LogTime,
       max(case when Status='InActive' then LogTime end) over(partition by User) as MaxInActive ,
       max(case when Status='Active' then LogTime end) over(partition by User) as MaxActive,
       lead(Status) over(partition by User order by LogTime) nextStatus,
       lag(Status) over(partition by User order by LogTime) prevStatus,
       lag(LogTime) over(partition by User order by LogTime) prevLogTime
  from data
)s
)s
where (Active is not NULL and InActive is not NULL)
      or (MaxInActive is NULL and Active is not NULL) --only active records exist
      or (MaxActive is NULL and MaxInActive is not NULL) --only inactive exists
 ;

结果：

OK
user    active  inactive
Tom     2019-09-06 20:46:08.133 2019-09-06 23:46:08.133
Tom     2019-09-08 10:46:08.133 2019-09-08 11:46:08.133
Time taken: 100.645 seconds, Fetched: 2 row(s)

如何根据另一个列值对数据进行透视

How to Pivot the data Based on another column value

hive

pivot

pandas

impala

pyspark