根据条件识别蜂巢中的第二个最大值

Question

我有一个 table，其中的行看起来像这样，其中一列对所有行进行排名，按票证 ID 分区，时间戳记描述。

所有行只能有一个标志等于一。

ticketID  |  flag 1  | flag 2 | flag 3 | flag 4 | Timestamp  |  Rank    |  stringvalue  |  
----------------------------------------------------------------------------------------|
   1      |    0     |    0   |    1   |    0   |  xxxxxx    |    2     |   aaaaaa      |
   1      |    0     |    0   |    0   |    1   |  xxxxxx    |    1     |   bbbbbb      |
   1      |    0     |    1   |    0   |    0   |  xxxxxx    |    3     |   aaaaaa      |
   2      |    1     |    0   |    0   |    0   |  xxxxxx    |    2     |   bbbbbb      |
   2      |    0     |    0   |    0   |    1   |  xxxxxx    |    1     |   xxxxxx      |
   3      |    0     |    0   |    1   |    0   |  xxxxxx    |    4     |   aaaaaa      |
   3      |    0     |    1   |    0   |    0   |  xxxxxx    |    3     |   bbbbbb      |
   3      |    1     |    0   |    0   |    0   |  xxxxxx    |    1     |   ssssss      |
   3      |    0     |    0   |    0   |    1   |  xxxxxx    |    2     |   nnnnnn      |
   4      |    0     |    1   |    0   |    0   |  xxxxxx    |    2     |   gggggg      |
   4      |    0     |    0   |    0   |    1   |  xxxxxx    |    1     |   iiiiii      |

对于每个 ticketID，我需要根据排名获取第一行，但特定标志除外：

当票的排名 1 是标志 4 = 1 的行时，我需要将第二位作为第一位。如果票的第二位是标志 3 = 1，那么我需要将第一位 (flag = 4) 的字符串值与第二位 (flag = 3) 连接起来。

如果第二个等级是 flag = 1 或 flag = 2 那么就忘记第一个等级和 return 第二个作为第一个。

我希望我的问题很清楚。

谢谢

编辑

示例输出：

----------------------------------------------------------------------------------------
ticketID  |  flag 1  | flag 2 | flag 3 | Timestamp  |  Rank    |  stringvalue          |  
---------------------------------------------------------------------------------------|
   1      |    0     |    0   |    1   |  xxxxxx    |    1     |   aaaaaa / bbbbbbb    |
   2      |    1     |    0   |    0   |  xxxxxx    |    1     |        bbbbbb         |
   3      |    1     |    0   |    0   |  xxxxxx    |    1     |        ssssss         |
   4      |    0     |    1   |    0   |  xxxxxx    |    1     |        gggggg         |
----------------------------------------------------------------------------------------

Answer 1

我将使用一些 sub-queries 和 struct group by。这将允许我们在不使用 window 的情况下询问有关多行的问题。可能会执行得更快，因为我们不必维护 window 状态。

create table theRanks (ticketID int, flag_1 int, flag_2 int, flag_3 int, flag_4 int, Timestamp string, Rank int, stringvalue string)
-- create some dummy data
insert into theRanks values ( 1 , 0, 0, 1, 0, 'xxxxxx', 2, 'aaaaaa')
insert into theRanks values ( 1 , 0, 0, 0, 1, 'xxxxxx', 1, 'bbbbbb')
insert into theRanks values ( 1 , 0, 1, 0, 0, 'xxxxxx', 3, 'aaaaaa')

with stuct_table as -- sub-query syntax
( 
  select 
   ticketID, 
   struct( -- struct will allow us to group rows together.
    Rank as rawRank, -- this has to be first in strut as we use it for sorting
    flag_1 , 
    flag_2, 
    flag_3, 
    flag_4 , 
    Timestamp , 
    stringvalue 
   ) as myRow 
 from 
  theRanks 
 where 
  rank in (1,2) -- only look at first two ranks
), 
constants as -- subquery
( 
 select 0 as rank1, 1 as rank2 -- strictly not needed just to help make it more readable 
), 
grouped_rows as --subquery
(
 select 
  ticketID, 
  array_sort(collect_list(myRow)) as row_list  -- will sort on rank all structs into a list
 from stuct_table 
 group by ticketID
) , 
raw_rows as (select --sub-query styntax
 ticketId, 
 case 
  when 
   row_list[constants.rank2].flag_1 + row_list[constants.rank2].flag_2 > 0 or (row_list[constants.rank1].flag_4 = 1 and row_list[constants.rank2].flag_3  = 0 )
 then
   row_list[constants.rank2]
 when 
   row_list[constants.rank1].flag_4 = 1 and row_list[constants.rank2].flag_3  = 1 -- condition to concat string
 then
   struct( -- this struct must match the original one we created
    row_list[constants.rank2].rawRank as rawRank, 
    row_list[constants.rank2].flag_1 as flag_1,
    row_list[constants.rank2].flag_2 as flag_2,
    row_list[constants.rank2].flag_3 as flag_3,
    row_list[constants.rank2].flag_4 as flag_4,
    row_list[constants.rank2].Timestamp as Timestamp,
    concat(
      row_list[constants.rank1].stringvalue, 
      ' / ', 
      row_list[constants.rank2].stringvalue) as stringvalue
    )
 else
   row_list[constants.rank1]
 end as rankedRow,
 1 as Rank
from grouped_rows
cross join constants) -- not strictly needed, just replace all constants.rank1 with 0 and constants.rank2 with 1.  I just use it to make it more clear what I'm doing.  Could be replaced in production.
select rankedRow.* , 1 as Rank from raw_rows; -- makes struct columns into table columns

根据条件识别蜂巢中的第二个最大值

Identify the second max value in hive based on condition

sql

hadoop

hive

hiveql