使用 happybase 扫描带有中文字符的 HBase table 时过滤器不起作用
Filter doesn't work when using happybase to scan an HBase table with Chinese character
我在HBase中有一个table,其中汉字存储在某个列中,比如'FLT:CREW_DEPT'。现在我需要过滤掉所有 'FLT:CREW_DEPT' 等于某个值的行。在 hbase shell 中执行此操作时,它工作正常,如下所示
scan 'PAX_EXP_FACT', {COLUMNS => ['FLT:CREW_DEPT'], FILTER => "SingleColumnValueFilter ('FLT', 'CREW_DEPT', =, 'binary:\xe4\xb8\x80\xe9\x83\xa8', true, true)", LIMIT => 5}
ROW COLUMN+CELL
CA101-20160808-PEK-001192753702 column=FLT:CREW_DEPT, timestamp=1500346136328, value=\xE4\xB8\x80\xE9\x83\xA8
CA101-20161103-PEK-001181988752 column=FLT:CREW_DEPT, timestamp=1500346230204, value=\xE4\xB8\x80\xE9\x83\xA8
CA101-20161105-PEK-000728690130 column=FLT:CREW_DEPT, timestamp=1500346244963, value=\xE4\xB8\x80\xE9\x83\xA8
CA101-20161201-PEK-006731936575 column=FLT:CREW_DEPT, timestamp=1500346233640, value=\xE4\xB8\x80\xE9\x83\xA8
CA101-20161212-PEK-001512808262 column=FLT:CREW_DEPT, timestamp=1500346223572, value=\xE4\xB8\x80\xE9\x83\xA8
5 row(s) in 0.0060 seconds
然而,当使用 happybase 在 python 中做类似的事情时,没有任何返回:
import happybase
import datetime
import pytz
connection = happybase.Connection('192.168.199.200', port=9090)
table = connection.table('PAX_EXP_FACT')
filter_str = ""
filter_str += "SingleColumnValueFilter('FLT', 'CREW_DEPT', =, 'binary:\xe4\xba\x8c\xe9\x83\xa8')"
results = table.scan(
filter=filter_str
# ,limit=100
)
count = 0
for key, data in results:
count += 1
print(data[b'FLT:CREW_DEPT'].decode('utf-8'))
print('No. of flight matches:', count)
connection.close()
返回 0 行...
有人可以帮忙吗?非常感谢!!!
原来答案很简单...我应该用过
"SingleColumnValueFilter('FLT', 'CREW_DEPT', =, 'binary:中文')"
而不是先将其转换为 utf-8 编码的字节字符串...即使在 hbase shell我也可以做同样的事情(尽管它以问号显示)
scan 'PAX_EXP_FACT', {COLUMNS => ['FLT:CREW_DEPT'], FILTER => "SingleColumnValueFilter ('FLT', 'CREW_DEPT', =, 'binary:??', true, true)", LIMIT => 5}
ROW COLUMN+CELL
CA101-20160808-PEK-001192753702 column=FLT:CREW_DEPT, timestamp=1500353334419, value=\xE4\xB8\x80\xE9\x83\xA8
CA101-20161103-PEK-001181988752 column=FLT:CREW_DEPT, timestamp=1500353426641, value=\xE4\xB8\x80\xE9\x83\xA8
CA101-20161105-PEK-000728690130 column=FLT:CREW_DEPT, timestamp=1500353447707, value=\xE4\xB8\x80\xE9\x83\xA8
CA101-20161201-PEK-006731936575 column=FLT:CREW_DEPT, timestamp=1500353432222, value=\xE4\xB8\x80\xE9\x83\xA8
CA101-20161212-PEK-001512808262 column=FLT:CREW_DEPT, timestamp=1500353417107, value=\xE4\xB8\x80\xE9\x83\xA8
5 row(s) in 0.0100 seconds
使用子字符串是行不通的,这是我在提出这个愚蠢的问题之前一直在尝试的...
我在HBase中有一个table,其中汉字存储在某个列中,比如'FLT:CREW_DEPT'。现在我需要过滤掉所有 'FLT:CREW_DEPT' 等于某个值的行。在 hbase shell 中执行此操作时,它工作正常,如下所示
scan 'PAX_EXP_FACT', {COLUMNS => ['FLT:CREW_DEPT'], FILTER => "SingleColumnValueFilter ('FLT', 'CREW_DEPT', =, 'binary:\xe4\xb8\x80\xe9\x83\xa8', true, true)", LIMIT => 5}
ROW COLUMN+CELL
CA101-20160808-PEK-001192753702 column=FLT:CREW_DEPT, timestamp=1500346136328, value=\xE4\xB8\x80\xE9\x83\xA8
CA101-20161103-PEK-001181988752 column=FLT:CREW_DEPT, timestamp=1500346230204, value=\xE4\xB8\x80\xE9\x83\xA8
CA101-20161105-PEK-000728690130 column=FLT:CREW_DEPT, timestamp=1500346244963, value=\xE4\xB8\x80\xE9\x83\xA8
CA101-20161201-PEK-006731936575 column=FLT:CREW_DEPT, timestamp=1500346233640, value=\xE4\xB8\x80\xE9\x83\xA8
CA101-20161212-PEK-001512808262 column=FLT:CREW_DEPT, timestamp=1500346223572, value=\xE4\xB8\x80\xE9\x83\xA8
5 row(s) in 0.0060 seconds
然而,当使用 happybase 在 python 中做类似的事情时,没有任何返回:
import happybase
import datetime
import pytz
connection = happybase.Connection('192.168.199.200', port=9090)
table = connection.table('PAX_EXP_FACT')
filter_str = ""
filter_str += "SingleColumnValueFilter('FLT', 'CREW_DEPT', =, 'binary:\xe4\xba\x8c\xe9\x83\xa8')"
results = table.scan(
filter=filter_str
# ,limit=100
)
count = 0
for key, data in results:
count += 1
print(data[b'FLT:CREW_DEPT'].decode('utf-8'))
print('No. of flight matches:', count)
connection.close()
返回 0 行...
有人可以帮忙吗?非常感谢!!!
原来答案很简单...我应该用过
"SingleColumnValueFilter('FLT', 'CREW_DEPT', =, 'binary:中文')"
而不是先将其转换为 utf-8 编码的字节字符串...即使在 hbase shell我也可以做同样的事情(尽管它以问号显示)
scan 'PAX_EXP_FACT', {COLUMNS => ['FLT:CREW_DEPT'], FILTER => "SingleColumnValueFilter ('FLT', 'CREW_DEPT', =, 'binary:??', true, true)", LIMIT => 5}
ROW COLUMN+CELL
CA101-20160808-PEK-001192753702 column=FLT:CREW_DEPT, timestamp=1500353334419, value=\xE4\xB8\x80\xE9\x83\xA8
CA101-20161103-PEK-001181988752 column=FLT:CREW_DEPT, timestamp=1500353426641, value=\xE4\xB8\x80\xE9\x83\xA8
CA101-20161105-PEK-000728690130 column=FLT:CREW_DEPT, timestamp=1500353447707, value=\xE4\xB8\x80\xE9\x83\xA8
CA101-20161201-PEK-006731936575 column=FLT:CREW_DEPT, timestamp=1500353432222, value=\xE4\xB8\x80\xE9\x83\xA8
CA101-20161212-PEK-001512808262 column=FLT:CREW_DEPT, timestamp=1500353417107, value=\xE4\xB8\x80\xE9\x83\xA8
5 row(s) in 0.0100 seconds
使用子字符串是行不通的,这是我在提出这个愚蠢的问题之前一直在尝试的...