KSQL select * from table returns 无结果且提示无响应
KSQL select * from table returns no results and prompt is not responsive
我正在使用 kafka-connect 将 MySQL table 中的行流式传输到 kafka 主题中。这很好用。
然后我创建一个 table 具有:
CREATE TABLE mytable (id INT, email VARCHAR, gender VARCHAR, first_name VARCHAR, last_name VARCHAR) WITH (KAFKA_TOPIC='mysql-my-table', VALUE_FORMAT='AVRO', KEY='id');
这也有效,我可以通过以下方式确认:
LIST TABLES;
DESCRIBE EXTENDED mytable;
我明白了mytable
。
问题是我执行的时候
SELECT * FROM mytable;
然后我没有得到任何结果,提示也没有反应,我必须按 ctrl+c
才能取回控制权。
可能是什么问题?
经过一段时间尝试不同的事情并阅读文档后,我发现了问题。
我主题中消息的字段 id
是 ? INT
并且根据 KSQL TABLES 文档,它们需要属于 VARCHAR
类型
所以我按照描述的步骤解决了这个问题here,现在一切正常。
所以步骤如下:
-- Create a stream on the original topic
CREATE STREAM users_with_wrong_key_format (userid INT, username VARCHAR, email VARCHAR)
WITH (KAFKA_TOPIC='users', VALUE_FORMAT='JSON');
-- Derive a new stream with the required key changes.
-- 1) The CAST statement converts the key to the required format.
-- 2) The PARTITION BY clause re-partitions the stream based on the new, converted key.
CREATE STREAM users_with_proper_key
WITH(KAFKA_TOPIC='users-with-proper-key') AS
SELECT CAST(userid as VARCHAR) as userid_string, username, email
FROM users_with_wrong_key_format
PARTITION BY userid_string;
-- Now you can create the table on the properly keyed stream.
CREATE TABLE users_table (userid_string VARCHAR, username VARCHAR, email VARCHAR)
WITH (KAFKA_TOPIC='users-with-proper-key',
VALUE_FORMAT='JSON',
KEY='userid_string');
-- Now you can create the table on the properly keyed stream.
CREATE TABLE users_table (userid_string VARCHAR, username VARCHAR, email VARCHAR)
WITH (KAFKA_TOPIC='users-with-proper-key',
VALUE_FORMAT='JSON',
KEY='userid_string');
我正在使用 kafka-connect 将 MySQL table 中的行流式传输到 kafka 主题中。这很好用。
然后我创建一个 table 具有:
CREATE TABLE mytable (id INT, email VARCHAR, gender VARCHAR, first_name VARCHAR, last_name VARCHAR) WITH (KAFKA_TOPIC='mysql-my-table', VALUE_FORMAT='AVRO', KEY='id');
这也有效,我可以通过以下方式确认:
LIST TABLES;
DESCRIBE EXTENDED mytable;
我明白了mytable
。
问题是我执行的时候
SELECT * FROM mytable;
然后我没有得到任何结果,提示也没有反应,我必须按 ctrl+c
才能取回控制权。
可能是什么问题?
经过一段时间尝试不同的事情并阅读文档后,我发现了问题。
我主题中消息的字段 id
是 ? INT
并且根据 KSQL TABLES 文档,它们需要属于 VARCHAR
所以我按照描述的步骤解决了这个问题here,现在一切正常。
所以步骤如下:
-- Create a stream on the original topic
CREATE STREAM users_with_wrong_key_format (userid INT, username VARCHAR, email VARCHAR)
WITH (KAFKA_TOPIC='users', VALUE_FORMAT='JSON');
-- Derive a new stream with the required key changes.
-- 1) The CAST statement converts the key to the required format.
-- 2) The PARTITION BY clause re-partitions the stream based on the new, converted key.
CREATE STREAM users_with_proper_key
WITH(KAFKA_TOPIC='users-with-proper-key') AS
SELECT CAST(userid as VARCHAR) as userid_string, username, email
FROM users_with_wrong_key_format
PARTITION BY userid_string;
-- Now you can create the table on the properly keyed stream.
CREATE TABLE users_table (userid_string VARCHAR, username VARCHAR, email VARCHAR)
WITH (KAFKA_TOPIC='users-with-proper-key',
VALUE_FORMAT='JSON',
KEY='userid_string');
-- Now you can create the table on the properly keyed stream.
CREATE TABLE users_table (userid_string VARCHAR, username VARCHAR, email VARCHAR)
WITH (KAFKA_TOPIC='users-with-proper-key',
VALUE_FORMAT='JSON',
KEY='userid_string');