使用 Kinesis Connector 到 s3 bucket 的 Kinesis

Question

我在使用 kinesis 连接器库从 kinesis 访问 s3 存储桶时收到以下错误。我使用了以下 link

中的示例

https://github.com/awslabs/amazon-kinesis-connectors/tree/master/src/main/samples

INFO: Stream twitterStreamNew is ACTIVE
Oct 16, 2015 3:45:00 PM com.yash.KinesisConnectorExecutor setupInputStream
INFO: Starting stream source.
Exception in thread "main" java.lang.NoClassDefFoundError: com/amazonaws/transform/JsonErrorUnmarshallerV2
    at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.init(AmazonDynamoDBClient.java:387)
    at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.(AmazonDynamoDBClient.java:383)
    at com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient.(AmazonDynamoDBClient.java:357)
    at com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker.(Worker.java:109)
    at com.amazonaws.services.kinesis.clientlibrary.lib.worker.Worker.(Worker.java:94)
    at com.amazonaws.services.kinesis.connectors.KinesisConnectorExecutorBase.initialize(KinesisConnectorExecutorBase.java:84)
    at com.amazonaws.services.kinesis.connectors.KinesisConnectorExecutorBase.initialize(KinesisConnectorExecutorBase.java:37)
    at com.example.KinesisConnectorExecutor.



    (KinesisConnectorExecutor.java:110)
    at com.example.S3Executor.(S3Executor.java:19)
    at com.example.S3Executor.main(S3Executor.java:34)
    Caused by: java.lang.ClassNotFoundException:       com.amazonaws.transform.JsonErrorUnmarshallerV2
    at java.net.URLClassLoader.findClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
    at java.lang.ClassLoader.loadClass(Unknown Source)
    ... 10 more

Answer 1

有同样的错误 - 仅仅是由于库版本冲突。

将您的运动连接器库和 aws java sdk 更新到 POM 中的最新版本。

    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk</artifactId>
        <version>1.10.27</version>
    </dependency>

    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>amazon-kinesis-connectors</artifactId>
        <version>1.2.0</version>
    </dependency>

使用 Kinesis Connector 到 s3 bucket 的 Kinesis

Kinesis to s3 bucket using Kinesis Connector

amazon-s3

amazon-web-services

amazon-kinesis