由于 IPv6，Avro 工具无法从 json 转换

Question

我正在尝试使用 avro 工具 (1.7.7) 将一个简单的 json 文件转换为 avro。

我的命令运行

java -jar ~/Downloads/avro-tools-1.7.7.jar fromjson 
--schema-file src/main/avro/twitter.avsc tweet.json > tweet.avro

在此架构上

{
"type": "record",
"name": "tweet",
"namespace": "co.feeb.avro",
"fields": [
    {
        "name": "username",
        "type": "string",
        "doc": "screen name of the user on twitter.com"
    },
    {
        "name": "text",
        "type": "string",
        "doc": "the content of the user's message"
    },
    {
        "name": "timestamp",
        "type": "long",
        "doc": "unix epoch time in seconds"
    }
],
"doc": "Schema for twitter messages"
}

我在运行这个命令后看到这个异常：

Exception in thread "main" java.lang.ExceptionInInitializerError
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:189)
at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:159)
at org.apache.hadoop.security.UserGroupInformation.isSecurityEnabled(UserGroupInformation.java:216)
at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:409)
at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:395)
at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:1436)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1337)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:244)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:122)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:228)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187)
at org.apache.avro.tool.Util.openFromFS(Util.java:88)
at org.apache.avro.tool.DataFileWriteTool.run(DataFileWriteTool.java:82)
at org.apache.avro.tool.Main.run(Main.java:84)
at org.apache.avro.tool.Main.main(Main.java:73)

Caused by: java.lang.NumberFormatException: For input string: "810d:340:1770::1"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580)
at java.lang.Integer.parseInt(Integer.java:615)
at com.sun.jndi.dns.DnsClient.<init>(DnsClient.java:127)
at com.sun.jndi.dns.Resolver.<init>(Resolver.java:61)
at com.sun.jndi.dns.DnsContext.getResolver(DnsContext.java:573)
at com.sun.jndi.dns.DnsContext.c_getAttributes(DnsContext.java:434)
at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_getAttributes(ComponentDirContext.java:235)
at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.getAttributes(PartialCompositeDirContext.java:141)
at com.sun.jndi.toolkit.url.GenericURLDirContext.getAttributes(GenericURLDirContext.java:103)
at sun.security.krb5.KrbServiceLocator.getKerberosService(KrbServiceLocator.java:85)
at sun.security.krb5.Config.checkRealm(Config.java:1120)
at sun.security.krb5.Config.getRealmFromDNS(Config.java:1093)
at sun.security.krb5.Config.getDefaultRealm(Config.java:987)
at org.apache.hadoop.security.KerberosName.<clinit>(KerberosName.java:81)

尝试使用 -Djava.net.preferIPv4Stack=true 优先使用 IPv4 而不是 IPv6 没有帮助。（我是运行 Mac OSX 10.10.3 和 Java 1.8.0_25-b17）。

Answer 1

哦，快点...发布后我自己解决了。我的本地路由器向我的本地计算机添加了一个 IPv6 名称服务器。

将分配的名称服务器手动更改为 Google 的 8.8.8.8 解决了该问题。

由于 IPv6，Avro 工具无法从 json 转换

Avro tool fails to convert from json due to IPv6

dns

json

ipv6

avro