Cygnus 中的 OAuth2
OAuth2 in Cygnus
自从宣布在 FIWARE 实验室中访问 Cosmos 的 WebHDFS 受 OAuth2 保护后，Cygnus（版本 0.8.1）无法在 Cosmos HDFS 中保留 Orion 的上下文数据：
15/07/09 08:45:21 INFO handlers.OrionRestHandler: Starting transaction (1436424291-649-0000000000)
15/07/09 08:45:21 INFO handlers.OrionRestHandler: Received data ({ "subscriptionId" : "51c0ac9ed714fb3b37d7d5a8", "originator" : "localhost", "contextResponses" : [ { "contextElement" : { "attributes" : [ { "name" : "temperature", "type" : "centigrade", "value" : "26.5" } ], "type" : "Room", "isPattern" : "false", "id" : "Room1" }, "statusCode" : { "code" : "200", "reasonPhrase" : "OK" } } ]})
15/07/09 08:45:21 INFO handlers.OrionRestHandler: Event put in the channel (id=109760451, ttl=10)
15/07/09 08:45:21 INFO sinks.OrionSink: Event got from the channel (id=109760451, headers={timestamp=1436424321536, content-type=application/json, transactionId=1436424291-649-0000000000, fiware-service=def_serv, fiware-servicepath=def_serv_path, ttl=10, destination=room1_room}, bodyLength=460)
15/07/09 08:45:22 INFO sinks.OrionHDFSSink: [hdfs-sink] Persisting data at OrionHDFSSink. HDFS file (def_serv/def_serv_path/room1_room/room1_room.txt), Data ({"recvTime":"2015-07-09T06:45:21.536Z","temperature":"26.5", "temperature_md":[]})
15/07/09 08:45:22 ERROR sinks.OrionSink: Persistence error (The /user/frb/def_serv/def_serv_path/room1_room directory could not be created in HDFS. HttpFS response: 503 Service unavailable)
15/07/09 08:45:22 INFO sinks.OrionSink: An event was put again in the channel (id=109760451, ttl=9)
15/07/09 08:45:22 INFO sinks.OrionSink: Finishing transaction (1436424291-649-0000000000)
我发现必须在某处配置 OAuth2 令牌，但如何配置？
重要提示：在执行以下任何命令之前，请备份您的 Cygnus 配置，因为它们在稍作更改后仍然有效。
遗憾的是，可用的 Cygnus 版本不支持 OAuth2。到目前为止。您将必须升级到最新的 0.8.2，它允许通过此 OrionHDFSSink 参数配置令牌：
<your_agent_nam>.sinks.<your_hdfs_sink_name>.oauth2_token = <token>
升级到 Cygnus 0.8.2 就像从 FIWARE 存储库安装一样简单：
$ (sudo) yum clean all # just to clean the yum cache
$ (sudo) yum list cygnus # this will show you 0.82 is available
$ (sudo) sudo rpm -e -vv --allmatches --nodeps --noscripts --notriggers cygnus # this is needed if you have installed a version < 0.8.0
$ (sudo) yum install cygnus # this installs 0.8.2
请记住，FIWARE 存储库是通过编辑 /etc/yum.repos.d/fiware.repo 设置的（很可能您已经拥有它）：
[Fiware]
name=FIWARE repository
baseurl=http://repositories.testbed.fi-ware.eu/repo/rpm/x86_64/
gpgcheck=0
enabled=1
在下面的示例中，我们从 0.7.0 版本移动到最新的 0.8.2 版本：
$ sudo yum clean all
Loaded plugins: fastestmirror, security
Cleaning repos: ...
Cleaning up Everything
Cleaning up list of fastest mirrors
$ sudo yum list cygnus
Loaded plugins: fastestmirror, security
Determining fastest mirrors
* base: sunsite.rediris.es
* epel: ftp.cica.es
* extras: sunsite.rediris.es
* updates: sunsite.rediris.es
...
Installed Packages
cygnus.x86_64 0.7.0-g4cddd26 @Fiware
Available Packages
cygnus.x86_64 0.8.2-0.g7353160 Fiware
$ sudo rpm -e -vv --allmatches --nodeps --noscripts --notriggers cygnus
D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key
D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key
D: loading keyring from rpmdb
...
D: closed db index /var/lib/rpm/Name
D: closed db index /var/lib/rpm/Packages
D: closed db environment /var/lib/rpm
$ sudo yum install cygnus
Loaded plugins: fastestmirror, security
Loading mirror speeds from cached hostfile
* base: sunsite.rediris.es
* epel: ftp.cica.es
* extras: sunsite.rediris.es
* updates: sunsite.rediris.es
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package cygnus.x86_64 0:0.7.0-g4cddd26 will be updated
---> Package cygnus.x86_64 0:0.8.2-0.g7353160 will be an update
--> Finished Dependency Resolution
Dependencies Resolved
==============================================================================================================================================================
Package Arch Version Repository Size
==============================================================================================================================================================
Updating:
cygnus x86_64 0.8.2-0.g7353160 Fiware 94 M
Transaction Summary
==============================================================================================================================================================
Upgrade 1 Package(s)
Total download size: 94 M
Is this ok [y/N]:
Downloading Packages:
cygnus-0.8.2-0.g7353160.x86_64.rpm | 94 MB 00:03
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Updating : cygnus-0.8.2-0.g7353160.x86_64 1/2
[INFO] Creating cygnus user
[INFO] Creating log directory
Done
Cleanup : cygnus-0.7.0-g4cddd26.x86_64 2/2
[INFO] Uninstall the cygnus
Stopping Cygnus chatrooms... [ OK ]
[INFO] Deleting links
rm: no se puede borrar «/etc/cygnus/flume.conf»: No existe el fichero o el directorio
[INFO] Removing application log files
[INFO] Deleting the cygnus folder
[INFO] Deleting the cygnus user
Done
Updated:
cygnus.x86_64 0:0.8.2-0.g7353160
Complete!
为了检查您的版本是否正确，您可以做几件事。例如，使用 rpm 命令：
$ rpm -qa | grep cygnus
cygnus-0.8.2-0.g7353160.x86_64
或使用 yum list:
$ (sudo) yum list cygnus
Loaded plugins: fastestmirror, security
Loading mirror speeds from cached hostfile
* base: sunsite.rediris.es
* epel: ftp.cica.es
* extras: sunsite.rediris.es
* updates: sunsite.rediris.es
Installed Packages
cygnus.x86_64 0.8.2-0.g7353160 @Fiware
或者只需检查 oauth2_token 参数是否存在 ;)
$ cat /usr/cygnus/conf/agent.conf.template |grep oauth2_token
cygnusagent.sinks.hdfs-sink.oauth2_token = xxxxxxxx
重要：备份您的 Cygnus 配置，因为它们在稍作更改后仍然有效：
- 如果您使用的是 Cygnus 版本 < 0.8.0，请注意 Java 包已更改。现在，他们不再是
es.tid.fiware.fiwareconnectors.cygnus.etc，而是 com.telefonica.iot.cygus.etc。
- 从 Cygnus 0.8.1 开始，配置文件
matching_table.conf（类似 CSV 的内容）现在被命名为 grouping_rules.conf（类似于 JSON 的内容，其语义与 CSV-喜欢的版本）。
自从宣布在 FIWARE 实验室中访问 Cosmos 的 WebHDFS 受 OAuth2 保护后，Cygnus（版本 0.8.1）无法在 Cosmos HDFS 中保留 Orion 的上下文数据：
15/07/09 08:45:21 INFO handlers.OrionRestHandler: Starting transaction (1436424291-649-0000000000)
15/07/09 08:45:21 INFO handlers.OrionRestHandler: Received data ({ "subscriptionId" : "51c0ac9ed714fb3b37d7d5a8", "originator" : "localhost", "contextResponses" : [ { "contextElement" : { "attributes" : [ { "name" : "temperature", "type" : "centigrade", "value" : "26.5" } ], "type" : "Room", "isPattern" : "false", "id" : "Room1" }, "statusCode" : { "code" : "200", "reasonPhrase" : "OK" } } ]})
15/07/09 08:45:21 INFO handlers.OrionRestHandler: Event put in the channel (id=109760451, ttl=10)
15/07/09 08:45:21 INFO sinks.OrionSink: Event got from the channel (id=109760451, headers={timestamp=1436424321536, content-type=application/json, transactionId=1436424291-649-0000000000, fiware-service=def_serv, fiware-servicepath=def_serv_path, ttl=10, destination=room1_room}, bodyLength=460)
15/07/09 08:45:22 INFO sinks.OrionHDFSSink: [hdfs-sink] Persisting data at OrionHDFSSink. HDFS file (def_serv/def_serv_path/room1_room/room1_room.txt), Data ({"recvTime":"2015-07-09T06:45:21.536Z","temperature":"26.5", "temperature_md":[]})
15/07/09 08:45:22 ERROR sinks.OrionSink: Persistence error (The /user/frb/def_serv/def_serv_path/room1_room directory could not be created in HDFS. HttpFS response: 503 Service unavailable)
15/07/09 08:45:22 INFO sinks.OrionSink: An event was put again in the channel (id=109760451, ttl=9)
15/07/09 08:45:22 INFO sinks.OrionSink: Finishing transaction (1436424291-649-0000000000)
我发现必须在某处配置 OAuth2 令牌，但如何配置？
重要提示：在执行以下任何命令之前，请备份您的 Cygnus 配置，因为它们在稍作更改后仍然有效。
遗憾的是，可用的 Cygnus 版本不支持 OAuth2。到目前为止。您将必须升级到最新的 0.8.2，它允许通过此 OrionHDFSSink 参数配置令牌：
<your_agent_nam>.sinks.<your_hdfs_sink_name>.oauth2_token = <token>
升级到 Cygnus 0.8.2 就像从 FIWARE 存储库安装一样简单：
$ (sudo) yum clean all # just to clean the yum cache
$ (sudo) yum list cygnus # this will show you 0.82 is available
$ (sudo) sudo rpm -e -vv --allmatches --nodeps --noscripts --notriggers cygnus # this is needed if you have installed a version < 0.8.0
$ (sudo) yum install cygnus # this installs 0.8.2
请记住，FIWARE 存储库是通过编辑 /etc/yum.repos.d/fiware.repo 设置的（很可能您已经拥有它）：
[Fiware]
name=FIWARE repository
baseurl=http://repositories.testbed.fi-ware.eu/repo/rpm/x86_64/
gpgcheck=0
enabled=1
在下面的示例中，我们从 0.7.0 版本移动到最新的 0.8.2 版本：
$ sudo yum clean all
Loaded plugins: fastestmirror, security
Cleaning repos: ...
Cleaning up Everything
Cleaning up list of fastest mirrors
$ sudo yum list cygnus
Loaded plugins: fastestmirror, security
Determining fastest mirrors
* base: sunsite.rediris.es
* epel: ftp.cica.es
* extras: sunsite.rediris.es
* updates: sunsite.rediris.es
...
Installed Packages
cygnus.x86_64 0.7.0-g4cddd26 @Fiware
Available Packages
cygnus.x86_64 0.8.2-0.g7353160 Fiware
$ sudo rpm -e -vv --allmatches --nodeps --noscripts --notriggers cygnus
D: loading keyring from pubkeys in /var/lib/rpm/pubkeys/*.key
D: couldn't find any keys in /var/lib/rpm/pubkeys/*.key
D: loading keyring from rpmdb
...
D: closed db index /var/lib/rpm/Name
D: closed db index /var/lib/rpm/Packages
D: closed db environment /var/lib/rpm
$ sudo yum install cygnus
Loaded plugins: fastestmirror, security
Loading mirror speeds from cached hostfile
* base: sunsite.rediris.es
* epel: ftp.cica.es
* extras: sunsite.rediris.es
* updates: sunsite.rediris.es
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package cygnus.x86_64 0:0.7.0-g4cddd26 will be updated
---> Package cygnus.x86_64 0:0.8.2-0.g7353160 will be an update
--> Finished Dependency Resolution
Dependencies Resolved
==============================================================================================================================================================
Package Arch Version Repository Size
==============================================================================================================================================================
Updating:
cygnus x86_64 0.8.2-0.g7353160 Fiware 94 M
Transaction Summary
==============================================================================================================================================================
Upgrade 1 Package(s)
Total download size: 94 M
Is this ok [y/N]:
Downloading Packages:
cygnus-0.8.2-0.g7353160.x86_64.rpm | 94 MB 00:03
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Updating : cygnus-0.8.2-0.g7353160.x86_64 1/2
[INFO] Creating cygnus user
[INFO] Creating log directory
Done
Cleanup : cygnus-0.7.0-g4cddd26.x86_64 2/2
[INFO] Uninstall the cygnus
Stopping Cygnus chatrooms... [ OK ]
[INFO] Deleting links
rm: no se puede borrar «/etc/cygnus/flume.conf»: No existe el fichero o el directorio
[INFO] Removing application log files
[INFO] Deleting the cygnus folder
[INFO] Deleting the cygnus user
Done
Updated:
cygnus.x86_64 0:0.8.2-0.g7353160
Complete!
为了检查您的版本是否正确，您可以做几件事。例如，使用 rpm 命令：
$ rpm -qa | grep cygnus
cygnus-0.8.2-0.g7353160.x86_64
或使用 yum list:
$ (sudo) yum list cygnus
Loaded plugins: fastestmirror, security
Loading mirror speeds from cached hostfile
* base: sunsite.rediris.es
* epel: ftp.cica.es
* extras: sunsite.rediris.es
* updates: sunsite.rediris.es
Installed Packages
cygnus.x86_64 0.8.2-0.g7353160 @Fiware
或者只需检查 oauth2_token 参数是否存在 ;)
$ cat /usr/cygnus/conf/agent.conf.template |grep oauth2_token
cygnusagent.sinks.hdfs-sink.oauth2_token = xxxxxxxx
重要：备份您的 Cygnus 配置，因为它们在稍作更改后仍然有效：
- 如果您使用的是 Cygnus 版本 < 0.8.0，请注意 Java 包已更改。现在，他们不再是
es.tid.fiware.fiwareconnectors.cygnus.etc，而是com.telefonica.iot.cygus.etc。 - 从 Cygnus 0.8.1 开始，配置文件
matching_table.conf（类似 CSV 的内容）现在被命名为grouping_rules.conf（类似于 JSON 的内容，其语义与 CSV-喜欢的版本）。