如何编写 SQL 查询以从由变量字段名称标识的嵌套 json 对象中提取值

Question

问题： 当所需的名称是动态的时，如何将 sqlite 语句写入 select 来自嵌套 json 对象的值/ 多变的。同样重要的是，这可以通过单个 sql 语句完成。最终，这将从 bash 脚本中执行。

在下面的对象示例中，我需要列出 sql 数据库中的所有 dot11.advertisedssid.ssid。一个可接受的解决方案是列出 json 对象中存在的 dot11.advertisedssid.ssid 的所有值，但我想了解如何查询动态 json 名称（这样我就可以得到另一个嵌套的值）。一般来说，我在我的 sql 语句中使用 json_extract 我只是不知道如何获得 ssid 值（在这个例子中）！

我如何知道 733545801 是字段名称，然后如何在 json_extract 语句中使用它？并对所有此类嵌套对象执行此操作。

示例：

通常这就是我查询其他 json 值的方式。

select json_extract(devices.device,'$."dot11.device"."dot11.device.typeset"') from devices;

来自数据库的对象样本：

 "dot11.device": {
    "dot11.device.typeset": 257,
    "dot11.device.client_map": {

    },
    "dot11.device.num_client_aps": 0,
    "dot11.device.advertised_ssid_map": {
      "733545801": {
        "dot11.advertisedssid.ssid": "SampleFES-WiFi",
        "dot11.advertisedssid.ssidlen": 15,
        "dot11.advertisedssid.beacon": 1,
        "dot11.advertisedssid.probe_response": 1,
        "dot11.advertisedssid.channel": "6",
        "dot11.advertisedssid.ht_mode": "HT20",
        "dot11.advertisedssid.ht_center_1": 0,
        "dot11.advertisedssid.ht_center_2": 0,
        "dot11.advertisedssid.first_time": 1559567379,
        "dot11.advertisedssid.last_time": 1559567379,
        "dot11.advertisedssid.beacon_info": "",
        "dot11.advertisedssid.cloaked": 0,
        "dot11.advertisedssid.crypt_set": 268436162,
        "dot11.advertisedssid.maxrate": 65.000000,
        "dot11.advertisedssid.beaconrate": 10,
        "dot11.advertisedssid.beacons_sec": 2,
        "dot11.advertisedssid.ietag_checksum": 1220416683,
        "dot11.advertisedssid.wpa_mfp_required": 0,
        "dot11.advertisedssid.wpa_mfp_supported": 0,
        "dot11.advertisedssid.dot11d_country": "",
        "dot11.advertisedssid.dot11d_list": [
        ],
        "dot11.advertisedssid.wps_state": 0,
        "dot11.advertisedssid.dot11r_mobility": 0,
        "dot11.advertisedssid.dot11r_mobility_domain_id": 0,
        "dot11.advertisedssid.dot11e_qbss": 0,
        "dot11.advertisedssid.dot11e_qbss_stations": 0,
        "dot11.advertisedssid.dot11e_channel_utilization_perc": 0.000000,
        "dot11.advertisedssid.ccx_txpower": 0,
        "dot11.advertisedssid.cisco_client_mfp": 0,
        "dot11.advertisedssid.ie_tag_list": [
          0.000000,
          1.000000,
          3.000000,
          5.000000,
          42.000000,
          50.000000,
          48.000000,
          45.000000,
          61.000000,
          127.000000,
          221.000000
        ]
      }
    }

感谢您的帮助！

PS。这来自新的 kismet 数据库和重新设计的架构。

这是整个对象：

   {
  "kismet.device.base.manuf": "Texas Instruments",
  "kismet.device.base.key": "4202770D00000000_AFB4F569D2380000",
  "kismet.device.base.macaddr": "38:D2:69:F5:B4:AF",
  "kismet.device.base.phyname": "IEEE802.11",
  "kismet.device.base.phyid": 0,
  "kismet.device.base.name": "LincolnFES-WiFi",
  "kismet.device.base.commonname": "LincolnFES-WiFi",
  "kismet.device.base.type": "Wi-Fi AP",
  "kismet.device.base.basic_type_set": 1,
  "kismet.device.base.crypt": "WPA2-PSK",
  "kismet.device.base.basic_crypt_set": 2,
  "kismet.device.base.first_time": 1559567379,
  "kismet.device.base.last_time": 1559567379,
  "kismet.device.base.mod_time": 1559567380,
  "kismet.device.base.packets.total": 3,
  "kismet.device.base.packets.rx": 0,
  "kismet.device.base.packets.tx": 0,
  "kismet.device.base.packets.llc": 3,
  "kismet.device.base.packets.error": 0,
  "kismet.device.base.packets.data": 0,
  "kismet.device.base.packets.crypt": 0,
  "kismet.device.base.packets.filtered": 0,
  "kismet.device.base.datasize": 0,
  "kismet.device.base.packets.rrd": {
    "kismet.common.rrd.last_time": 1559567383,
    "kismet.common.rrd.minute_vec": [
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      1,
      2,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0
    ],
    "kismet.common.rrd.blank_val": 0,
    "kismet.common.rrd.aggregator": "default",
    "kismet.common.rrd.hour_vec": [
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0
    ],
    "kismet.common.rrd.day_vec": [
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0
    ]
  },
  "kismet.device.base.signal": {
    "kismet.common.signal.type": "dbm",
    "kismet.common.signal.last_signal": -56,
    "kismet.common.signal.last_noise": 0,
    "kismet.common.signal.min_signal": -74,
    "kismet.common.signal.min_noise": 0,
    "kismet.common.signal.max_signal": -56,
    "kismet.common.signal.max_noise": 0,
    "kismet.common.signal.maxseenrate": 10,
    "kismet.common.signal.encodingset": 1,
    "kismet.common.signal.carrierset": 1,
    "kismet.common.signal.signal_rrd": {
      "kismet.common.rrd.last_time": 1559567383,
      "kismet.common.rrd.minute_vec": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
      ],
      "kismet.common.rrd.blank_val": 0,
      "kismet.common.rrd.aggregator": "peak_signal"
    }
  },
  "kismet.device.base.freq_khz_map": {
    "2437000.000000": 1,
    "2442000.000000": 1,
    "5500000.000000": 1
  },
  "kismet.device.base.channel": "6",
  "kismet.device.base.frequency": 2442000,
  "kismet.device.base.num_alerts": 0,
  "kismet.device.base.tags": {

  },
  "kismet.device.base.seenby": {
    "-1970862229": {
      "kismet.common.seenby.uuid": "5FE308BD-0000-0000-0000-00C0CAA60413",
      "kismet.common.seenby.first_time": 1559567379,
      "kismet.common.seenby.last_time": 1559567379,
      "kismet.common.seenby.num_packets": 3,
      "kismet.common.seenby.freq_khz_map": {
        "2437000.000000": 1,
        "2442000.000000": 1,
        "5500000.000000": 1
      },
      "kismet.common.seenby.signal": {
        "kismet.common.signal.type": "dbm",
        "kismet.common.signal.last_signal": -56,
        "kismet.common.signal.last_noise": 0,
        "kismet.common.signal.min_signal": -74,
        "kismet.common.signal.min_noise": 0,
        "kismet.common.signal.max_signal": -56,
        "kismet.common.signal.max_noise": 0,
        "kismet.common.signal.maxseenrate": 10,
        "kismet.common.signal.encodingset": 1,
        "kismet.common.signal.carrierset": 1,
        "kismet.common.signal.signal_rrd": {
          "kismet.common.rrd.last_time": 1559567383,
          "kismet.common.rrd.minute_vec": [
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0,
            0
          ],
          "kismet.common.rrd.blank_val": 0,
          "kismet.common.rrd.aggregator": "peak_signal"
        }
      }
    }
  },
  "kismet.device.base.server_uuid": "A8F71A2C-85F8-11E9-BA41-4B49534D4554",
  "dot11.device": {
    "dot11.device.typeset": 257,
    "dot11.device.client_map": {

    },
    "dot11.device.num_client_aps": 0,
    "dot11.device.advertised_ssid_map": {
      "733545801": {
        "dot11.advertisedssid.ssid": "LincolnFES-WiFi",
        "dot11.advertisedssid.ssidlen": 15,
        "dot11.advertisedssid.beacon": 1,
        "dot11.advertisedssid.probe_response": 1,
        "dot11.advertisedssid.channel": "6",
        "dot11.advertisedssid.ht_mode": "HT20",
        "dot11.advertisedssid.ht_center_1": 0,
        "dot11.advertisedssid.ht_center_2": 0,
        "dot11.advertisedssid.first_time": 1559567379,
        "dot11.advertisedssid.last_time": 1559567379,
        "dot11.advertisedssid.beacon_info": "",
        "dot11.advertisedssid.cloaked": 0,
        "dot11.advertisedssid.crypt_set": 268436162,
        "dot11.advertisedssid.maxrate": 65,
        "dot11.advertisedssid.beaconrate": 10,
        "dot11.advertisedssid.beacons_sec": 2,
        "dot11.advertisedssid.ietag_checksum": 1220416683,
        "dot11.advertisedssid.wpa_mfp_required": 0,
        "dot11.advertisedssid.wpa_mfp_supported": 0,
        "dot11.advertisedssid.dot11d_country": "",
        "dot11.advertisedssid.dot11d_list": [

        ],
        "dot11.advertisedssid.wps_state": 0,
        "dot11.advertisedssid.dot11r_mobility": 0,
        "dot11.advertisedssid.dot11r_mobility_domain_id": 0,
        "dot11.advertisedssid.dot11e_qbss": 0,
        "dot11.advertisedssid.dot11e_qbss_stations": 0,
        "dot11.advertisedssid.dot11e_channel_utilization_perc": 0,
        "dot11.advertisedssid.ccx_txpower": 0,
        "dot11.advertisedssid.cisco_client_mfp": 0,
        "dot11.advertisedssid.ie_tag_list": [
          0,
          1,
          3,
          5,
          42,
          50,
          48,
          45,
          61,
          127,
          221
        ]
      }
    },
    "dot11.device.num_advertised_ssids": 1,
    "dot11.device.probed_ssid_map": {

    },
    "dot11.device.num_probed_ssids": 0,
    "dot11.device.associated_client_map": {

    },
    "dot11.device.num_associated_clients": 0,
    "dot11.device.client_disconnects": 0,
    "dot11.device.last_sequence": 0,
    "dot11.device.bss_timestamp": 0,
    "dot11.device.num_fragments": 0,
    "dot11.device.num_retries": 0,
    "dot11.device.datasize": 0,
    "dot11.device.datasize_retry": 0,
    "dot11.device.last_probed_ssid_csum": 0,
    "dot11.device.last_beaconed_ssid": "LincolnFES-WiFi",
    "dot11.device.last_beaconed_ssid_checksum": 733545801,
    "dot11.device.last_bssid": "38:D2:69:F5:B4:AF",
    "dot11.device.last_beacon_timestamp": 1559567379,
    "dot11.device.wps_m3_count": 0,
    "dot11.device.wps_m3_last": 0,
    "dot11.device.wpa_handshake_list": [

    ],
    "dot11.device.wpa_nonce_list": [

    ],
    "dot11.device.wpa_anonce_list": [

    ],
    "dot11.device.wpa_present_handshake": 0,
    "dot11.device.min_tx_power": 0,
    "dot11.device.max_tx_power": 0,
    "dot11.device.supported_channels": [

    ],
    "dot11.device.link_measurement_capable": 0,
    "dot11.device.neighbor_report_capable": 0,
    "dot11.device.extended_capabilities": [

    ],
    "dot11.device.beacon_fingerprint": 4212996422,
    "dot11.device.probe_fingerprint": 0,
    "dot11.device.response_fingerprint": 0
  }
}

Answer 1

当你想递归遍历整个对象的字段及其内容时，你需要json_tree():

SELECT j.value
FROM devices AS d
JOIN json_tree(d.device) AS j
WHERE j.key = 'dot11.advertisedssid.ssid';

给予

value         
--------------
SampleFES-WiFi

当运行在 table 上持有该示例对象的固定版本时。

Answer 2

我知道这有点老了，但 OP 似乎（在评论中）想要一个更完整的解决方案。我知道当我第一次遇到这个答案时我就这样做了。接受的解决方案允许您从 JSON blob 中提取 one 字段，但 OP 示例中的常见用例是提取来自该 blob 的多个字段。经过一番搜索后，我发现一旦您意识到 "dot11.device.advertised_ssid_map" 对象是一个数组，json_extract() 函数就可以很好地解决这个问题。一旦你为它提供了一个索引，他的正常查询方法就可以工作了。

考虑因素：

OP 的示例与设备 table 中的 Kismet 设备字段相关，因此我的示例将使用我在 table
使用 Kismet，这些 JSON blob 中使用的键很长并且包含点，因此在 SQLite3 中指定它们的语法对于某些嵌套值来说有点麻烦
SQLite3 的 JSON1 扩展似乎不喜欢 JSON路径规范中通常允许的某些通配符语法，因此需要很长的显式路径

所以这是我的解决方案：

SELECT devmac, strongest_signal, 
  <b>json_extract(d.device, '$."dot11.device"."dot11.device.advertised_ssid_map"[0]."dot11.advertisedssid.ssid"') AS ssid,</b>
  json_extract(d.device, '$."dot11.device"."dot11.device.advertised_ssid_map"[0]."dot11.advertisedssid.cloaked"') AS cloaked,
  json_extract(d.device, '$."kismet.device.base.signal"."kismet.common.signal.min_signal"') AS weakest_signal,
  json_extract(d.device, '$."kismet.device.base.channel"') AS channel,
  json_extract(d.device, '$."dot11.device"."dot11.device.num_associated_clients"') AS clientCnt,
  json_extract(d.device, '$."kismet.device.base.crypt"') AS crypt,
  json_extract(d.device, '$."kismet.device.base.manuf"') AS manuf
FROM devices AS d
WHERE type = 'Wi-Fi AP'
;

如何编写 SQL 查询以从由变量字段名称标识的嵌套 json 对象中提取值

How to write a SQL query to pull a value from a nested json object identified by a variable field name

sql

sqlite

sqlite-json1

kismet-wireless

json-extract