PYODBC 截断 SQL 服务器 FOR JSON 查询的响应

Question

我在尝试让 SQL 服务器 JSON 结果集与 PYODBC 良好配合时遇到问题。我的 Python/Flask 技能不是最好的，所以我不完全确定这是我在做的愚蠢的事情，还是驱动程序不支持 JSON。

失败的进程正在调用一个存储过程，该存储过程returns 使用FOR JSON PATH 的SELECT 语句的结果。该过程本身很好，结果集在 SSMS 中看起来是正确的。

但是，在下面的代码块中，为 search_results 赋值的行抛出错误。

with DB() as cnxn:
    results = cnxn.query('dbo.getLocationByTrain', params)
    search_results = json.loads(results.fetchone()[0]);

错误是：

Traceback (most recent call last):
  File "C:\Users\Mark\source\repos\CommuteTo\CommuteTo\env\lib\site-packages\flask\app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "C:\Users\Mark\source\repos\CommuteTo\CommuteTo\env\lib\site-packages\flask\app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "C:\Users\Mark\source\repos\CommuteTo\CommuteTo\env\lib\site-packages\flask\app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "C:\Users\Mark\source\repos\CommuteTo\CommuteTo\env\lib\site-packages\flask\_compat.py", line 33, in reraise
    raise value
  File "C:\Users\Mark\source\repos\CommuteTo\CommuteTo\env\lib\site-packages\flask\app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "C:\Users\Mark\source\repos\CommuteTo\CommuteTo\env\lib\site-packages\flask\app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "C:\Users\Mark\source\repos\CommuteTo\CommuteTo\CommuteTo\views.py", line 81, in search
    search_results = json.loads(results.fetchone()[0]);
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\json\__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\json\decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\json\decoder.py", line 355, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 2028 (char 2027)

如果我删除 json.loads 而不是 jsonify 结果，错误显然消失了，但我可以看到返回给 Flask 的结果集是不完整的：

with DB() as cnxn:
    results = cnxn.query('dbo.getLocationByTrain', params)
    search_results = results.fetchone()[0];
return jsonify(search_results)

Flask 中的结果：

"[{\"Name\":\"ABBEY WOOD\",\"Latitude\":\"51.490772\",\"Longitude\":\"0.120326\",\"LatestDepartureTime\":\"08:18:00\",\"MinJourneyTime\":14,\"MaxJourneyTime\":17,\"schools\":[{\"Name\":\"Abbey Wood Nursery School\",\"Address1\":\"Dahlia Road\",\"PostCode\":\"SE2 0SX\",\"OverallEffectiveNess\":2},{\"Name\":\"Bannockburn Primary School\",\"Address1\":\"Plumstead High Street\",\"PostCode\":\"SE18 1HE\",\"OverallEffectiveNess\":1},{\"Name\":\"Gallions Mount Primary School\",\"Address1\":\"Purrett Road\",\"PostCode\":\"SE18 1JR\",\"OverallEffectiveNess\":2},{\"Name\":\"Boxgrove Primary School\",\"Address1\":\"Boxgrove Road\",\"PostCode\":\"SE2 9JP\",\"OverallEffectiveNess\":2},{\"Name\":\"De Lucy Primary School\",\"Address1\":\"Cookhill Road\",\"PostCode\":\"SE2 9PD\",\"OverallEffectiveNess\":2},{\"Name\":\"Heronsgate Primary School\",\"Address1\":\"Whinchat Road\",\"PostCode\":\"SE28 0EA\",\"OverallEffectiveNess\":1},{\"Name\":\"Linton Mead Primary School\",\"Address1\":\"Central Way\",\"PostCode\":\"SE28 8DT\",\"OverallEffectiveNess\":2},{\"Name\":\"Greenslade Primary School\",\"Address1\":\"Erindale\",\"PostCode\":\"SE18 2QQ\",\"OverallEffectiveNess\":2},{\"Name\":\"St Patrick's Catholic Primary School\",\"Address1\":\"Griffin Road\",\"PostCode\":\"SE18 7QG\",\"OverallEffectiveNess\":2},{\"Name\":\"St Thomas A Becket Roman Catholic Primary School\",\"Address1\":\"Mottisfont Road\",\"PostCode\":\"SE2 9LY\",\"OverallEffectiveNess\":2},{\"Name\":\"St Margaret Clitherow Catholic Primary School\",\"Address1\":\"Cole Close\",\"PostCode\":\"SE28 8GB\",\"OverallEffectiveNess\":2},{\"Name\":\"Bishop John Robinson Church of England Primary School\",\"Address1\":\"Hoveton Road\",\"PostCode\":\"SE28 8LW\",\"OverallEffectiveNess\":2},{\"Name\":\"Plumstead Manor School\",\"Address1\":\"Old Mill Road\",\"PostCode\":\"SE18 1QF\",\"OverallEffectiveNess\":3},{\"Name\":\"Hawksmoor School\",\"Address1\":\"Bentham Road\",\"PostCode\":\"SE28 8AS\",\"OverallEffectiveNess\":1},{\"Name\":\"Belmont Primary School\",\"Address1\":\"Belmont Road\",\"PostCode\":\"DA8 1LE\",\"OverallEffectiveNess\":2},{\"Name\":\"Parkway Primary School\",\"Address1\":\"Alsike Road\",\"PostCode\":\"DA18 4DP\",\"OverallEffectiveNess\":2},{\"Name\":\"Belvedere Infant School\",\"Address1\":\"Mitchell"

SQL 服务器结果：

[{"Name":"ABBEY WOOD","Latitude":"51.490772","Longitude":"0.120326","LatestDepartureTime":"08:18:00","MinJourneyTime":14,"MaxJourneyTime":17,"schools":[{"Name":"Abbey Wood Nursery School","Address1":"Dahlia Road","PostCode":"SE2 0SX","OverallEffectiveNess":2},{"Name":"Bannockburn Primary School","Address1":"Plumstead High Street","PostCode":"SE18 1HE","OverallEffectiveNess":1},{"Name":"Gallions Mount Primary School","Address1":"Purrett Road","PostCode":"SE18 1JR","OverallEffectiveNess":2},{"Name":"Boxgrove Primary School","Address1":"Boxgrove Road","PostCode":"SE2 9JP","OverallEffectiveNess":2},{"Name":"De Lucy Primary School","Address1":"Cookhill Road","PostCode":"SE2 9PD","OverallEffectiveNess":2},{"Name":"Heronsgate Primary School","Address1":"Whinchat Road","PostCode":"SE28 0EA","OverallEffectiveNess":1},{"Name":"Linton Mead Primary School","Address1":"Central Way","PostCode":"SE28 8DT","OverallEffectiveNess":2},{"Name":"Greenslade Primary School","Address1":"Erindale","PostCode":"SE18 2QQ","OverallEffectiveNess":2},{"Name":"St Patrick's Catholic Primary School","Address1":"Griffin Road","PostCode":"SE18 7QG","OverallEffectiveNess":2},{"Name":"St Thomas A Becket Roman Catholic Primary School","Address1":"Mottisfont Road","PostCode":"SE2 9LY","OverallEffectiveNess":2},{"Name":"St Margaret Clitherow Catholic Primary School","Address1":"Cole Close","PostCode":"SE28 8GB","OverallEffectiveNess":2},{"Name":"Bishop John Robinson Church of England Primary School","Address1":"Hoveton Road","PostCode":"SE28 8LW","OverallEffectiveNess":2},{"Name":"Plumstead Manor School","Address1":"Old Mill Road","PostCode":"SE18 1QF","OverallEffectiveNess":3},{"Name":"Hawksmoor School","Address1":"Bentham Road","PostCode":"SE28 8AS","OverallEffectiveNess":1},{"Name":"Belmont Primary School","Address1":"Belmont Road","PostCode":"DA8 1LE","OverallEffectiveNess":2},{"Name":"Parkway Primary School","Address1":"Alsike Road","PostCode":"DA18 4DP","OverallEffectiveNess":2},{"Name":"Belvedere Infant School","Address1":"Mitchell Close","PostCode":"DA17 6AA","OverallEffectiveNess":2},{"Name":"Castilion Primary School","Address1":"Copperfield Road","PostCode":"SE28 8QA","OverallEffectiveNess":1},{"Name":"St Thomas More Catholic Primary School","Address1":"Sheldon Road","PostCode":"DA7 4PH","OverallEffectiveNess":1},{"Name":"St John Fisher Catholic Primary School","Address1":"Kale Road","PostCode":"DA18 4BA","OverallEffectiveNess":2},{"Name":"St Paul's Academy","Address1":"Finchale Road","PostCode":"SE2 9PX","OverallEffectiveNess":2},{"Name":"Discovery Primary School","Address1":"Battery Road","PostCode":"SE28 0JN","OverallEffectiveNess":2},{"Name":"Alexander McLeod Primary School","Address1":"Fuchsia Street","PostCode":"SE2 0QS","OverallEffectiveNess":2},{"Name":"Conway Primary School","Address1":"Gallosson Road","PostCode":"SE18 1QY","OverallEffectiveNess":2},{"Name":"Waterside School","Address1":"Robert Street","PostCode":"SE18 7NB","OverallEffectiveNess":2},{"Name":"Trinity Church of England School, Belvedere","Address1":"Erith Road","PostCode":"DA17 6HT","OverallEffectiveNess":2},{"Name":"East Wickham Primary Academy","Address1":"Wickham Street","PostCode":"DA16 3BP","OverallEffectiveNess":2},{"Name":"Welling School","Address1":"Elsa Road","PostCode":"DA16 1LB","OverallEffectiveNess":2},{"Name":"Willow Bank Primary School","Address1":"Seacourt Road","PostCode":"SE2 9XB","OverallEffectiveNess":2},{"Name":"Belvedere Junior School","Address1":"Mitchell Close","PostCode":"DA17 6AA","OverallEffectiveNess":2},{"Name":"St Augustine of Canterbury CofE Primary School","Address1":"St Augustine's Road","PostCode":"DA17 5HP","OverallEffectiveNess":2},{"Name":"Brampton Primary Academy","Address1":"Brampton Road","PostCode":"DA7 4SL","OverallEffectiveNess":4},{"Name":"Woolwich Polytechnic School","Address1":"Hutchins Road","PostCode":"SE28 8AT","OverallEffectiveNess":1},{"Name":"Hillsgrove Primary School","Address1":"Sidmouth Road","PostCode":"DA16 1DR","OverallEffectiveNess":3},{"Name":"Windrush Primary School","Address1":"2 Bentham Road","PostCode":"SE28 8AR","OverallEffectiveNess":1},{"Name":"Timbercroft Primary School","Address1":"Timbercroft Lane","PostCode":"SE18 2SG","OverallEffectiveNess":2},{"Name":"Rockliffe Manor Primary School","Address1":"Bassant Road","PostCode":"SE18 2NP","OverallEffectiveNess":2},{"Name":"Willow Dene School","Address1":"Swingate Lane","PostCode":"SE18 2JD","OverallEffectiveNess":2},{"Name":"South Rise Primary School","Address1":"Brewery Road","PostCode":"SE18 7PX","OverallEffectiveNess":2},{"Name":"Bedonwell Infant and Nursery School","Address1":"Bedonwell Road","PostCode":"DA17 5PF","OverallEffectiveNess":1},{"Name":"Bedonwell Junior School","Address1":"Bedonwell Road","PostCode":"DA17 5PF","OverallEffectiveNess":2},{"Name":"Pathways Short Stay School","Address1":"Pathways Short Stay School","PostCode":"SE2 9TA","OverallEffectiveNess":2},{"Name":"Northwood Primary School","Address1":"Northwood Place","PostCode":"DA18 4HN","OverallEffectiveNess":2},{"Name":"St Michael's East Wickham Church of England Voluntary Aided Primary School","Address1":"Wrotham Road","PostCode":"DA16 1LS","OverallEffectiveNess":1},{"Name":"Jubilee Primary School","Address1":"Crowden Way","PostCode":"SE28 8JB","OverallEffectiveNess":3}]}]

PYODBC 在某处似乎被截断为大约 2050 个字符？

我想我不是第一个尝试使用 FOR JSON 和 PYODBC 的人，所以我做错了什么？

我的连接字符串（如果它很重要）：

cnxn = r'Driver={SQL Server};Server=MARKSLAPTOP\MSSQLSERVER17;Database=MyDBName;Trusted_Connection=yes;'

而 cnxn.query 只是调用 cursor.execute(sql, params)。

编辑这个问题听起来类似于查询，但我已经在使用标准 SQL 服务器驱动程序...

Answer 1

多亏了以上评论的组合，我才成功地完成了这项工作。

如果其他人遇到我的问题，我会总结一下：

根据@GordThompson 的评论，我将驱动程序从旧的 SQL Server 更改为 ODBC Driver 13 for SQL Server。升级驱动程序版本后，我的参数传递需要进行一些调整，但总的来说，这似乎是一个明智之举
也许更重要的是，@FlipperPA 的评论建议我 CAST SQL 服务器查询中的 JSON 对象到 VARCHAR(MAX)。

SELECT CAST( (SELECT ..... FOR JSON PATH) AS VARCHAR(MAX))

这返回了被选中的完整 JSON 对象，现在一切正常。

我只能假设在 SQL Server 2017/Azure 中支持 JSON 的 pyodbc 存在一些问题？

Answer 2

我运行今天遇到了同样奇怪的问题。 JSON 结果本应是单行单列，但在 2033 个字符处被截断。

但是，结果集都在那里，只是分布在许多行中！你可以重建它：

rows = cursor.fetchall()
json_result = ''
json_result = json_result.join([row[0] for row in rows])

或者：

rows = cursor.fetchall()
search_results = json.loads(''.join([row[0] for row in rows]))

在 ODBC Driver 13 for SQL Server 和 SQL Server 驱动程序之间切换没有帮助。我在 Windows 10、Python 3.7、pyodbc 4.0.26 和 SQL Server 2017。

这似乎是一个错误。

PYODBC 截断 SQL 服务器 FOR JSON 查询的响应

PYODBC truncates the response of a SQL Server FOR JSON query

json

sql-server

flask

pyodbc

sql-server-2017