From 7569eeff270275e1ddacb66b28f50fbf101542b7 Mon Sep 17 00:00:00 2001 From: Leila Lali Date: Tue, 30 Sep 2025 21:21:51 +0000 Subject: [PATCH 1/7] Merged PR 1818918: Adding for Order Clause to JSON_ARRAYAGG # Pull Request Template for ScriptDom ## Description Please provide a detailed description, include the link to the design specification or SQL feature document for the new TSQL syntaxes. Make sure to add links to the Github or DevDiv issue Before submitting your pull request, please ensure you have completed the following: ## Code Change - [ ] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed - [ ] Code changes are accompanied by appropriate unit tests - [ ] Identified and included SMEs needed to review code changes - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code ## Testing - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature ## Documentation - [ ] Update relevant documentation in the [wiki](https://dev.azure.com/msdata/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki) and the README.md file ## Additional Information Please provide any additional information that might be helpful for the reviewers Adding for Order Clause to JSON_ARRAYAGG ---- #### AI description (iteration 1) #### PR Classification New feature addition to support an ORDER BY clause in JSON_ARRAYAGG. #### PR Summary This pull request adds comprehensive support for an ORDER BY clause in JSON_ARRAYAGG function calls by updating the parser grammar, AST definitions, and script generation logic, accompanied by new tests ensuring correct implementation. - In `SqlScriptDom/Parser/TSql/TSql170.g`, updated grammar rules to include an optional `OrderByClause` for JSON_ARRAYAGG. - In `SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.FunctionCall.cs`, added logic to generate the ORDER BY clause when present. - In `SqlScriptDom/Parser/TSql/Ast.xml`, appended a new member `JsonOrderByClause` to support the syntax. - Added new test scripts and baselines in `Test/SqlDom/TestScripts/JsonArrayAggOrderBy170.sql` and `Test/SqlDom/Baselines170/JsonArrayAggOrderBy170.sql` to validate the functionality. --- CONTRIBUTING.md | 30 ++++++++++++++++--- SqlScriptDom/Parser/TSql/Ast.xml | 1 + SqlScriptDom/Parser/TSql/TSql170.g | 9 ++++++ .../SqlScriptGeneratorVisitor.FunctionCall.cs | 2 ++ .../Baselines170/JsonArrayAggOrderBy170.sql | 19 ++++++++++++ Test/SqlDom/Only170SyntaxTests.cs | 1 + Test/SqlDom/ParserErrorsTests.cs | 8 +++++ .../TestScripts/JsonArrayAggOrderBy170.sql | 21 +++++++++++++ 8 files changed, 87 insertions(+), 4 deletions(-) create mode 100644 Test/SqlDom/Baselines170/JsonArrayAggOrderBy170.sql create mode 100644 Test/SqlDom/TestScripts/JsonArrayAggOrderBy170.sql diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f4b000c..cc61129 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -106,14 +106,36 @@ Example: To run all priority 0 tests dotnet test --filter Priority=0 ``` +#### ⚠️ CRITICAL: Full Test Suite for Parser Changes + +**If you make ANY changes to grammar files (`.g` files) or AST definitions (`Ast.xml`), you MUST run the complete test suite** to ensure no regressions: + +```cmd +dotnet test Test/SqlDom/UTSqlScriptDom.csproj -c Debug +``` + +**Why this is critical for parser changes:** +- Grammar changes can have far-reaching effects on seemingly unrelated functionality +- Shared grammar rules are used in multiple contexts throughout the parser +- AST modifications can affect script generation and visitor patterns across the entire codebase +- Token recognition changes can impact parsing of statements that don't even use the modified feature + +**Example of unexpected failures:** +- Modifying a shared rule like `identifierColumnReferenceExpression` can cause other tests to fail because the rule now accepts syntax that should be rejected in different contexts +- Changes to operator precedence can affect unrelated expressions +- Adding new AST members without proper script generation support can break round-trip parsing + +Always verify that all ~557 tests pass before submitting your changes. + ### Pull Request Process Before sending a Pull Request, please do the following: -1. Ensure builds are still successful and tests, including any added or updated tests, pass prior to submitting the pull request. -2. Update any documentation, user and contributor, that is impacted by your changes. -3. Include your change description in `CHANGELOG.md` file as part of pull request. -4. You may merge the pull request in once you have the sign-off of two other developers, or if you do not have permission to do that, you may request the second reviewer to merge it for you. +1. **For parser changes (grammar/AST modifications): Run the complete test suite** (`dotnet test Test/SqlDom/UTSqlScriptDom.csproj -c Debug`) and ensure all ~557 tests pass. Grammar changes can have unexpected side effects. +2. Ensure builds are still successful and tests, including any added or updated tests, pass prior to submitting the pull request. +3. Update any documentation, user and contributor, that is impacted by your changes. +4. Include your change description in `CHANGELOG.md` file as part of pull request. +5. You may merge the pull request in once you have the sign-off of two other developers, or if you do not have permission to do that, you may request the second reviewer to merge it for you. ### Helpful notes for SQLDOM extensions diff --git a/SqlScriptDom/Parser/TSql/Ast.xml b/SqlScriptDom/Parser/TSql/Ast.xml index 347befd..8412790 100644 --- a/SqlScriptDom/Parser/TSql/Ast.xml +++ b/SqlScriptDom/Parser/TSql/Ast.xml @@ -652,6 +652,7 @@ + diff --git a/SqlScriptDom/Parser/TSql/TSql170.g b/SqlScriptDom/Parser/TSql/TSql170.g index d746306..a6fbef4 100644 --- a/SqlScriptDom/Parser/TSql/TSql170.g +++ b/SqlScriptDom/Parser/TSql/TSql170.g @@ -33017,6 +33017,7 @@ jsonArrayBuiltInFunctionCall [FunctionCall vParent] jsonArrayAggBuiltInFunctionCall [FunctionCall vParent] { ScalarExpression vExpression; + OrderByClause vOrderByClause; } : ( vExpression=expression @@ -33024,6 +33025,14 @@ jsonArrayAggBuiltInFunctionCall [FunctionCall vParent] AddAndUpdateTokenInfo(vParent, vParent.Parameters, vExpression); } ) + ( + vOrderByClause=orderByClause + { + vParent.JsonOrderByClause = vOrderByClause; + } + | + /* empty */ + ) ( jsonNullClauseFunction[vParent] | diff --git a/SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.FunctionCall.cs b/SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.FunctionCall.cs index 25c8bd0..3aad8e6 100644 --- a/SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.FunctionCall.cs +++ b/SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.FunctionCall.cs @@ -85,6 +85,8 @@ public override void ExplicitVisit(FunctionCall node) else if (node.FunctionName.Value.ToUpper(CultureInfo.InvariantCulture) == CodeGenerationSupporter.JsonArrayAgg) { GenerateCommaSeparatedList(node.Parameters); + // Generate ORDER BY clause if present + GenerateSpaceAndFragmentIfNotNull(node.JsonOrderByClause); if (node.Parameters?.Count > 0 && node?.AbsentOrNullOnNull?.Count > 0) //If there are values and null on null or absent on null present then generate space in between them GenerateSpace(); GenerateNullOnNullOrAbsentOnNull(node?.AbsentOrNullOnNull); diff --git a/Test/SqlDom/Baselines170/JsonArrayAggOrderBy170.sql b/Test/SqlDom/Baselines170/JsonArrayAggOrderBy170.sql new file mode 100644 index 0000000..d2ab25f --- /dev/null +++ b/Test/SqlDom/Baselines170/JsonArrayAggOrderBy170.sql @@ -0,0 +1,19 @@ +SELECT JSON_ARRAYAGG(value ORDER BY value) +FROM mytable; + +SELECT JSON_ARRAYAGG(name ORDER BY name ASC) +FROM users; + +SELECT JSON_ARRAYAGG(score ORDER BY score DESC) +FROM scores; + +SELECT JSON_ARRAYAGG(data ORDER BY priority DESC, created_at ASC) +FROM records; + +SELECT JSON_ARRAYAGG(value ORDER BY value NULL ON NULL) +FROM data; + +SELECT TOP (5) c.object_id, + JSON_ARRAYAGG(c.name ORDER BY c.column_id) AS column_list +FROM sys.columns AS c +GROUP BY c.object_id; \ No newline at end of file diff --git a/Test/SqlDom/Only170SyntaxTests.cs b/Test/SqlDom/Only170SyntaxTests.cs index c0d7765..4bcde4f 100644 --- a/Test/SqlDom/Only170SyntaxTests.cs +++ b/Test/SqlDom/Only170SyntaxTests.cs @@ -20,6 +20,7 @@ public partial class SqlDomTests new ParserTest170("RegexpTests170.sql", nErrors80: 0, nErrors90: 0, nErrors100: 0, nErrors110: 0, nErrors120: 0, nErrors130: 0, nErrors140: 0, nErrors150: 0, nErrors160: 0), new ParserTest170("AiGenerateChunksTests170.sql", nErrors80: 19, nErrors90: 16, nErrors100: 15, nErrors110: 15, nErrors120: 15, nErrors130: 15, nErrors140: 15, nErrors150: 15, nErrors160: 15), new ParserTest170("JsonFunctionTests170.sql", nErrors80: 13, nErrors90: 8, nErrors100: 38, nErrors110: 38, nErrors120: 38, nErrors130: 38, nErrors140: 38, nErrors150: 38, nErrors160: 38), + new ParserTest170("JsonArrayAggOrderBy170.sql", nErrors80: 6, nErrors90: 6, nErrors100: 6, nErrors110: 6, nErrors120: 6, nErrors130: 6, nErrors140: 6, nErrors150: 6, nErrors160: 6), new ParserTest170("AiGenerateEmbeddingsTests170.sql", nErrors80: 14, nErrors90: 11, nErrors100: 11, nErrors110: 11, nErrors120: 11, nErrors130: 11, nErrors140: 11, nErrors150: 11, nErrors160: 11), new ParserTest170("CreateExternalModelStatementTests170.sql", nErrors80: 2, nErrors90: 2, nErrors100: 2, nErrors110: 2, nErrors120: 2, nErrors130: 4, nErrors140: 4, nErrors150: 4, nErrors160: 4), new ParserTest170("AlterExternalModelStatementTests170.sql", nErrors80: 2, nErrors90: 2, nErrors100: 2, nErrors110: 2, nErrors120: 2, nErrors130: 5, nErrors140: 5, nErrors150: 5, nErrors160: 5), diff --git a/Test/SqlDom/ParserErrorsTests.cs b/Test/SqlDom/ParserErrorsTests.cs index 3644819..796ec80 100644 --- a/Test/SqlDom/ParserErrorsTests.cs +++ b/Test/SqlDom/ParserErrorsTests.cs @@ -501,6 +501,14 @@ public void JsonArraySyntaxNegativeTest() // Cannot use incomplete null on null clause cases ParserTestUtils.ErrorTest160("SELECT JSON_ARRAY('name', 'value', NULL, 'type' ON NULL)", new ParserErrorInfo(48, "SQL46010", "ON")); + + // Incorrect usage of ORDER BY in JSON_ARRAY + ParserTestUtils.ErrorTest170("SELECT JSON_ARRAYAGG(data ORDER priority DESC, created_at ASC) FROM records;", + new ParserErrorInfo(32, "SQL46010", "priority")); + + // Incorrect usage of ORDER BY in JSON_ARRAY + ParserTestUtils.ErrorTest170("SELECT JSON_ARRAYAGG(data ORDER By) FROM records;", + new ParserErrorInfo(34, "SQL46010", ")")); } /// diff --git a/Test/SqlDom/TestScripts/JsonArrayAggOrderBy170.sql b/Test/SqlDom/TestScripts/JsonArrayAggOrderBy170.sql new file mode 100644 index 0000000..094fb21 --- /dev/null +++ b/Test/SqlDom/TestScripts/JsonArrayAggOrderBy170.sql @@ -0,0 +1,21 @@ +-- Test JSON_ARRAYAGG with ORDER BY clause + +-- Basic ORDER BY +SELECT JSON_ARRAYAGG(value ORDER BY value) FROM mytable; + +-- ORDER BY with ASC +SELECT JSON_ARRAYAGG(name ORDER BY name ASC) FROM users; + +-- ORDER BY with DESC +SELECT JSON_ARRAYAGG(score ORDER BY score DESC) FROM scores; + +-- ORDER BY with multiple columns +SELECT JSON_ARRAYAGG(data ORDER BY priority DESC, created_at ASC) FROM records; + +-- ORDER BY with NULL CLAUSE +SELECT JSON_ARRAYAGG(value ORDER BY value NULL ON NULL) FROM data; + +-- Real-world example with GROUP BY and system tables +SELECT TOP(5) c.object_id, JSON_ARRAYAGG(c.name ORDER BY c.column_id) AS column_list +FROM sys.columns AS c +GROUP BY c.object_id; \ No newline at end of file From 5fae80e76e41a5a534537b9e972592731d6bc928 Mon Sep 17 00:00:00 2001 From: Leila Lali Date: Tue, 30 Sep 2025 21:35:49 +0000 Subject: [PATCH 2/7] Merged PR 1819059: Fix parsing 'WITH ARRAY WRAPPER' in json_query function # Pull Request Template for ScriptDom ## Description Please provide a detailed description, include the link to the design specification or SQL feature document for the new TSQL syntaxes. Make sure to add links to the Github or DevDiv issue Before submitting your pull request, please ensure you have completed the following: ## Code Change - [ ] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed - [ ] Code changes are accompanied by appropriate unit tests - [ ] Identified and included SMEs needed to review code changes - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code ## Testing - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature ## Documentation - [ ] Update relevant documentation in the [wiki](https://dev.azure.com/msdata/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki) and the README.md file ## Additional Information Please provide any additional information that might be helpful for the reviewers Fix parsing 'WITH ARRAY WRAPPER' in json_query function ---- #### AI description (iteration 1) #### PR Classification Bug fix addressing incorrect parsing of the JSON_QUERY function when using the WITH ARRAY WRAPPER clause. #### PR Summary This pull request corrects the token generation for the JSON_QUERY function so that the WITH ARRAY WRAPPER clause is correctly parsed inside the function's parameter list. The changes adjust the grammar, script generator, and test expectations to match the proper syntax. - `Test/SqlDom/ParserErrorsTests.cs`: Updated error tests to reflect new token positions for the WITH ARRAY WRAPPER clause. - `SqlScriptDom/Parser/TSql/TSql170.g`: Modified grammar rules to move the closing parenthesis generation after handling the WITH clause. - `SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.FunctionCall.cs`: Reordered token generation to include the WITH ARRAY WRAPPER clause within the function call parameters. - `Test/SqlDom/Baselines170/JsonFunctionTests170.sql` and `Test/SqlDom/TestScripts/JsonFunctionTests170.sql`: Updated baseline and test scripts to match the corrected function syntax. Related work items: #4716932 --- SqlScriptDom/Parser/TSql/TSql170.g | 8 ++++---- .../SqlScriptGeneratorVisitor.FunctionCall.cs | 5 +++-- .../Baselines170/JsonFunctionTests170.sql | 2 +- Test/SqlDom/ParserErrorsTests.cs | 20 ++++++++----------- .../TestScripts/JsonFunctionTests170.sql | 2 +- 5 files changed, 17 insertions(+), 20 deletions(-) diff --git a/SqlScriptDom/Parser/TSql/TSql170.g b/SqlScriptDom/Parser/TSql/TSql170.g index a6fbef4..7392162 100644 --- a/SqlScriptDom/Parser/TSql/TSql170.g +++ b/SqlScriptDom/Parser/TSql/TSql170.g @@ -33094,10 +33094,6 @@ jsonQueryBuiltInFunctionCall [FunctionCall vParent] AddAndUpdateTokenInfo(vParent, vParent.Parameters, vPath); } )? - tRParen:RightParenthesis - { - UpdateTokenInfo(vParent, tRParen); - } ( With tArray:Identifier tWrapper:Identifier { @@ -33112,6 +33108,10 @@ jsonQueryBuiltInFunctionCall [FunctionCall vParent] vParent.WithArrayWrapper = true; } )? + tRParen:RightParenthesis + { + UpdateTokenInfo(vParent, tRParen); + } ; regularBuiltInFunctionCall [FunctionCall vParent] diff --git a/SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.FunctionCall.cs b/SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.FunctionCall.cs index 3aad8e6..0003a92 100644 --- a/SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.FunctionCall.cs +++ b/SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.FunctionCall.cs @@ -98,9 +98,8 @@ public override void ExplicitVisit(FunctionCall node) else if (node.FunctionName.Value.ToUpper(CultureInfo.InvariantCulture) == CodeGenerationSupporter.JsonQuery) { GenerateCommaSeparatedList(node.Parameters); - GenerateSymbol(TSqlTokenType.RightParenthesis); - // Handle WITH ARRAY WRAPPER clause + // Handle WITH ARRAY WRAPPER clause - inside parentheses if (node.WithArrayWrapper) { GenerateSpace(); @@ -110,6 +109,8 @@ public override void ExplicitVisit(FunctionCall node) GenerateSpace(); GenerateIdentifier(CodeGenerationSupporter.Wrapper); } + + GenerateSymbol(TSqlTokenType.RightParenthesis); } else { diff --git a/Test/SqlDom/Baselines170/JsonFunctionTests170.sql b/Test/SqlDom/Baselines170/JsonFunctionTests170.sql index c8042bc..7808061 100644 --- a/Test/SqlDom/Baselines170/JsonFunctionTests170.sql +++ b/Test/SqlDom/Baselines170/JsonFunctionTests170.sql @@ -140,7 +140,7 @@ SELECT JSON_QUERY('{ "a": 1 }'); SELECT JSON_QUERY('{ "a": 1 }', '$.a'); -SELECT JSON_QUERY('{ "a": [1,2,3] }', '$.a') WITH ARRAY WRAPPER; +SELECT JSON_QUERY('{ "a": [1,2,3] }', '$.a' WITH ARRAY WRAPPER); GO diff --git a/Test/SqlDom/ParserErrorsTests.cs b/Test/SqlDom/ParserErrorsTests.cs index 796ec80..1181fd3 100644 --- a/Test/SqlDom/ParserErrorsTests.cs +++ b/Test/SqlDom/ParserErrorsTests.cs @@ -598,28 +598,24 @@ public void JsonQuerySyntaxNegativeTest() new ParserErrorInfo(32, "SQL46010", "WITH")); // Cannot use WITH ARRAY without WRAPPER (unexpected end of file) - ParserTestUtils.ErrorTest170("SELECT JSON_QUERY('{ \"a\": 1 }') WITH ARRAY", - new ParserErrorInfo(42, "SQL46029", "")); + ParserTestUtils.ErrorTest170("SELECT JSON_QUERY('{ \"a\": 1 }' WITH ARRAY", + new ParserErrorInfo(41, "SQL46029", "")); // Cannot use WITH WRAPPER without ARRAY (unexpected end of file) - ParserTestUtils.ErrorTest170("SELECT JSON_QUERY('{ \"a\": 1 }') WITH WRAPPER", - new ParserErrorInfo(44, "SQL46029", "")); + ParserTestUtils.ErrorTest170("SELECT JSON_QUERY('{ \"a\": 1 }' WITH WRAPPER", + new ParserErrorInfo(43, "SQL46029", "")); // Cannot use incorrect keyword instead of ARRAY - ParserTestUtils.ErrorTest170("SELECT JSON_QUERY('{ \"a\": 1 }') WITH OBJECT WRAPPER", - new ParserErrorInfo(37, "SQL46010", "OBJECT")); + ParserTestUtils.ErrorTest170("SELECT JSON_QUERY('{ \"a\": 1 }' WITH OBJECT WRAPPER", + new ParserErrorInfo(36, "SQL46010", "OBJECT")); // Cannot use incorrect keyword instead of WRAPPER - ParserTestUtils.ErrorTest170("SELECT JSON_QUERY('{ \"a\": 1 }') WITH ARRAY OBJECT", - new ParserErrorInfo(43, "SQL46010", "OBJECT")); + ParserTestUtils.ErrorTest170("SELECT JSON_QUERY('{ \"a\": 1 }' WITH ARRAY OBJECT", + new ParserErrorInfo(42, "SQL46010", "OBJECT")); // Cannot use JSON_QUERY with colon syntax (key:value pairs like JSON_OBJECT) ParserTestUtils.ErrorTest170("SELECT JSON_QUERY('name':'value')", new ParserErrorInfo(24, "SQL46010", ":")); - - // WITH ARRAY WRAPPER must come after closing parenthesis, not before - ParserTestUtils.ErrorTest170("SELECT JSON_QUERY('{ \"a\": 1 }' WITH ARRAY WRAPPER)", - new ParserErrorInfo(31, "SQL46010", "WITH")); } /// diff --git a/Test/SqlDom/TestScripts/JsonFunctionTests170.sql b/Test/SqlDom/TestScripts/JsonFunctionTests170.sql index 89564cf..4265c6a 100644 --- a/Test/SqlDom/TestScripts/JsonFunctionTests170.sql +++ b/Test/SqlDom/TestScripts/JsonFunctionTests170.sql @@ -133,7 +133,7 @@ GO SELECT JSON_QUERY('{ "a": 1 }'); SELECT JSON_QUERY('{ "a": 1 }', '$.a'); -SELECT JSON_QUERY('{ "a": [1,2,3] }', '$.a') WITH ARRAY WRAPPER; +SELECT JSON_QUERY('{ "a": [1,2,3] }', '$.a' WITH ARRAY WRAPPER); GO CREATE VIEW dbo.jsonfunctest AS From 8ef4f29018042cc6a08de79ace54fd9c211eedec Mon Sep 17 00:00:00 2001 From: Urvashi Raj Date: Wed, 1 Oct 2025 05:57:01 +0000 Subject: [PATCH 3/7] Merged PR 1810300: Updated syntax for EM permissions tests # Pull Request Template for ScriptDom ## Description External Model Object are not schema - scoped and only database scoped. hence, tests scripts are updated accordingly. Before submitting your pull request, please ensure you have completed the following: ## Code Change - [ ] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed - [ ] Code changes are accompanied by appropriate unit tests - [ ] Identified and included SMEs needed to review code changes - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code ## Testing - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature ## Documentation - [ ] Update relevant documentation in the [wiki](https://dev.azure.com/msdata/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki) and the README.md file ## Additional Information Please provide any additional information that might be helpful for the reviewers Updated syntax for EM permissions ---- #### AI description (iteration 1) #### PR Classification Syntax update for external model permissions tests. #### PR Summary This pull request revises the SQL syntax in EM permissions tests by removing explicit schema prefixes from external model names, ensuring consistent and simplified permission statements. - Changes in `Test/SqlDom/Baselines170/SecurityStatementExternalModelTests170.sql`: Removed redundant schema qualifiers (e.g., `prod.`, `dbo.`, `schema1.`, `test.`) from external model object names. - Changes in `Test/SqlDom/TestScripts/SecurityStatementExternalModelTests170.sql`: Updated SQL statements to strip schema qualifiers, standardizing external model naming across the test scripts. --- .../SecurityStatementExternalModelTests170.sql | 18 +++++++++--------- .../SecurityStatementExternalModelTests170.sql | 18 +++++++++--------- 2 files changed, 18 insertions(+), 18 deletions(-) diff --git a/Test/SqlDom/Baselines170/SecurityStatementExternalModelTests170.sql b/Test/SqlDom/Baselines170/SecurityStatementExternalModelTests170.sql index a64b7b7..4800677 100644 --- a/Test/SqlDom/Baselines170/SecurityStatementExternalModelTests170.sql +++ b/Test/SqlDom/Baselines170/SecurityStatementExternalModelTests170.sql @@ -10,15 +10,15 @@ GRANT EXECUTE ON EXTERNAL MODEL::MyPredictionModel TO analyst; GRANT EXECUTE - ON EXTERNAL MODEL::schema1.ModelName TO user1, user2 + ON EXTERNAL MODEL::ModelName TO user1, user2 WITH GRANT OPTION; GRANT CONTROL - ON EXTERNAL MODEL::dbo.SalesModel TO mladmin + ON EXTERNAL MODEL::SalesModel TO mladmin AS dbo; GRANT VIEW DEFINITION - ON EXTERNAL MODEL::MySchema.MyModel TO PUBLIC; + ON EXTERNAL MODEL::MyModel TO PUBLIC; DENY ALTER ANY EXTERNAL MODEL TO restricteduser; @@ -26,7 +26,7 @@ DENY EXECUTE ON EXTERNAL MODEL::TestModel TO guest CASCADE; DENY CONTROL - ON EXTERNAL MODEL::test.model1 TO user1, user2 CASCADE + ON EXTERNAL MODEL::model1 TO user1, user2 CASCADE AS admin; REVOKE ALTER ANY EXTERNAL MODEL TO olduser; @@ -35,22 +35,22 @@ REVOKE GRANT OPTION FOR EXECUTE ON EXTERNAL MODEL::MyModel TO tempuser CASCADE; REVOKE CONTROL - ON EXTERNAL MODEL::prod.RecommendationModel TO contractor + ON EXTERNAL MODEL::RecommendationModel TO contractor AS manager; REVOKE EXECUTE - ON EXTERNAL MODEL::dbo.CustomerModel TO PUBLIC CASCADE; + ON EXTERNAL MODEL::CustomerModel TO PUBLIC CASCADE; ALTER AUTHORIZATION ON EXTERNAL MODEL::MyModel TO newowner; ALTER AUTHORIZATION - ON EXTERNAL MODEL::schema1.ModelName + ON EXTERNAL MODEL::ModelName TO SCHEMA OWNER; ALTER AUTHORIZATION - ON EXTERNAL MODEL::test.experimentalmodel + ON EXTERNAL MODEL::experimentalmodel TO datateam; GRANT EXECUTE, VIEW DEFINITION @@ -63,7 +63,7 @@ REVOKE EXECUTE, VIEW DEFINITION ON EXTERNAL MODEL::OldModel TO formeranalyst; GRANT EXECUTE - ON EXTERNAL MODEL::MyDatabase.MySchema.ModelWithSpaces TO testuser; + ON EXTERNAL MODEL::ModelWithSpaces TO testuser; DENY ALTER ANY EXTERNAL MODEL TO userwithspaces; diff --git a/Test/SqlDom/TestScripts/SecurityStatementExternalModelTests170.sql b/Test/SqlDom/TestScripts/SecurityStatementExternalModelTests170.sql index 04442d6..679953a 100644 --- a/Test/SqlDom/TestScripts/SecurityStatementExternalModelTests170.sql +++ b/Test/SqlDom/TestScripts/SecurityStatementExternalModelTests170.sql @@ -6,19 +6,19 @@ GRANT ALTER ANY EXTERNAL MODEL TO mlrole, adminrole GRANT EXECUTE ON EXTERNAL MODEL::MyPredictionModel TO analyst; GRANT EXECUTE - ON EXTERNAL MODEL::schema1.ModelName TO user1, user2 + ON EXTERNAL MODEL::ModelName TO user1, user2 WITH GRANT OPTION; GRANT CONTROL - ON EXTERNAL MODEL::dbo.SalesModel TO mladmin + ON EXTERNAL MODEL::SalesModel TO mladmin AS dbo; GRANT VIEW DEFINITION - ON EXTERNAL MODEL::MySchema.MyModel TO PUBLIC; + ON EXTERNAL MODEL::MyModel TO PUBLIC; DENY ALTER ANY EXTERNAL MODEL TO restricteduser; DENY EXECUTE ON EXTERNAL MODEL::TestModel TO guest CASCADE; DENY CONTROL - ON EXTERNAL MODEL::test.model1 TO user1, user2 + ON EXTERNAL MODEL::model1 TO user1, user2 CASCADE AS admin; REVOKE ALTER ANY EXTERNAL MODEL FROM olduser; @@ -26,14 +26,14 @@ REVOKE GRANT OPTION FOR EXECUTE ON EXTERNAL MODEL::MyModel FROM tempuser CASCADE; REVOKE CONTROL - ON EXTERNAL MODEL::prod.RecommendationModel FROM contractor + ON EXTERNAL MODEL::RecommendationModel FROM contractor AS manager; REVOKE EXECUTE - ON EXTERNAL MODEL::dbo.CustomerModel FROM PUBLIC + ON EXTERNAL MODEL::CustomerModel FROM PUBLIC CASCADE; ALTER AUTHORIZATION ON EXTERNAL MODEL::MyModel TO newowner; -ALTER AUTHORIZATION ON EXTERNAL MODEL::schema1.ModelName TO SCHEMA OWNER; -ALTER AUTHORIZATION ON EXTERNAL MODEL::test.experimentalmodel TO datateam; +ALTER AUTHORIZATION ON EXTERNAL MODEL::ModelName TO SCHEMA OWNER; +ALTER AUTHORIZATION ON EXTERNAL MODEL::experimentalmodel TO datateam; GRANT EXECUTE, VIEW DEFINITION ON EXTERNAL MODEL::MultiPermModel TO analyst; DENY EXECUTE, CONTROL @@ -41,7 +41,7 @@ DENY EXECUTE, CONTROL REVOKE EXECUTE, VIEW DEFINITION ON EXTERNAL MODEL::OldModel FROM formeranalyst; GRANT EXECUTE - ON EXTERNAL MODEL::MyDatabase.MySchema.ModelWithSpaces TO testuser; + ON EXTERNAL MODEL::ModelWithSpaces TO testuser; DENY ALTER ANY EXTERNAL MODEL TO userwithspaces; REVOKE CONTROL ON EXTERNAL MODEL::QuotedModel FROM QuotedUser; \ No newline at end of file From 959a0954338d30b832ae521522d2524c68e69099 Mon Sep 17 00:00:00 2001 From: Shiv Prashant Sood Date: Thu, 2 Oct 2025 15:26:08 +0000 Subject: [PATCH 4/7] Merged PR 1818887: [JSON] ScriptDom support for json_value Returning syntax Adds support to scriptdom for RETURNING option for json_value. Specifically support for json_value('a', 'b' RETURNING ) Code Change - [ ] The Common checklist has been reviewed and followed - [x] Code changes are accompanied by appropriate unit tests - [x] Identified and included SMEs needed to review code changes - [x] Follow the steps here to make changes in the code Testing Follow the steps here to add new tests for your feature Test are added to respective json test file. Documentation Update relevant documentation in the wiki and the README.md file Additional Information Support by updating grammar rules, AST definitions, code generation, and tests to handle a returning clause for JSON_VALUE with typed data. - `SqlScriptDom/Parser/TSql/TSql170.g`: Revised grammar rules to replace the JSON returning clause with one that uses a DataTypeReference, added a new jsonValueReturningClause and jsonDataType rule to enforce JSON type restrictions, and integrated JSON_VALUE into the built-in function call logic. - `SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.FunctionCall.cs`: Modified the return type generation logic to work with DataTypeReference and introduced specific handling for JSON_VALUE, ensuring proper generation of the returning clause. - `SqlScriptDom/Parser/TSql/Ast.xml`: Updated the AST definition for the ReturnType member from Identifier to DataTypeReference to support the new syntax. - `Test/SqlDom`: Expanded test cases in multiple SQL test scripts to include JSON_VALUE with various returning types, validating the new syntax. --- SqlScriptDom/Parser/TSql/Ast.xml | 2 +- .../Parser/TSql/CodeGenerationSupporter.cs | 1 + SqlScriptDom/Parser/TSql/TSql170.g | 105 ++++++++++++++++-- .../SqlScriptGeneratorVisitor.FunctionCall.cs | 50 ++++++++- .../Baselines170/JsonFunctionTests170.sql | 19 +++- Test/SqlDom/Only170SyntaxTests.cs | 2 +- Test/SqlDom/ParserErrorsTests.cs | 49 +++++++- .../TestScripts/JsonFunctionTests170.sql | 17 +++ 8 files changed, 230 insertions(+), 15 deletions(-) diff --git a/SqlScriptDom/Parser/TSql/Ast.xml b/SqlScriptDom/Parser/TSql/Ast.xml index 8412790..4e0e182 100644 --- a/SqlScriptDom/Parser/TSql/Ast.xml +++ b/SqlScriptDom/Parser/TSql/Ast.xml @@ -654,7 +654,7 @@ - + diff --git a/SqlScriptDom/Parser/TSql/CodeGenerationSupporter.cs b/SqlScriptDom/Parser/TSql/CodeGenerationSupporter.cs index 0a64b28..6d285f5 100644 --- a/SqlScriptDom/Parser/TSql/CodeGenerationSupporter.cs +++ b/SqlScriptDom/Parser/TSql/CodeGenerationSupporter.cs @@ -527,6 +527,7 @@ internal static class CodeGenerationSupporter internal const string JsonObjectAgg = "JSON_OBJECTAGG"; internal const string JsonArrayAgg = "JSON_ARRAYAGG"; internal const string JsonQuery = "JSON_QUERY"; + internal const string JsonValue = "JSON_VALUE"; internal const string Array = "ARRAY"; internal const string Wrapper = "WRAPPER"; internal const string Keep = "KEEP"; diff --git a/SqlScriptDom/Parser/TSql/TSql170.g b/SqlScriptDom/Parser/TSql/TSql170.g index 7392162..1850132 100644 --- a/SqlScriptDom/Parser/TSql/TSql170.g +++ b/SqlScriptDom/Parser/TSql/TSql170.g @@ -32631,22 +32631,86 @@ expressionList [TSqlFragment vParent, IList expressions] ) ; +/* jsonReturningClause is used by json_object, json_objectagg, json_array, json_arrayagg where only + RETURNING JSON is supported. Any other type with JSON should return in error */ jsonReturningClause [FunctionCall vParent] { - Identifier vJson; + DataTypeReference vDataType; } : - tReturning:Identifier tJson:Identifier + tReturning:Identifier vDataType=jsonDataType { Match(tReturning, CodeGenerationSupporter.Returning); - Match(tJson, CodeGenerationSupporter.Json); - UpdateTokenInfo(vParent,tJson); - vJson = FragmentFactory.CreateFragment(); - AddAndUpdateTokenInfo(vParent, vParent.ReturnType, vJson); - vJson.SetUnquotedIdentifier(tJson.getText()); + UpdateTokenInfo(vParent, tReturning); + vParent.ReturnType.Add(vDataType); } ; +jsonDataType returns [SqlDataTypeReference vResult = null] +{ + SchemaObjectName vJsonTypeName; +} +: + vJsonTypeName=schemaObjectTwoPartName + { + // Only allow JSON as the data type + if (vJsonTypeName.BaseIdentifier.Value.ToUpper(CultureInfo.InvariantCulture) != CodeGenerationSupporter.Json) + { + ThrowParseErrorException("SQL46005", vJsonTypeName, + TSqlParserResource.SQL46005Message, CodeGenerationSupporter.Json, vJsonTypeName.BaseIdentifier.Value); + } + + vResult = FragmentFactory.CreateFragment(); + vResult.Name = vJsonTypeName; + vResult.SqlDataTypeOption = SqlDataTypeOption.Json; + vResult.UpdateTokenInfo(vJsonTypeName); + } +; + +/* jsonValueReturningClause is used by json_value. Only json_value support RETURNING syntax*/ +jsonValueReturningClause [FunctionCall vParent] +{ + DataTypeReference vDataType; +} +: + tReturning:Identifier vDataType=scalarDataType + { + Match(tReturning, CodeGenerationSupporter.Returning); + UpdateTokenInfo(vParent, tReturning); + + // JSON_VALUE only supports specific SQL data types in RETURNING clause + if (vDataType is SqlDataTypeReference sqlDataType) + { + bool isAllowedType = sqlDataType.SqlDataTypeOption == SqlDataTypeOption.Int || + sqlDataType.SqlDataTypeOption == SqlDataTypeOption.TinyInt || + sqlDataType.SqlDataTypeOption == SqlDataTypeOption.SmallInt || + sqlDataType.SqlDataTypeOption == SqlDataTypeOption.BigInt || + sqlDataType.SqlDataTypeOption == SqlDataTypeOption.Float || + sqlDataType.SqlDataTypeOption == SqlDataTypeOption.Real || + sqlDataType.SqlDataTypeOption == SqlDataTypeOption.Decimal || + sqlDataType.SqlDataTypeOption == SqlDataTypeOption.Numeric || + sqlDataType.SqlDataTypeOption == SqlDataTypeOption.Bit || + sqlDataType.SqlDataTypeOption == SqlDataTypeOption.VarChar || + sqlDataType.SqlDataTypeOption == SqlDataTypeOption.NVarChar || + sqlDataType.SqlDataTypeOption == SqlDataTypeOption.Char || + sqlDataType.SqlDataTypeOption == SqlDataTypeOption.NChar || + sqlDataType.SqlDataTypeOption == SqlDataTypeOption.Date || + sqlDataType.SqlDataTypeOption == SqlDataTypeOption.Time || + sqlDataType.SqlDataTypeOption == SqlDataTypeOption.DateTime2 || + sqlDataType.SqlDataTypeOption == SqlDataTypeOption.DateTimeOffset; + + if (!isAllowedType) + { + ThrowParseErrorException("SQL46005", vDataType, + TSqlParserResource.SQL46005Message, "supported data type", sqlDataType.SqlDataTypeOption.ToString()); + } + } + + vParent.ReturnType.Add(vDataType); + } +; + + jsonKeyValueExpression returns [JsonKeyValue vResult = FragmentFactory.CreateFragment()] { ScalarExpression vKey; @@ -32970,6 +33034,9 @@ builtInFunctionCall returns [FunctionCall vResult = FragmentFactory.CreateFragme | {(vResult.FunctionName != null && vResult.FunctionName.Value.ToUpper(CultureInfo.InvariantCulture) == CodeGenerationSupporter.JsonQuery)}? jsonQueryBuiltInFunctionCall[vResult] + | + {(vResult.FunctionName != null && vResult.FunctionName.Value.ToUpper(CultureInfo.InvariantCulture) == CodeGenerationSupporter.JsonValue)}? + jsonValueBuiltInFunctionCall[vResult] | {(vResult.FunctionName != null && vResult.FunctionName.Value.ToUpper(CultureInfo.InvariantCulture) == CodeGenerationSupporter.Trim) && (NextTokenMatches(CodeGenerationSupporter.Leading) | NextTokenMatches(CodeGenerationSupporter.Trailing) | NextTokenMatches(CodeGenerationSupporter.Both))}? @@ -33114,6 +33181,30 @@ jsonQueryBuiltInFunctionCall [FunctionCall vParent] } ; +jsonValueBuiltInFunctionCall [FunctionCall vParent] +{ + ScalarExpression vExpression; + ScalarExpression vPath; +} + : vExpression=expression + { + AddAndUpdateTokenInfo(vParent, vParent.Parameters, vExpression); + } + Comma vPath=expression + { + AddAndUpdateTokenInfo(vParent, vParent.Parameters, vPath); + } + ( + jsonValueReturningClause[vParent] + | + /* empty */ + ) + tRParen:RightParenthesis + { + UpdateTokenInfo(vParent, tRParen); + } + ; + regularBuiltInFunctionCall [FunctionCall vParent] { ColumnReferenceExpression vColumn; diff --git a/SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.FunctionCall.cs b/SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.FunctionCall.cs index 0003a92..20de09c 100644 --- a/SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.FunctionCall.cs +++ b/SqlScriptDom/ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.FunctionCall.cs @@ -112,6 +112,16 @@ public override void ExplicitVisit(FunctionCall node) GenerateSymbol(TSqlTokenType.RightParenthesis); } + else if (node.FunctionName.Value.ToUpper(CultureInfo.InvariantCulture) == CodeGenerationSupporter.JsonValue) + { + GenerateCommaSeparatedList(node.Parameters); + if (node.ReturnType?.Count > 0) //If there are return types then generate space and return type clause + { + GenerateSpace(); + GenerateReturnType(node?.ReturnType); + } + GenerateSymbol(TSqlTokenType.RightParenthesis); + } else { GenerateUniqueRowFilter(node.UniqueRowFilter, false); @@ -164,13 +174,47 @@ private void GenerateNullOnNullOrAbsentOnNull(IList list) GenerateKeyword(TSqlTokenType.Null); } } - private void GenerateReturnType(IList list) + + // Generate returning clause with SQLType. + private void GenerateReturnType(IList list) { - if (list?.Count > 0 && list[0].Value?.ToUpper(CultureInfo.InvariantCulture) == CodeGenerationSupporter.Json) + if (list?.Count > 0) { GenerateIdentifier("RETURNING"); GenerateSpace(); - GenerateSpaceSeparatedList(list); + + // Generate each data type correctly + for (int i = 0; i < list.Count; i++) + { + if (i > 0) + GenerateSpace(); + + // Handle SqlDataTypeReference properly - need to generate the type name and parameters separately + if (list[i] is SqlDataTypeReference sqlDataType) + { + // Generate the data type name (e.g., NVARCHAR) + string dataTypeName = sqlDataType.SqlDataTypeOption.ToString().ToUpper(CultureInfo.InvariantCulture); + GenerateIdentifier(dataTypeName); + + // Generate parameters if any (e.g., (50)) + if (sqlDataType.Parameters?.Count > 0) + { + GenerateSymbol(TSqlTokenType.LeftParenthesis); + for (int j = 0; j < sqlDataType.Parameters.Count; j++) + { + if (j > 0) + GenerateSymbol(TSqlTokenType.Comma); + GenerateFragmentIfNotNull(sqlDataType.Parameters[j]); + } + GenerateSymbol(TSqlTokenType.RightParenthesis); + } + } + else + { + // For other data type references, use the default generation + GenerateFragmentIfNotNull(list[i]); + } + } } } } diff --git a/Test/SqlDom/Baselines170/JsonFunctionTests170.sql b/Test/SqlDom/Baselines170/JsonFunctionTests170.sql index 7808061..c2a8a56 100644 --- a/Test/SqlDom/Baselines170/JsonFunctionTests170.sql +++ b/Test/SqlDom/Baselines170/JsonFunctionTests170.sql @@ -154,4 +154,21 @@ GO SELECT TOP (5) c.object_id, JSON_OBJECTAGG(c.name:c.column_id) AS columns FROM sys.columns AS c -GROUP BY c.object_id; \ No newline at end of file +GROUP BY c.object_id; + +SELECT JSON_VALUE('a', '$'); +SELECT JSON_VALUE('c', '$' RETURNING INT); +SELECT JSON_VALUE('c', '$' RETURNING SMALLINT); +SELECT JSON_VALUE('c', '$' RETURNING BIGINT); +SELECT JSON_VALUE('c', '$' RETURNING TINYINT); +SELECT JSON_VALUE('c', '$' RETURNING NUMERIC); +SELECT JSON_VALUE('c', '$' RETURNING FLOAT); +SELECT JSON_VALUE('c', '$' RETURNING REAL); +SELECT JSON_VALUE('c', '$' RETURNING DECIMAL); +SELECT JSON_VALUE('c', '$' RETURNING CHAR); +SELECT JSON_VALUE('c', '$' RETURNING NVARCHAR(50)); +SELECT JSON_VALUE('c', '$' RETURNING NCHAR); +SELECT JSON_VALUE('c', '$' RETURNING DATE); +SELECT JSON_VALUE('c', '$' RETURNING DATETIME2); +SELECT JSON_VALUE('c', '$' RETURNING TIME); +SELECT JSON_VALUE('c', '$' RETURNING BIT); \ No newline at end of file diff --git a/Test/SqlDom/Only170SyntaxTests.cs b/Test/SqlDom/Only170SyntaxTests.cs index 4bcde4f..79e81f0 100644 --- a/Test/SqlDom/Only170SyntaxTests.cs +++ b/Test/SqlDom/Only170SyntaxTests.cs @@ -19,7 +19,7 @@ public partial class SqlDomTests new ParserTest170("CreateColumnStoreIndexTests170.sql", nErrors80: 3, nErrors90: 3, nErrors100: 3, nErrors110: 3, nErrors120: 3, nErrors130: 0, nErrors140: 0, nErrors150: 0, nErrors160: 0), new ParserTest170("RegexpTests170.sql", nErrors80: 0, nErrors90: 0, nErrors100: 0, nErrors110: 0, nErrors120: 0, nErrors130: 0, nErrors140: 0, nErrors150: 0, nErrors160: 0), new ParserTest170("AiGenerateChunksTests170.sql", nErrors80: 19, nErrors90: 16, nErrors100: 15, nErrors110: 15, nErrors120: 15, nErrors130: 15, nErrors140: 15, nErrors150: 15, nErrors160: 15), - new ParserTest170("JsonFunctionTests170.sql", nErrors80: 13, nErrors90: 8, nErrors100: 38, nErrors110: 38, nErrors120: 38, nErrors130: 38, nErrors140: 38, nErrors150: 38, nErrors160: 38), + new ParserTest170("JsonFunctionTests170.sql", nErrors80: 28, nErrors90: 8, nErrors100: 53, nErrors110: 53, nErrors120: 53, nErrors130: 53, nErrors140: 53, nErrors150: 53, nErrors160: 53), new ParserTest170("JsonArrayAggOrderBy170.sql", nErrors80: 6, nErrors90: 6, nErrors100: 6, nErrors110: 6, nErrors120: 6, nErrors130: 6, nErrors140: 6, nErrors150: 6, nErrors160: 6), new ParserTest170("AiGenerateEmbeddingsTests170.sql", nErrors80: 14, nErrors90: 11, nErrors100: 11, nErrors110: 11, nErrors120: 11, nErrors130: 11, nErrors140: 11, nErrors150: 11, nErrors160: 11), new ParserTest170("CreateExternalModelStatementTests170.sql", nErrors80: 2, nErrors90: 2, nErrors100: 2, nErrors110: 2, nErrors120: 2, nErrors130: 4, nErrors140: 4, nErrors150: 4, nErrors160: 4), diff --git a/Test/SqlDom/ParserErrorsTests.cs b/Test/SqlDom/ParserErrorsTests.cs index 1181fd3..99dc213 100644 --- a/Test/SqlDom/ParserErrorsTests.cs +++ b/Test/SqlDom/ParserErrorsTests.cs @@ -472,6 +472,14 @@ public void JsonObjectSyntaxNegativeTest() // Cannot use incomplete null on null clause cases ParserTestUtils.ErrorTest160("SELECT JSON_OBJECT('name':'value', 'type':NULL NULL ON)", new ParserErrorInfo(54, "SQL46010", ")")); + + // Cannot use anything other than JSON in RETURNING clause + ParserTestUtils.ErrorTest170("SELECT JSON_OBJECT('name':'a' RETURNING VARCHAR(10))", + new ParserErrorInfo(40, "SQL46005", "JSON", "VARCHAR")); + + // Cannot use anything other than JSON in RETURNING clause + ParserTestUtils.ErrorTest170("SELECT JSON_OBJECT('name':'a' RETURNING SMALLINT)", + new ParserErrorInfo(40, "SQL46005", "JSON", "SMALLINT")); } /// @@ -509,6 +517,18 @@ public void JsonArraySyntaxNegativeTest() // Incorrect usage of ORDER BY in JSON_ARRAY ParserTestUtils.ErrorTest170("SELECT JSON_ARRAYAGG(data ORDER By) FROM records;", new ParserErrorInfo(34, "SQL46010", ")")); + + // Cannot use anything other than JSON in RETURNING clause + ParserTestUtils.ErrorTest170("SELECT JSON_ARRAYAGG('name' RETURNING INT)", + new ParserErrorInfo(38, "SQL46005", "JSON", "INT")); + + // Cannot use anything other than JSON in RETURNING clause + ParserTestUtils.ErrorTest170("SELECT JSON_ARRAY('name' RETURNING VARCHAR(10))", + new ParserErrorInfo(35, "SQL46005", "JSON", "VARCHAR")); + + // Cannot use anything other than JSON in RETURNING clause + ParserTestUtils.ErrorTest170("SELECT JSON_ARRAY('name' RETURNING Float)", + new ParserErrorInfo(35, "SQL46005", "JSON", "Float")); } /// @@ -583,6 +603,31 @@ public void JsonArrayAggSyntaxNegativeTest() // Cannot use null on null incorrectly ParserTestUtils.ErrorTest160("SELECT JSON_ARRAYAGG('name', NULL NULL ON)", new ParserErrorInfo(34, "SQL46010", "NULL")); + + // Cannot use anything other than JSON in RETURNING clause + ParserTestUtils.ErrorTest170("SELECT JSON_ARRAYAGG('name' RETURNING INT)", + new ParserErrorInfo(38, "SQL46005", "JSON", "INT")); + + // Cannot use anything other than JSON in RETURNING clause + ParserTestUtils.ErrorTest170("SELECT JSON_ARRAYAGG('name' RETURNING NVARCHAR(10))", + new ParserErrorInfo(38, "SQL46005", "JSON", "NVARCHAR")); + } + + /// + /// Negative tests for Json_Value syntax in functions + /// + [TestMethod] + [Priority(0)] + [SqlStudioTestCategory(Category.UnitTest)] + public void JsonValueSyntaxNegativeTest() + { + // Cannot use anything other than JSON in RETURNING clause + ParserTestUtils.ErrorTest170("SELECT JSON_VALUE('', '$' RETURNING JSON)", + new ParserErrorInfo(36, "SQL46005", "supported data type", "Json")); + + // Cannot use anything other than JSON in RETURNING clause + ParserTestUtils.ErrorTest170("SELECT JSON_VALUE('', '$' RETURNING Vector)", + new ParserErrorInfo(36, "SQL46005", "supported data type", "Vector")); } /// @@ -7056,14 +7101,14 @@ ID INT IDENTITY(1,1), Name VARCHAR(50) ); "; - ParserTestUtils.ErrorTestFabricDW(identityColumnSyntax, new ParserErrorInfo(identityColumnSyntax.IndexOf("IDENTITY(") + 8, "SQL46010", "(")); + ParserTestUtils.ErrorTestFabricDW(identityColumnSyntax, new ParserErrorInfo(89, "SQL46010", "(")); string identityColumnSyntax2 = @"CREATE TABLE TestTable2 ( RecordID BIGINT IDENTITY(100,5), Description NVARCHAR(200) ); "; - ParserTestUtils.ErrorTestFabricDW(identityColumnSyntax2, new ParserErrorInfo(identityColumnSyntax2.IndexOf("IDENTITY(") + 8, "SQL46010", "(")); + ParserTestUtils.ErrorTestFabricDW(identityColumnSyntax2, new ParserErrorInfo(98, "SQL46010", "(")); } /// diff --git a/Test/SqlDom/TestScripts/JsonFunctionTests170.sql b/Test/SqlDom/TestScripts/JsonFunctionTests170.sql index 4265c6a..83ac909 100644 --- a/Test/SqlDom/TestScripts/JsonFunctionTests170.sql +++ b/Test/SqlDom/TestScripts/JsonFunctionTests170.sql @@ -146,3 +146,20 @@ GO SELECT TOP(5) c.object_id, JSON_OBJECTAGG(c.name:c.column_id) AS columns FROM sys.columns AS c GROUP BY c.object_id; + +SELECT JSON_VALUE('a', '$'); +SELECT JSON_VALUE('c', '$' RETURNING INT); +SELECT JSON_VALUE('c', '$' RETURNING SMALLINT); +SELECT JSON_VALUE('c', '$' RETURNING BIGINT); +SELECT JSON_VALUE('c', '$' RETURNING TINYINT); +SELECT JSON_VALUE('c', '$' RETURNING NUMERIC); +SELECT JSON_VALUE('c', '$' RETURNING FLOAT); +SELECT JSON_VALUE('c', '$' RETURNING REAL); +SELECT JSON_VALUE('c', '$' RETURNING DECIMAL); +SELECT JSON_VALUE('c', '$' RETURNING CHAR); +SELECT JSON_VALUE('c', '$' RETURNING NVARCHAR(50)); +SELECT JSON_VALUE('c', '$' RETURNING NCHAR); +SELECT JSON_VALUE('c', '$' RETURNING DATE); +SELECT JSON_VALUE('c', '$' RETURNING DATETIME2); +SELECT JSON_VALUE('c', '$' RETURNING TIME); +SELECT JSON_VALUE('c', '$' RETURNING BIT); \ No newline at end of file From 2f5ee60ca21179ff64f3fde11c499ad361d6f630 Mon Sep 17 00:00:00 2001 From: Lijun Ji Date: Fri, 3 Oct 2025 16:26:41 +0000 Subject: [PATCH 5/7] Merged PR 1823138: Add test cases for Json_contains and Json_modify # Pull Request Template for ScriptDom ## Description This PR added several test cases for Json_contains and Json_modify. These 2 Json functions don't have special syntax that needs special handling, so just added a few test cases. Before submitting your pull request, please ensure you have completed the following: ## Code Change - [ ] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed - [ ] Code changes are accompanied by appropriate unit tests - [ ] Identified and included SMEs needed to review code changes - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code ## Testing - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature ## Documentation - [ ] Update relevant documentation in the [wiki](https://dev.azure.com/msdata/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki) and the README.md file ## Additional Information Please provide any additional information that might be helpful for the reviewers add test cases for Json_contains and Json_modify Related work items: #4724160 --- .../Baselines170/JsonFunctionTests170.sql | 15 ++++++++++++++- .../TestScripts/JsonFunctionTests170.sql | 18 +++++++++++++++++- 2 files changed, 31 insertions(+), 2 deletions(-) diff --git a/Test/SqlDom/Baselines170/JsonFunctionTests170.sql b/Test/SqlDom/Baselines170/JsonFunctionTests170.sql index c2a8a56..75f07eb 100644 --- a/Test/SqlDom/Baselines170/JsonFunctionTests170.sql +++ b/Test/SqlDom/Baselines170/JsonFunctionTests170.sql @@ -171,4 +171,17 @@ SELECT JSON_VALUE('c', '$' RETURNING NCHAR); SELECT JSON_VALUE('c', '$' RETURNING DATE); SELECT JSON_VALUE('c', '$' RETURNING DATETIME2); SELECT JSON_VALUE('c', '$' RETURNING TIME); -SELECT JSON_VALUE('c', '$' RETURNING BIT); \ No newline at end of file +SELECT JSON_VALUE('c', '$' RETURNING BIT); + +SELECT id, + json_col +FROM tab1 +WHERE JSON_CONTAINS(json_col, 'abc', '$.a') = 1; + +SELECT id, + json_col +FROM tab1 +WHERE JSON_CONTAINS(json_col, 'abc%', '$.a', 1) = 1; + +SELECT JSON_MODIFY(json_col, '$.a', 30) +FROM tab1; diff --git a/Test/SqlDom/TestScripts/JsonFunctionTests170.sql b/Test/SqlDom/TestScripts/JsonFunctionTests170.sql index 83ac909..39b2901 100644 --- a/Test/SqlDom/TestScripts/JsonFunctionTests170.sql +++ b/Test/SqlDom/TestScripts/JsonFunctionTests170.sql @@ -162,4 +162,20 @@ SELECT JSON_VALUE('c', '$' RETURNING NCHAR); SELECT JSON_VALUE('c', '$' RETURNING DATE); SELECT JSON_VALUE('c', '$' RETURNING DATETIME2); SELECT JSON_VALUE('c', '$' RETURNING TIME); -SELECT JSON_VALUE('c', '$' RETURNING BIT); \ No newline at end of file +SELECT JSON_VALUE('c', '$' RETURNING BIT); + +-- Json_Contains +SELECT id, + json_col +FROM tab1 +WHERE JSON_CONTAINS(json_col, 'abc', '$.a') = 1; + +-- Json_Contains as LIKE +SELECT id, + json_col +FROM tab1 +WHERE JSON_CONTAINS(json_col, 'abc%', '$.a', 1) = 1; + +-- Json_Modify +SELECT JSON_MODIFY(json_col, '$.a', 30) +FROM tab1; From 24c70c388b18f10491c2750fd45bab01b112b570 Mon Sep 17 00:00:00 2001 From: Leila Lali Date: Fri, 3 Oct 2025 17:08:22 +0000 Subject: [PATCH 6/7] Merged PR 1824015: Adding the release notes for 170.128.0 Please provide a detailed description, include the link to the design specification or SQL feature document for the new TSQL syntaxes. Make sure to add links to the Github or DevDiv issue Before submitting your pull request, please ensure you have completed the following: - [ ] The [Common checklist](https://msdata.visualstudio.com/SQLToolsAndLibraries/_git/Common?path=/Templates/PR%20Checklist%20for%20SQLToolsAndLibraries.md&version=GBmain&_a=preview) has been reviewed and followed - [ ] Code changes are accompanied by appropriate unit tests - [ ] Identified and included SMEs needed to review code changes - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=make-the-changes-in) here to make changes in the code - [ ] Follow the [steps](https://msdata.visualstudio.com/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki/33838/Adding-or-Extending-TSql-support-in-Sql-Dom?anchor=to-extend-the-tests-do-the-following%3A) here to add new tests for your feature - [ ] Update relevant documentation in the [wiki](https://dev.azure.com/msdata/SQLToolsAndLibraries/_wiki/wikis/SQLToolsAndLibraries.wiki) and the README.md file Please provide any additional information that might be helpful for the reviewers Adding the release notes for 170.128.0 --- release-notes/170/170.128.0.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 release-notes/170/170.128.0.md diff --git a/release-notes/170/170.128.0.md b/release-notes/170/170.128.0.md new file mode 100644 index 0000000..77acacb --- /dev/null +++ b/release-notes/170/170.128.0.md @@ -0,0 +1,26 @@ +# Release Notes + +## Microsoft.SqlServer.TransactSql.ScriptDom 170.128.0 +This update brings the following changes over the previous release: + +### Target Platform Support + +* .NET Framework 4.7.2 (Windows x86, Windows x64) +* .NET 8 (Windows x86, Windows x64, Linux, macOS) +* .NET Standard 2.0+ (Windows x86, Windows x64, Linux, macOS) + +### Dependencies +* Updates .NET SDK to latest patch version 8.0.414 + +#### .NET Framework +#### .NET Core + +### New Features +* Adds support for RETURNING option for JSON functions + +### Fixed +* Fixes https://github.com/microsoft/SqlScriptDOM/issues/159 +* Fixes https://github.com/microsoft/SqlScriptDOM/issues/170 +### Changes + +### Known Issues From feda3e726aa8207c76748bc5b88dab29d84e0da9 Mon Sep 17 00:00:00 2001 From: Leila Lali Date: Fri, 3 Oct 2025 11:20:28 -0700 Subject: [PATCH 7/7] fixing the test --- Test/SqlDom/ParserErrorsTests.cs | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-) diff --git a/Test/SqlDom/ParserErrorsTests.cs b/Test/SqlDom/ParserErrorsTests.cs index 99dc213..1fb3965 100644 --- a/Test/SqlDom/ParserErrorsTests.cs +++ b/Test/SqlDom/ParserErrorsTests.cs @@ -7096,19 +7096,11 @@ FROM @SalesData [SqlStudioTestCategory(Category.UnitTest)] public void IdentityColumnNegativeTestsFabricDW() { - string identityColumnSyntax = @"CREATE TABLE TestTable1 ( - ID INT IDENTITY(1,1), - Name VARCHAR(50) - ); - "; - ParserTestUtils.ErrorTestFabricDW(identityColumnSyntax, new ParserErrorInfo(89, "SQL46010", "(")); + string identityColumnSyntax = @"CREATE TABLE TestTable1 (ID INT IDENTITY(1,1), Name VARCHAR(50));"; + ParserTestUtils.ErrorTestFabricDW(identityColumnSyntax, new ParserErrorInfo(40, "SQL46010", "(")); - string identityColumnSyntax2 = @"CREATE TABLE TestTable2 ( - RecordID BIGINT IDENTITY(100,5), - Description NVARCHAR(200) - ); - "; - ParserTestUtils.ErrorTestFabricDW(identityColumnSyntax2, new ParserErrorInfo(98, "SQL46010", "(")); + string identityColumnSyntax2 = @"CREATE TABLE TestTable2 (RecordID BIGINT IDENTITY(100,5), Description NVARCHAR(200));"; + ParserTestUtils.ErrorTestFabricDW(identityColumnSyntax2, new ParserErrorInfo(49, "SQL46010", "(")); } ///