Databricks variant_explode usage leads to unparsable section #6679
Description
Search before asking
- I searched the issues and found no similar issues.
What Happened
Using the Databricks dialect variant_explode table valued function like below produces an error
select src.account_id, item.*
from catalog.schema.source_table as src,
lateral variant_explode(parse_json(src.rawresponse):value) as itemThe error produced:
L: 3 | P: 25 | PRS | Line 3, Position 25: Found unparsable section: ',\nlateral variant_explode(parse_json(src...'
WARNING: Parsing errors found and dialect is set to 'databricks'. Have you configured your dialect correctly?
Expected Behaviour
sqlfluff Databricks dialect supports this valid Databricks SQL
Observed Behaviour
N/A
How to reproduce
Run sqlfluff lint <file> while using the databricks dialect on the provided SQL code
Dialect
Databricks
Version
Python 3.13.1
sqlfluff, version 3.3.0
Configuration
pyproject.toml:
# SQLFluff Core
[tool.sqlfluff.core]
templater = "dbt"
dialect = "databricks"
verbose = 0
processes = 0
max_line_length = 120
output_line_length = 120
large_file_skip_byte_limit = 30000
ignore = ['examples/**/*']
exclude_rules = ["layout.indent", "layout.cte_bracket", "layout.long_lines", "layout.select_targets", "layout.spacing", "layout.keyword_newline", "layout.cte_newline", "aliasing.unused", "structure.using", "structure.column_order", "structure.unused_cte", "references.consistent"]
ignore_paths = [
"**venv**",
"dbt_packages/",
"target/",
"macros/",
"examples/"
]
# SQLFluff Templater
[tool.sqlfluff.templater.jinja]
apply_dbt_builtins = true
[tool.sqlfluff.templater.dbt]
project_dir = "."
profiles_dir = "./.dbt"
# SQLFluff Base Rules
[tool.sqlfluff.rules]
aliasing = "explicit"
min_alias_length = 3
capitalisation_policy = 'lower'
extended_capitalisation_policy = 'lower'
# Capitalization
[tool.sqlfluff.rules.capitalisation.keywords]
capitalisation_policy = "lower"
[tool.sqlfluff.rules.capitalisation.identifiers]
capitalisation_policy = "lower"
[tool.sqlfluff.rules.capitalisation.functions]
extended_capitalisation_policy = "lower"
[tool.sqlfluff.rules.capitalisation.literals]
capitalisation_policy = "lower"
[tool.sqlfluff.rules.capitalisation.types]
extended_capitalisation_policy = "lower"
# SQLFluff Indentation
[tool.sqlfluff.indentation]
indented_joins = false
indented_using_on = true
indented_on_contents = false
allow_implicit_indents = true
# SQLFluff Convention
[tool.sqlfluff.rules.convention.quoted_literals]
preferred_quoted_literal_style = "single_quotes"
[tool.sqlfluff.rules.convention.terminator]
multiline_newline = trueAre you willing to work on and submit a PR to address the issue?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct