Skip to content

Commit 88084b6

Browse files
esacteksabesacteksab
authored
fix: Dot in a string breaks the highlighting in Terraform. Fixes #988 (#1065)
Admittedly, I set how to make Chroma look more like Pygments. Pygments: ![pygments-string-capture](https://github.com/user-attachments/assets/8f19bfc6-533c-41d8-859b-219fc13d506e) Chroma: ![chroma-string-current-capture](https://github.com/user-attachments/assets/eda1fd37-d0e1-4b69-a92c-03299d540663) Pygments in my opinion does this correctly, with everything between `" ... "` is a string. Chroma does some inner interpolation to pull out those inner pieces. I can see both sides and in that spirit, I chose to keep the current convention. I do have a branch that will make Chroma like Pygments if that's the desired behavior. While looking into existing Terraform issues and Pull requests, I encountered #988 and wondered if I could solve it. As it currently exists: ![988-current-capture](https://github.com/user-attachments/assets/ff78195d-c9b6-4384-8395-f0de9c10bd97) Fixed in this PR: ![chroma-988-fixed-capture](https://github.com/user-attachments/assets/fd8db050-f3e1-4804-8717-f29f6e411e17) A silver lining is punctuation actually works and mirrors Pygments. There is a possible regression depending on one's point of view and I wanted to call it out or not hide it. Current: ![current-chroma-iam-policy-doc-github-dark-capture](https://github.com/user-attachments/assets/707f8c78-f440-44cb-beea-b29b3c913d5c) This PR: ![chroma-proposed-fixed-iam-policy-document-github-dark-capture](https://github.com/user-attachments/assets/c7927eb9-a4a0-4a28-a399-371b6c49c807) The punctuation inside strings is highlighted differently. I don't know whether this solution is worth it or if it is a regression not worth considering, of if it is even a regression at all. But I've spent too much time and admittedly, I don't understand the syntax well enough. I solved my problem, maybe another one, maybe not. I'll understand if you close this PR, perhaps someone else can pick it up and address the regression if it is considered a regression. If you treat everything inside `" ... "` as a string, then this possible regression becomes moot.
1 parent 62197e9 commit 88084b6

File tree

3 files changed

+318
-48
lines changed

3 files changed

+318
-48
lines changed

‎lexers/embedded/terraform.xml

Lines changed: 18 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,9 @@
33
<name>Terraform</name>
44
<alias>terraform</alias>
55
<alias>tf</alias>
6+
<alias>hcl</alias>
67
<filename>*.tf</filename>
8+
<filename>*.hcl</filename>
79
<mime_type>application/x-tf</mime_type>
810
<mime_type>application/x-terraform</mime_type>
911
</config>
@@ -13,22 +15,19 @@
1315
<token type="LiteralStringDouble"/>
1416
<pop depth="1"/>
1517
</rule>
16-
<rule pattern="\\\\">
18+
<rule pattern="\\\\&#34;">
1719
<token type="LiteralStringDouble"/>
1820
</rule>
19-
<rule pattern="\\\\&#34;">
21+
<rule pattern="[^&#34;\\\\$]+">
22+
<token type="LiteralStringDouble"/>
23+
</rule>
24+
<rule pattern="[^\\\\&#34;$]+">
2025
<token type="LiteralStringDouble"/>
2126
</rule>
2227
<rule pattern="\$\{">
2328
<token type="LiteralStringInterpol"/>
2429
<push state="interp-inside"/>
2530
</rule>
26-
<rule pattern="\$">
27-
<token type="LiteralStringDouble"/>
28-
</rule>
29-
<rule pattern="[^&#34;\\\\$]+">
30-
<token type="LiteralStringDouble"/>
31-
</rule>
3231
</state>
3332
<state name="interp-inside">
3433
<rule pattern="\}">
@@ -43,6 +42,10 @@
4342
<rule pattern="[\[\](),.{}]">
4443
<token type="Punctuation"/>
4544
</rule>
45+
<rule pattern="&#34;">
46+
<token type="LiteralStringDouble"/>
47+
<push state="string"/>
48+
</rule>
4649
<rule pattern="-?[0-9]+">
4750
<token type="LiteralNumber"/>
4851
</rule>
@@ -58,14 +61,14 @@
5861
<rule pattern="\s*(#|//).*\n">
5962
<token type="CommentSingle"/>
6063
</rule>
61-
<rule pattern="([a-zA-Z]\w*)(\s*)(=(?!&gt;))">
64+
<rule pattern="(?!\s*)(variable)(\s*)">
6265
<bygroups>
63-
<token type="NameAttribute"/>
66+
<token type="Name"/>
6467
<token type="Text"/>
6568
<token type="Text"/>
6669
</bygroups>
6770
</rule>
68-
<rule pattern="^\s*(provisioner|variable|resource|provider|module|output|data)\b">
71+
<rule pattern="^(provisioner|variable|resource|provider|module|output|data)(?!\.)\b">
6972
<token type="KeywordReserved"/>
7073
</rule>
7174
<rule pattern="(for|in)\b">
@@ -101,6 +104,9 @@
101104
<rule pattern="(tostring|tonumber|tobool|tolist|tomap|toset|can|try)\b">
102105
<token type="NameBuiltin"/>
103106
</rule>
107+
<rule pattern="(name|x|default|type|description|value)">
108+
<token type="NameAttribute"/>
109+
</rule>
104110
<rule pattern="=(?!&gt;)|\+|-|\*|\/|:|!|%|&gt;|&lt;(?!&lt;)|&gt;=|&lt;=|==|!=|&amp;&amp;|\||\?">
105111
<token type="Operator"/>
106112
</rule>
@@ -110,10 +116,6 @@
110116
<rule pattern="[a-zA-Z]\w*">
111117
<token type="NameOther"/>
112118
</rule>
113-
<rule pattern="&#34;">
114-
<token type="LiteralStringDouble"/>
115-
<push state="string"/>
116-
</rule>
117119
<rule pattern="(?s)(&lt;&lt;-?)(\w+)(\n\s*(?:(?!\2).)*\s*\n\s*)(\2)">
118120
<bygroups>
119121
<token type="Operator"/>
@@ -127,7 +129,7 @@
127129
<rule pattern="(\s*)(&#34;(?:\\\\|\\&#34;|[^&#34;])*&#34;)(\s*)">
128130
<bygroups>
129131
<token type="Text"/>
130-
<token type="NameVariable"/>
132+
<token type="NameAttribute"/>
131133
<token type="Text"/>
132134
</bygroups>
133135
</rule>

‎lexers/testdata/terraform.actual

Lines changed: 51 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,11 @@ resource "something" "nice" {
66
name = "hello"
77
value = var.some_var
88
x = var.some_var > 5 ? var.some_var : 10
9-
109
cidr_blocks = [
1110
for num in [1,2,3]:
1211
cidrhost("10.0.0.0/24", num)
1312
]
13+
1414
str = <<-EOT
1515
hello
1616
world
@@ -19,8 +19,8 @@ resource "something" "nice" {
1919
/*
2020
Multiline comment
2121
*/
22-
# Single comment
2322

23+
# Single comment
2424
dynamic "setting" {
2525
for_each = var.settings
2626
content {
@@ -31,9 +31,57 @@ resource "something" "nice" {
3131
}
3232
}
3333

34-
3534
resource "other" "resource" {
3635
count = 3
3736
name = "resource${count.index+1}"
3837
}
3938

39+
# #988 'Dot in string breaks highlighting in Terraform'
40+
resource "aws_s3_bucket" "this" {
41+
bucket = "my-tf-test-bucket"
42+
website {
43+
index_document = "index.html"
44+
error_document = "error.html"
45+
}
46+
}
47+
48+
# An IAM Policy Document that has a parameter 'variable' that overlaps with the variable class
49+
# variable = below should be variable (text) = (punctuation). variable above should be NameReserved
50+
data "aws_iam_policy_document" "example" {
51+
statement {
52+
sid = "1"
53+
actions = [
54+
"s3:ListAllMyBuckets",
55+
"s3:GetBucketLocation",
56+
]
57+
resources = [
58+
"arn:aws:s3:::*",
59+
]
60+
}
61+
statement {
62+
actions = [
63+
"s3:ListBucket",
64+
]
65+
resources = [
66+
"arn:aws:s3:::${var.s3_bucket_name}",
67+
]
68+
condition {
69+
test = "StringLike"
70+
variable = "s3:prefix"
71+
values = [
72+
"",
73+
"home/",
74+
"home/&{aws:username}/",
75+
]
76+
}
77+
}
78+
statement {
79+
actions = [
80+
"s3:*",
81+
]
82+
resources = [
83+
"arn:aws:s3:::${var.s3_bucket_name}/home/&{aws:username}",
84+
"arn:aws:s3:::${var.s3_bucket_name}/home/&{aws:username}/*",
85+
]
86+
}
87+
}

0 commit comments

Comments
 (0)