htmx should preserve form element order when encoding form data for submitting via GET or POST #2703
Description
htmx currently changes the order of input elements when encoding form data before submitting a form via POST.
To Recreate
Given the following HTML:
<!DOCTYPE html>
<html>
<head>
<script src="https://unpkg.com/htmx.org@2.0.0" integrity="sha384-wS5l5IKJBvK6sPTKa2WZ1js3d947pvWXbPJ1OmWfEuxLgeHcEbjUUA5i9V5ZkpCw" crossorigin="anonymous"></script>
</head>
<body hx-boost="true">
<form id="test" action="/team/13/edit" method="POST">
<p>Team:</p>
<p><input type="text" name="team[id]" value="13"></p>
<p><input type="text" name="team[name]" value="Bad News Bears"></p>
<p>Players:</p>
<div>
<input type="text" name="team[players][][name]" value="Amanda Whurlitzer">
<input type="text" name="team[players][][position]" value="Pitcher">
</div>
<div>
<input type="text" name="team[players][][name]" value="Tanner Boyle">
<input type="text" name="team[players][][position]" value="Short Stop">
</div>
<div>
<input type="text" name="team[players][][name]" value="Ahmad Abdul-Rahim">
<input type="text" name="team[players][][position]" value="Center Field">
</div>
<p><button type="submit">Save</button>
</form>
</body>
</html>Expected Behavior
When submitted, the data for this form should be encoded in same order as the inputs appear in the document, more or less as follows:
unescape(new URLSearchParams(new FormData(document.forms[0])).toString())
team[id]=13&
team[name]=Bad+News+Bears&
team[players][][name]=Amanda+Whurlitzer&
team[players][][position]=Pitcher&
team[players][][name]=Tanner+Boyle&
team[players][][position]=Short+Stop&
team[players][][name]=Ahmad+Abdul-Rahim&
team[players][][position]=Center+FieldActual Behavior
Instead of the above, htmx sorts the inputs so that name and position are grouped separately.
htmx.values(htmx.find('#test'))Shows the following:
0: "team[id]" → "13"
1: "team[name]" → "Bad News Bears"
2: "team[players][][name]" → "Amanda Whurlitzer"
3: "team[players][][name]" → "Tanner Boyle"
4: "team[players][][name]" → "Ahmad Abdul-Rahim"
5: "team[players][][position]" → "Pitcher"
6: "team[players][][position]" → "Short Stop"
7: "team[players][][position]" → "Center Field"The Problem
- This is unexpected behavior.
- This appears to be contrary to the HTML specification.
- Server-side code should be entitled to rely on the order of inputs it created.
This is unexpected behavior
The htmx docs state:
A feature of hx-boost is that it degrades gracefully if javascript is not enabled: the links and forms continue to work, they simply don’t use ajax requests. This is known as Progressive Enhancement, and it allows a wider audience to use your site’s functionality.
Browsers encode form data in the same order as the inputs in the document. Therefore, the "progressive enhancement" is not faithful if form data returned by the browser is encoded differently with and without hx-boost.
This appears to be contrary to the HTML specification
The HTML specification states:
The order of parts must be the same as the order of fields in entry list. Multiple entries with the same name must be treated as distinct fields.
The specification also includes an algorithm for compiling the entry list. This algorithm guarantees that form data be encoded in the order of the inputs in the document:
For each element field in controls, in tree order . . .
Server-side code should be entitled to rely on the order of inputs it created
If the server provides the HTML for a form, it should be entitled to rely on the order of input controls to dictate the structure of the data returned by the browser. Long-standing conventions have arisen for converting HTTP urlencoded form data to language-native data structures. Changing the order of the form data breaks these conventions.
For example, in Ruby, the Rack gem uses the name and order of form data keys to construct native data structures like arrays and hashes. A good discussion of this can be found here. The current behavior breaks this convention for all Rack apps.
If we follow the instructions in the above-linked article on Rack params parsing, we can see that Rack parses the unaltered parameters from the browser like so, creating a nested collection of child objects:
~ irb
> require "rack"
=> true
> def p(params)
> Rack::Utils.parse_nested_query(params)
> end
=> :p
> p('team[id]=13&team[name]=Bad+News+Bears&team[players][][name]=Amanda+Whurlitzer&team[players][]position]=Pitcher&team[players][][name]=Tanner+Boyle&team[players][][position]=Short+Stop&team[players][]name]=Ahmad+Abdul-Rahim&team[players][][position]=Center+Field')
=>
{"team"=>
{"id"=>"13",
"name"=>"Bad News Bears",
"players"=>
[{"name"=>"Amanda Whurlitzer", "position"=>"Pitcher"},
{"name"=>"Tanner Boyle", "position"=>"Short Stop"},
{"name"=>"Ahmad Abdul-Rahim", "position"=>"Center Field"}]}}But with hx-boost, Rack is confused and our parameters are hopelessly garbled:
p('team[id]=13&team[name]=Bad+News+Bears&team[players][][name]=Amanda+Whurlitzer&team[players][]name]=Tanner+Boyle&team[players][][name]=Ahmad+Abdul-Rahim&team[players][][position]=Pitcher&team[players][][position]=Short+Stop&team[players][][position]=Center+Field')
=>
{"team"=>{"id"=>"13"},
"team"=>
{"name"=>"Bad News Bears",
"players"=>
[{"name"=>"Amanda Whurlitzer"},
{"name"=>"Tanner Boyle"},
{"name"=>"Ahmad Abdul-Rahim", "position"=>"Pitcher"},
{"position"=>"Short Stop"},
{"position"=>"Center Field"}]}}There is no easy way to fix this on the server side. Even if we bypass Rack and implement parameter parsing ourselves, it is impossible to reconstruct the original data due to the possibility of missing keys within the child collections. The only practical solution is to turn off hx-boost for the form. :'(
Similar Issues & Related Info
- A similar issue was raised last year with regard to query-string encoded data on
GETrequests. The issue appears to have been fixed forGETrequests, but not forPOSTform submissions. - This issue was also raised at least once on the htmx Discord, without resolution.
- Here is a StackOverflow discussion on this general issue.
Thank You
Thank you for htmx! And thank you for reading all of this! 😄