This is meant to be a tracking ("epic") issue that gathers all of the deficiencies and pain points in the current k6 configuration. Individual issues, whenever possible, should likely be tackled in their own separate issues and pull requests, unless at some point we decide it'd just be easier to overhaul everything remaining at once...

I'll start with a description of how the configuration works today AFAIK, and its shortcomings. Since things are quite complicated, I'll likely miss some aspects of it, so we should update the description or add comments in the issue when we notice something else amiss. We also have to decide how we want the configuration to behave in the future, and make the changes so we get from here to there... Maybe not immediately, but the current situation is untenable in the long run.

I'll start with the current state. Generally, users can configure k6 with a mix of 4 different option sources - CLI flags, environment variables, exported script options and global JSON config. They are hierarchical, so as described here , CLI flags can override environment variables, which can override script options, which can override JSON config options, which can override default values .

There are various issues and complications with the actual implementation though:

• Adding a new option often necessitates changes in at least 4 different places in the k6 code (here, here, here, here, and where we actually use it...)
• Setting default values for options is very illogical, it mostly happens the CLI flags, like this... unless it doesn't, like here or here...
• Incompatibilities in the libraries we use (envconfig especially, due to this and this issues)
• Null-able type incompatibilities, especially with serialization and deserialization - we use a mix of this null library and our own types, but occasional incompatibility issues pop up and the whole thing isn't very well tested
• Different value formats in different config sources (for example, stages are configured quite differently with the JS options and the CLI flags)
• Some options have no CLI flag (example + all collector/output options) or have only CLI and env vars options (i.e. no JS ones, even when it makes sense, like these)or can only be enabled with a flag (all outputs), etc.
• Different k6 subcommands have different option sets (example)
• Serialization and deserialization issues in general, especially when we need the result to be human-readable (eg. in the HAR converter we want to emit some consolidated options in a human-readable format, so without all of the null values)
• We can actually use environment variables (tier 2) to configure in-script options (which are on the lower tier 3!) - that's cool and very often useful, but with the current architecture it's also a bit complicated
• Validation of different options is very piecemeal and quite insufficient (case in point)
• Various other individual issues like this
• etc., etc... (I'm sure that I'm missing a lot of corner cases)

Also, a lot of the issues with the current config implementation are only catch-able during runtime. That's party a result of Go's relatively weak type system, but I'm sure it could be improved upon. And on top of the weak compile-time assurances, the behavior (default value setting, overriding, serialization, de-serialization, etc.) of different config entity types is difficult to test individually, since they're part of huge structs that also contain part of the behavior (esp. overriding and default value management). That makes the current implementation quite error prone, and unfortunately, the k6 users are usually the first ones to notice that something doesn't work correctly 😒

Back on the topic of specific configuration issues, by far the biggest problem are the options for collectors (i.e. outputs), and by far the biggest problem from them is actually the Load Impact cloud options... They break the aforementioned nice "4 config tiers" picture in so many ways it's actually difficult to know if I can list them all or consider them corner cases...

First, the general output issues are mostly described in this github issue: #587.
In short:

• Outputs can be defined only via the CLI flags (i.e. k6 run -o cloud) and can be 0, 1, or more than 1 output, even from the same type
• Some of them can be configured via CLI flags (eg. k6 run -o json=results.json -o influxdb=http://localhost:8086/test script.js), while others, like the cloud one, cannot...
• All of them can also be configured (but not initialized!) via environment variables or the global JSON config (though as mentioned in the issue above, not when there are multiple outputs of the same type)
• All outputs except the cloud one cannot be configured from the exported script options. The collectors configuration is outside of the lib.Options struct we use to store all of the JS-exported options, so it's only applicable for the JSON config, not the exported script options...

All of the above issues apply to the cloud options as well, but as mentioned, they're "special" for at least two other reasons...

The first reason is that we actually reuse the same configuration for both cloud execution ( k6 cloud script.js ) and for outputting stuff to the cloud ingest ( k6 run -o cloud script.js ). That makes some sense, since both operations have common options (token, projectID and name at the very least), but at the same time it makes everything super tightly-coupled, complicated and corner-case prone.

The second issue is that we actually can configure the cloud collector from the exported script options. Only, we don't do it in some sane uniform way, instead we have that special snowflake ext.loadimpact, where we can put stuff that affects both the cloud collector and the cloud ingest output... And since need to bundle the script options in the tar archive's metadata.json when we run k6 cloud, and that metadata.json file contains only the lib.Options instead of the full config map, k6 actually needs to populate stuff in that special ext.loadimpact snowflake as well as read stuff from it. So on top of everything, it's also kind of outside of the hierarchy of all of the other config sources. And on top of that, ext.loadimpact also contains stuff that k6 doesn't know about (like distribution) 😅

Add to the whole mess that the commands k6 run script.js, k6 run archive.tar, k6 archive script.js, k6 cloud script.js, and k6 cloud archive.tar build their configuration and behave in subtly different ways, and we have a great recipe for endless corner cases and annoying bugs...

So yeah, the k6 configuration is huge, multi-faceted pile of... technical debt 🙂 Any ideas and suggestions are quite welcome at this point, or even reports of other configuration issues that weren't mentioned above 😄