stores: test for duplicate keys, reserve keyword (yaml only now) #1203

holiman · 2023-04-21T08:24:57Z

I started looking into sops, which I find to be a really nice idea and implementation. Kudos!
However, when testing some edgecase scenarios, I found some rough corners.

Specifically, it was possible to, when editing the file, add a sops key. This was then encrypted, and when trying to open it later on, I got

go run ./cmd/sops/ mytestfile2.yaml 
Error unmarshalling input yaml: yaml: unmarshal errors:
  line 12: mapping key "sops" already defined at line 7

This is a bit so-so UX wise, since the only way to recover is to remove the sops-line, and then reopen the file with the --ignore-mac enabled, and then save it again.

In general, it seems to me that sops should be strict about validation. In that sense, this PR:

Applies a check for disallowing duplicate keys in YAML
Tests for, but does not disallow, duplicate keys in json.
Applies a check for disallowing reserved keywords in YAML (only sops now).
Remaining small changes come from go fmt.

If you think this is a good idea, the reserved keyword should be applied for other formats too, and I can go ahead and make that change too.

holiman · 2023-04-21T08:30:47Z

Looking at #596 (comment), it seems that perhaps data should be a reserved key too (?)

felixfontein · 2023-04-21T11:01:57Z

This should fix #851.

felixfontein · 2023-09-16T12:21:49Z

Looking at #596 (comment), it seems that perhaps data should be a reserved key too (?)

The README explicitly shows using data for other things in quite a few examples, so I guess it's not a good idea to disallow it globally.

I think #596 should probably be handled differently, like only treating it as binary if there is no other key than data and its value is a string. (I'm not sure which mechanism is used right now. Maybe detection is only based on the file name.)

felixfontein · 2023-09-16T12:34:48Z

I did look a bit into detection; the detection is done purely by filename, not by anything else.

Something that definitely can be improved is error handling in BinaryStore.EmitPlainFile, if data's value is not a string. That would produce a better readable error message instead of a panic (and that error message could mention --output-type so that users qiuckly get an idea what to do). (Edit: #1289 does that.)

stores/json/store_test.go

felixfontein · 2023-09-16T13:29:07Z

stores/json/store.go

@@ -211,7 +211,9 @@ func (store Store) jsonFromTreeBranch(branch sops.TreeBranch) ([]byte, error) {

 func (store Store) treeBranchFromJSON(in []byte) (sops.TreeBranch, error) {
 	dec := json.NewDecoder(bytes.NewReader(in))
-	dec.Token()
+	if _, err := dec.Token(); err != nil {
+		return nil, err


Could you please document what exactly this is doing? I guess this is "Tests for, but does not disallow, duplicate keys in json.", but it is totally unclear to me how this code achieves that.

This change is (as far as I can tell, been a while) not required, it's just a drive-by change that I made: Token() does return an error, and checking+returning it is correct.
Later on, the store.treeBranchFromJSONDecoder(dec) call will internally do the same thing

func (store Store) treeItemFromJSONDecoder(dec *json.Decoder) (sops.TreeItem, error) { var item sops.TreeItem key, err := dec.Token() if err != nil && err != io.EOF { return item, err }

So I guess I figured that we should do it on the very first instance too

Except that this one fails when err == io.EOF. Which I guess is OK. I would add a comment explaining that.

Happy to add a comment, but I'm not sure what to write: I don't quite see why we need to document why we handle an error. Usually, motivation/explanation is required as for why one omits to handle an error.

After dwelling in sources of the JSON module, I now understand why this change is correct.

The main problem I have with this change is that it takes a long time to figure out what it does, and since it is totally unrelated to the commit message it appears in it is very confusing to figure out where it comes from.

It would be a lot better to move this to a separate commit (or even a separate PR) with a clear commit message (something like "Make sure errors returned when extracting the first token are handled").

Also there is another potential error lurking here: the first return value also needs to be checked whether it's a {. Otherwise LoadPlainFile produces "strange" error messages that aren't very helpful:

$ echo '[]' | sops --encrypt --input-type json /dev/stdin Error unmarshalling file: Could not unmarshal input data: Expected JSON object key, got ] of type json.Delim instead $ echo '[}' | sops --encrypt --input-type json /dev/stdin Error unmarshalling file: Could not unmarshal input data: invalid character '}' looking for beginning of value $ echo '["foo","bar"]' | sops --encrypt --input-type json /dev/stdin Error unmarshalling file: Could not unmarshal input data: invalid character '}' after array element $ echo '["foo":"bar"}' | sops --encrypt --input-type json /dev/stdin Error unmarshalling file: Could not unmarshal input data: invalid character ':' after array element

I don't see what difference this makes. As far as I can tell this PR does not change the behaviour against master

This PR:

$ echo '[]' | go run ./cmd/sops/ --encrypt --input-type json /dev/stdin Error unmarshalling file: Could not unmarshal input data: Expected JSON object key, got ] of type json.Delim instead $ echo '[}' | go run ./cmd/sops/ --encrypt --input-type json /dev/stdin Error unmarshalling file: Could not unmarshal input data: invalid character '}' looking for beginning of value $ echo '["foo","bar"]' | go run ./cmd/sops/ --encrypt --input-type json /dev/stdin Error unmarshalling file: Could not unmarshal input data: Expected JSON object key, got ] of type json.Delim instead $ echo '["foo":"bar"}' | go run ./cmd/sops/ --encrypt --input-type json /dev/stdin Error unmarshalling file: Could not unmarshal input data: invalid character ':' after array element

Master:

$ echo '[]' | go run ./cmd/sops/ --encrypt --input-type json /dev/stdin Error unmarshalling file: Could not unmarshal input data: Expected JSON object key, got ] of type json.Delim instead $ echo '[}' | go run ./cmd/sops/ --encrypt --input-type json /dev/stdin Error unmarshalling file: Could not unmarshal input data: invalid character '}' looking for beginning of value $ echo '["foo","bar"]' | go run ./cmd/sops/ --encrypt --input-type json /dev/stdin Error unmarshalling file: Could not unmarshal input data: Expected JSON object key, got ] of type json.Delim instead $ echo '["foo":"bar"}' | go run ./cmd/sops/ --encrypt --input-type json /dev/stdin Error unmarshalling file: Could not unmarshal input data: invalid character ':' after array element

But sure I can remove this change if you want

Also there is another potential error lurking here: the first return value also needs to be checked whether it's a {.

It should not literally check that it's a {, since whitespace is also accepted. I think the error messages are pretty good.

Consider ["foo":"bar"}. If we say error: it must start with {, then I think we are mixing different layers, with different concerns.

First of all, the input needs to be valid json. The value ["foo":"bar"} is not valid json, because a json list cannot contain key-value pairs, so the parse fails. The user will have to fix up his json, there are many tools to validate / lint json.

Secondly, there are app-specific constraints on what is required of the json object: such as that it is a dict, not a list. But those come later, IMO.

If this app starts validating specific characters in the json stream, like "checked whether it's a {.", then it overreaches. And before you know it, you are basically taking on the maintenance of an in-house json parser.

It should not literally check that it's a {, since whitespace is also accepted. I think the error messages are pretty good.

Yes, it should for a literal {, since whitespace has already been processed by the decoder.

First of all, the input needs to be valid json. The value ["foo":"bar"} is not valid json, because a json list cannot contain key-value pairs, so the parse fails. The user will have to fix up his json, there are many tools to validate / lint json.

True, but did you also consider the [] example? Or tried out something like [{}]? The error messages are not really helpful, because the code already mixes the two layers: it parses valid JSON, and at the same time only parses JSON objects.

Secondly, there are app-specific constraints on what is required of the json object: such as that it is a dict, not a list. But those come later, IMO.

The function is called treeBranchFromJSON. It cannot process anything else than a JSON object.

In any case, this discusison IMO shows why this should not part of a totally unrelated commit. This has nothing to do with (not) checking for duplicate keys.

holiman · 2023-09-19T12:12:35Z

Rebased on main, a few tests seem to be failing, not quite sure why

hiddeco · 2023-09-19T12:39:49Z

You did not sign-off your commits, which is required to accept our DCO. Please refer to https://github.com/getsops/sops/pull/1203/checks?check_run_id=16924447836 for more information.

holiman · 2023-09-19T13:14:10Z

You did not sign-off your commits

Fixed (on my branch, seems to be taking github some time to propagate the changes here)

felixfontein · 2023-09-20T19:50:58Z

stores/json/store.go

@@ -211,7 +211,9 @@ func (store Store) jsonFromTreeBranch(branch sops.TreeBranch) ([]byte, error) {

 func (store Store) treeBranchFromJSON(in []byte) (sops.TreeBranch, error) {
 	dec := json.NewDecoder(bytes.NewReader(in))
-	dec.Token()
+	if _, err := dec.Token(); err != nil {
+		return nil, err


Except that this one fails when err == io.EOF. Which I guess is OK. I would add a comment explaining that.

stores/yaml/store_test.go

felixfontein · 2023-09-27T20:32:41Z

Do you want to fix the error handling in a separate PR, or should I do that?

Also you need to sign-off the commit (DCO check). Finally, it's probably better to squash b44f024 and 4bad36e are merged into one commit.

holiman · 2023-09-28T06:59:03Z

I squashed everything now.

Do you want to fix the error handling in a separate PR, or should I do that?

I'd prefer to leave that to you, thanks

felixfontein · 2023-09-28T11:27:43Z

stores/json/store_test.go

+`
+	s := new(Store)
+	_, err := s.LoadPlainFile([]byte(data))
+	assert.Nil(t, err)


At some point, we should extend this test to verify that re-serializing yields the same result.

felixfontein · 2023-09-28T11:28:28Z

stores/yaml/store_test.go

+	assert.NotNil(t, err)
+	assert.Equal(t, `yaml: unmarshal errors:
+  line 3: mapping key "hello" already defined at line 2`, err.Error())
+}


We should also add a test which looks at multi-document YAML streams (with collisions in later documents).

hiddeco · 2023-09-28T13:18:02Z

This is looking good to me, but I think we should release a patch first before moving ahead with changes which force a new minor.

felixfontein · 2023-09-28T20:17:57Z

Do you want to fix the error handling in a separate PR, or should I do that?

I'd prefer to leave that to you, thanks

#1307.

felixfontein · 2023-11-05T13:59:58Z

@hiddeco should we merge this now?

enmanuelmoreira · 2024-01-23T14:56:16Z

Any updates about this? the problem stills happening at version 3.8.1

hiddeco

Needs a rebase to resolve the conflict, but other than this (and a tiny nit) it looks good to me.

Sorry for the wait @holiman, and thanks for your contribution 🍍

stores/yaml/store.go

felixfontein · 2024-06-15T16:12:13Z

@holiman can you rebase and use the constant?

stores/json: use assert stores/yaml: fix failing test (empty data) stores/yaml: use assert in tests unfix error handling and ignore error Signed-off-by: Martin Holst Swende <martin@swende.se>

holiman · 2024-09-27T13:59:57Z

Const fixed + rebased

felixfontein · 2025-02-15T13:16:25Z

@holiman thanks for your contribution!
@hiddeco thanks for reviewing!

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [getsops/sops](https://github.com/getsops/sops) | minor | `v3.9.4` -> `v3.10.2` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>getsops/sops (getsops/sops)</summary> ### [`v3.10.2`](https://github.com/getsops/sops/releases/tag/v3.10.2) [Compare Source](getsops/sops@v3.10.1...v3.10.2) #### Installation To install `sops`, download one of the pre-built binaries provided for your platform from the artifacts attached to this release. For instance, if you are using Linux on an AMD64 architecture: ```shell ### Download the binary curl -LO https://github.com/getsops/sops/releases/download/v3.10.2/sops-v3.10.2.linux.amd64 ### Move the binary in to your PATH mv sops-v3.10.2.linux.amd64 /usr/local/bin/sops ### Make the binary executable chmod +x /usr/local/bin/sops ``` ##### Verify checksums file signature The checksums file provided within the artifacts attached to this release is signed using [Cosign](https://docs.sigstore.dev/cosign/overview/) with GitHub OIDC. To validate the signature of this file, run the following commands: ```shell ### Download the checksums file, certificate and signature curl -LO https://github.com/getsops/sops/releases/download/v3.10.2/sops-v3.10.2.checksums.txt curl -LO https://github.com/getsops/sops/releases/download/v3.10.2/sops-v3.10.2.checksums.pem curl -LO https://github.com/getsops/sops/releases/download/v3.10.2/sops-v3.10.2.checksums.sig ### Verify the checksums file cosign verify-blob sops-v3.10.2.checksums.txt \ --certificate sops-v3.10.2.checksums.pem \ --signature sops-v3.10.2.checksums.sig \ --certificate-identity-regexp=https://github.com/getsops \ --certificate-oidc-issuer=https://token.actions.githubusercontent.com ``` ##### Verify binary integrity To verify the integrity of the downloaded binary, you can utilize the checksums file after having validated its signature: ```shell ### Verify the binary using the checksums file sha256sum -c sops-v3.10.2.checksums.txt --ignore-missing ``` ##### Verify artifact provenance The [SLSA provenance](https://slsa.dev/provenance/v0.2) of the binaries, packages, and SBOMs can be found within the artifacts associated with this release. It is presented through an [in-toto](https://in-toto.io/) link metadata file named `sops-v3.10.2.intoto.jsonl`. To verify the provenance of an artifact, you can utilize the [`slsa-verifier`](https://github.com/slsa-framework/slsa-verifier#artifacts) tool: ```shell ### Download the metadata file curl -LO https://github.com/getsops/sops/releases/download/v3.10.2/sops-v3.10.2.intoto.jsonl ### Verify the provenance of the artifact slsa-verifier verify-artifact <artifact> \ --provenance-path sops-v3.10.2.intoto.jsonl \ --source-uri github.com/getsops/sops \ --source-tag v3.10.2 ``` #### Container Images The `sops` binaries are also available as container images, based on Debian (slim) and Alpine Linux. The Debian-based container images include any dependencies which may be required to make use of certain key services, such as GnuPG, AWS KMS, Azure Key Vault, and Google Cloud KMS. The Alpine-based container images are smaller in size, but do not include these dependencies. These container images are available for the following architectures: `linux/amd64` and `linux/arm64`. ##### GitHub Container Registry - `ghcr.io/getsops/sops:v3.10.2` - `ghcr.io/getsops/sops:v3.10.2-alpine` ##### Quay.io - `quay.io/getsops/sops:v3.10.2` - `quay.io/getsops/sops:v3.10.2-alpine` ##### Verify container image signature The container images are signed using [Cosign](https://docs.sigstore.dev/cosign/overview/) with GitHub OIDC. To validate the signature of an image, run the following command: ```shell cosign verify ghcr.io/getsops/sops:v3.10.2 \ --certificate-identity-regexp=https://github.com/getsops \ --certificate-oidc-issuer=https://token.actions.githubusercontent.com \ -o text ``` ##### Verify container image provenance The container images include [SLSA provenance](https://slsa.dev/provenance/v0.2) attestations. For more information around the verification of this, please refer to the [`slsa-verifier` documentation](https://github.com/slsa-framework/slsa-verifier#containers). #### Software Bill of Materials The Software Bill of Materials (SBOM) for each binary is accessible within the artifacts enclosed with this release. It is presented as an [SPDX](https://spdx.dev/) JSON file, formatted as `<binary>.spdx.sbom.json`. #### What's Changed - build(deps): Bump the go group with 13 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1834 - Use latest 1.24 Go version for release build by [@hiddeco](https://github.com/hiddeco) in getsops/sops#1836 - Remove reserved keyword check from YAML store's `LoadPlainFile()` by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1829 - build(deps): Bump the go group with 9 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1839 - build(deps): Bump github/codeql-action from 3.28.13 to 3.28.15 in the ci group by [@dependabot](https://github.com/dependabot) in getsops/sops#1840 - Release 3.10.2 release by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1841 **Full Changelog**: getsops/sops@v3.10.1...v3.10.2 ### [`v3.10.1`](https://github.com/getsops/sops/releases/tag/v3.10.1) [Compare Source](getsops/sops@v3.10.0...v3.10.1) #### Installation To install `sops`, download one of the pre-built binaries provided for your platform from the artifacts attached to this release. For instance, if you are using Linux on an AMD64 architecture: ```shell ### Download the binary curl -LO https://github.com/getsops/sops/releases/download/v3.10.1/sops-v3.10.1.linux.amd64 ### Move the binary in to your PATH mv sops-v3.10.1.linux.amd64 /usr/local/bin/sops ### Make the binary executable chmod +x /usr/local/bin/sops ``` ##### Verify checksums file signature The checksums file provided within the artifacts attached to this release is signed using [Cosign](https://docs.sigstore.dev/cosign/overview/) with GitHub OIDC. To validate the signature of this file, run the following commands: ```shell ### Download the checksums file, certificate and signature curl -LO https://github.com/getsops/sops/releases/download/v3.10.1/sops-v3.10.1.checksums.txt curl -LO https://github.com/getsops/sops/releases/download/v3.10.1/sops-v3.10.1.checksums.pem curl -LO https://github.com/getsops/sops/releases/download/v3.10.1/sops-v3.10.1.checksums.sig ### Verify the checksums file cosign verify-blob sops-v3.10.1.checksums.txt \ --certificate sops-v3.10.1.checksums.pem \ --signature sops-v3.10.1.checksums.sig \ --certificate-identity-regexp=https://github.com/getsops \ --certificate-oidc-issuer=https://token.actions.githubusercontent.com ``` ##### Verify binary integrity To verify the integrity of the downloaded binary, you can utilize the checksums file after having validated its signature: ```shell ### Verify the binary using the checksums file sha256sum -c sops-v3.10.1.checksums.txt --ignore-missing ``` ##### Verify artifact provenance The [SLSA provenance](https://slsa.dev/provenance/v0.2) of the binaries, packages, and SBOMs can be found within the artifacts associated with this release. It is presented through an [in-toto](https://in-toto.io/) link metadata file named `sops-v3.10.1.intoto.jsonl`. To verify the provenance of an artifact, you can utilize the [`slsa-verifier`](https://github.com/slsa-framework/slsa-verifier#artifacts) tool: ```shell ### Download the metadata file curl -LO https://github.com/getsops/sops/releases/download/v3.10.1/sops-v3.10.1.intoto.jsonl ### Verify the provenance of the artifact slsa-verifier verify-artifact <artifact> \ --provenance-path sops-v3.10.1.intoto.jsonl \ --source-uri github.com/getsops/sops \ --source-tag v3.10.1 ``` #### Container Images The `sops` binaries are also available as container images, based on Debian (slim) and Alpine Linux. The Debian-based container images include any dependencies which may be required to make use of certain key services, such as GnuPG, AWS KMS, Azure Key Vault, and Google Cloud KMS. The Alpine-based container images are smaller in size, but do not include these dependencies. These container images are available for the following architectures: `linux/amd64` and `linux/arm64`. ##### GitHub Container Registry - `ghcr.io/getsops/sops:v3.10.1` - `ghcr.io/getsops/sops:v3.10.1-alpine` ##### Quay.io - `quay.io/getsops/sops:v3.10.1` - `quay.io/getsops/sops:v3.10.1-alpine` ##### Verify container image signature The container images are signed using [Cosign](https://docs.sigstore.dev/cosign/overview/) with GitHub OIDC. To validate the signature of an image, run the following command: ```shell cosign verify ghcr.io/getsops/sops:v3.10.1 \ --certificate-identity-regexp=https://github.com/getsops \ --certificate-oidc-issuer=https://token.actions.githubusercontent.com \ -o text ``` ##### Verify container image provenance The container images include [SLSA provenance](https://slsa.dev/provenance/v0.2) attestations. For more information around the verification of this, please refer to the [`slsa-verifier` documentation](https://github.com/slsa-framework/slsa-verifier#containers). #### Software Bill of Materials The Software Bill of Materials (SBOM) for each binary is accessible within the artifacts enclosed with this release. It is presented as an [SPDX](https://spdx.dev/) JSON file, formatted as `<binary>.spdx.sbom.json`. #### What's Changed - build(deps): Bump the ci group with 2 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1826 - Release 3.10.1 by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1827 **Full Changelog**: getsops/sops@v3.10.0...v3.10.1 ### [`v3.10.0`](https://github.com/getsops/sops/releases/tag/v3.10.0) [Compare Source](getsops/sops@v3.9.4...v3.10.0) #### Installation To install `sops`, download one of the pre-built binaries provided for your platform from the artifacts attached to this release. For instance, if you are using Linux on an AMD64 architecture: ```shell ### Download the binary curl -LO https://github.com/getsops/sops/releases/download/v3.10.0/sops-v3.10.0.linux.amd64 ### Move the binary in to your PATH mv sops-v3.10.0.linux.amd64 /usr/local/bin/sops ### Make the binary executable chmod +x /usr/local/bin/sops ``` ##### Verify checksums file signature The checksums file provided within the artifacts attached to this release is signed using [Cosign](https://docs.sigstore.dev/cosign/overview/) with GitHub OIDC. To validate the signature of this file, run the following commands: ```shell ### Download the checksums file, certificate and signature curl -LO https://github.com/getsops/sops/releases/download/v3.10.0/sops-v3.10.0.checksums.txt curl -LO https://github.com/getsops/sops/releases/download/v3.10.0/sops-v3.10.0.checksums.pem curl -LO https://github.com/getsops/sops/releases/download/v3.10.0/sops-v3.10.0.checksums.sig ### Verify the checksums file cosign verify-blob sops-v3.10.0.checksums.txt \ --certificate sops-v3.10.0.checksums.pem \ --signature sops-v3.10.0.checksums.sig \ --certificate-identity-regexp=https://github.com/getsops \ --certificate-oidc-issuer=https://token.actions.githubusercontent.com ``` ##### Verify binary integrity To verify the integrity of the downloaded binary, you can utilize the checksums file after having validated its signature: ```shell ### Verify the binary using the checksums file sha256sum -c sops-v3.10.0.checksums.txt --ignore-missing ``` ##### Verify artifact provenance The [SLSA provenance](https://slsa.dev/provenance/v0.2) of the binaries, packages, and SBOMs can be found within the artifacts associated with this release. It is presented through an [in-toto](https://in-toto.io/) link metadata file named `sops-v3.10.0.intoto.jsonl`. To verify the provenance of an artifact, you can utilize the [`slsa-verifier`](https://github.com/slsa-framework/slsa-verifier#artifacts) tool: ```shell ### Download the metadata file curl -LO https://github.com/getsops/sops/releases/download/v3.10.0/sops-v3.10.0.intoto.jsonl ### Verify the provenance of the artifact slsa-verifier verify-artifact <artifact> \ --provenance-path sops-v3.10.0.intoto.jsonl \ --source-uri github.com/getsops/sops \ --source-tag v3.10.0 ``` #### Container Images The `sops` binaries are also available as container images, based on Debian (slim) and Alpine Linux. The Debian-based container images include any dependencies which may be required to make use of certain key services, such as GnuPG, AWS KMS, Azure Key Vault, and Google Cloud KMS. The Alpine-based container images are smaller in size, but do not include these dependencies. These container images are available for the following architectures: `linux/amd64` and `linux/arm64`. ##### GitHub Container Registry - `ghcr.io/getsops/sops:v3.10.0` - `ghcr.io/getsops/sops:v3.10.0-alpine` ##### Quay.io - `quay.io/getsops/sops:v3.10.0` - `quay.io/getsops/sops:v3.10.0-alpine` ##### Verify container image signature The container images are signed using [Cosign](https://docs.sigstore.dev/cosign/overview/) with GitHub OIDC. To validate the signature of an image, run the following command: ```shell cosign verify ghcr.io/getsops/sops:v3.10.0 \ --certificate-identity-regexp=https://github.com/getsops \ --certificate-oidc-issuer=https://token.actions.githubusercontent.com \ -o text ``` ##### Verify container image provenance The container images include [SLSA provenance](https://slsa.dev/provenance/v0.2) attestations. For more information around the verification of this, please refer to the [`slsa-verifier` documentation](https://github.com/slsa-framework/slsa-verifier#containers). #### Software Bill of Materials The Software Bill of Materials (SBOM) for each binary is accessible within the artifacts enclosed with this release. It is presented as an [SPDX](https://spdx.dev/) JSON file, formatted as `<binary>.spdx.sbom.json`. #### What's Changed - build(deps): Bump alpine from 3.18 to 3.21 in /.release in the docker group across 1 directory by [@dependabot](https://github.com/dependabot) in getsops/sops#1700 - Convert changelog to MarkDown by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1741 - build(deps): Bump the go group with 7 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1743 - build(deps): Bump the go group with 9 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1745 - build(deps): Bump the rust group in /functional-tests with 2 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1744 - build(deps): Bump github/codeql-action from 3.28.5 to 3.28.8 in the ci group by [@dependabot](https://github.com/dependabot) in getsops/sops#1746 - build(deps): Bump the go group with 12 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1751 - build(deps): Bump the ci group with 4 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1750 - Add --input-type option for filestatus subcommand by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1601 - Use SOPS_EDITOR before EDITOR by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1611 - Allow users to disable version check via environment variable by [@nicklasfrahm](https://github.com/nicklasfrahm) in getsops/sops#1684 - add duplicate section support to ini store by [@reindlt](https://github.com/reindlt) in getsops/sops#1452 - stores: test for duplicate keys, reserve keyword (yaml only now) by [@holiman](https://github.com/holiman) in getsops/sops#1203 - Add same process option for exec-env by [@ricmatsui](https://github.com/ricmatsui) in getsops/sops#880 - outputs: add trailing newline at the end of JSON files by [@duthils](https://github.com/duthils) in getsops/sops#1476 - set subcommand: add --idempotent flag that will not write the file if no change happened by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1754 - Encrypt and decrypt time.Time objects by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1759 - build(deps): Bump the go group with 8 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1763 - build(deps): Bump tempfile from 3.16.0 to 3.17.0 in /functional-tests in the rust group by [@dependabot](https://github.com/dependabot) in getsops/sops#1762 - build(deps): Bump goreleaser/goreleaser-action from 6.1.0 to 6.2.1 in the ci group by [@dependabot](https://github.com/dependabot) in getsops/sops#1761 - fix(docs): typo in README.rst by [@Paredev](https://github.com/Paredev) in getsops/sops#1765 - Add SSH support for age by [@haoqixu](https://github.com/haoqixu) in getsops/sops#1692 - make sure that tests do not pick keys.txt from user's HOME dir by [@tomaszduda23](https://github.com/tomaszduda23) in getsops/sops#1766 - support for age identity with passphrase by [@tomaszduda23](https://github.com/tomaszduda23) in getsops/sops#1400 - build(deps): Bump the rust group in /functional-tests with 4 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1768 - build(deps): Bump the go group with 12 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1769 - build(deps): Bump the ci group with 4 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1770 - build(deps): Bump github.com/go-jose/go-jose/v4 from 4.0.4 to 4.0.5 by [@dependabot](https://github.com/dependabot) in getsops/sops#1773 - feat: add age plugin support by [@brianmcgee](https://github.com/brianmcgee) in getsops/sops#1641 - Check GnuPG decryption result for non-empty size by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1776 - vendored age code: consolidate passphrase reading functionality by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1775 - Allow to encrypt and decrypt from stdin by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1690 - build(deps): Bump the go group with 11 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1784 - build(deps): Bump the ci group with 5 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1782 - build(deps): Bump serde_json from 1.0.139 to 1.0.140 in /functional-tests in the rust group; bump Rust to 1.85.0 by [@dependabot](https://github.com/dependabot) in getsops/sops#1783 - build(deps): Bump the go group with 13 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1797 - build(deps): Bump the rust group in /functional-tests with 3 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1796 - build(deps): Bump github/codeql-action from 3.28.10 to 3.28.11 in the ci group by [@dependabot](https://github.com/dependabot) in getsops/sops#1795 - build(deps): Bump the go group with 6 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1802 - build(deps): Bump tempfile from 3.18.0 to 3.19.0 in /functional-tests in the rust group by [@dependabot](https://github.com/dependabot) in getsops/sops#1800 - build(deps): Bump docker/login-action from 3.3.0 to 3.4.0 in the ci group by [@dependabot](https://github.com/dependabot) in getsops/sops#1801 - build(deps): Bump github.com/golang-jwt/jwt/v5 from 5.2.1 to 5.2.2 by [@dependabot](https://github.com/dependabot) in getsops/sops#1806 - goreleaser: add windows arm64 by [@duthils](https://github.com/duthils) in getsops/sops#1791 - Add a way to set sops config location via envvar by [@sledzikowy](https://github.com/sledzikowy) in getsops/sops#1701 - build(deps): Bump the go group with 3 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1809 - build(deps): Bump the ci group with 5 updates by [@dependabot](https://github.com/dependabot) in getsops/sops#1808 - build(deps): Bump tempfile from 3.19.0 to 3.19.1 in /functional-tests in the rust group by [@dependabot](https://github.com/dependabot) in getsops/sops#1807 - Support --config option in publish subcommand by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1779 - Fix caching of Metadata.DataKey by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1781 - Lint by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1780 - updatekeys subcommand: rename GroupQuorum to ShamirThreshold by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1631 - If --filename-override is specified, convert it to an absolute path same as regular filenames by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1793 - Add support for `oauth2.TokenSource` in GCP KMS by [@matheuscscp](https://github.com/matheuscscp) in getsops/sops#1794 - Added SOPS_AGE_KEY_CMD option to age, fixes [#1323](getsops/sops#1323) by [@danilobuerger](https://github.com/danilobuerger) in getsops/sops#1811 - chore: omitempty metadata to reduce the size of stored config by [@cgetzen](https://github.com/cgetzen) in getsops/sops#1571 - Improve Shamir Secret Sharing code by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1813 - Update all dependencies by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1814 - Run 'gofmt -w' on all .go files by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1817 - Add option to explicitly check for the latest version; deprecate current default of always doing that unless disabled by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1816 - Build using Go 1.24 and drop support for 1.22 by [@hiddeco](https://github.com/hiddeco) in getsops/sops#1819 - Support `GOOGLE_OAUTH_ACCESS_TOKEN` for Google Cloud Platform by [@marensofier](https://github.com/marensofier) in getsops/sops#1578 - Warn about `.sops.yml` files found while searching for `.sops.yaml` by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1820 - Release 3.10.0 by [@felixfontein](https://github.com/felixfontein) in getsops/sops#1815 - Added support for multiarch windows release binaries by [@sabre1041](https://github.com/sabre1041) in getsops/sops#1823 #### New Contributors - [@reindlt](https://github.com/reindlt) made their first contribution in getsops/sops#1452 - [@ricmatsui](https://github.com/ricmatsui) made their first contribution in getsops/sops#880 - [@Paredev](https://github.com/Paredev) made their first contribution in getsops/sops#1765 - [@haoqixu](https://github.com/haoqixu) made their first contribution in getsops/sops#1692 - [@brianmcgee](https://github.com/brianmcgee) made their first contribution in getsops/sops#1641 - [@sledzikowy](https://github.com/sledzikowy) made their first contribution in getsops/sops#1701 - [@matheuscscp](https://github.com/matheuscscp) made their first contribution in getsops/sops#1794 - [@danilobuerger](https://github.com/danilobuerger) made their first contribution in getsops/sops#1811 - [@cgetzen](https://github.com/cgetzen) made their first contribution in getsops/sops#1571 - [@marensofier](https://github.com/marensofier) made their first contribution in getsops/sops#1578 - [@sabre1041](https://github.com/sabre1041) made their first contribution in getsops/sops#1823 **Full Changelog**: getsops/sops@v3.9.4...v3.10.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

holiman changed the title ~~stores: test for duplicate keys, reseve keyword (yaml only now)~~ stores: test for duplicate keys, reserve keyword (yaml only now) Apr 21, 2023

felixfontein mentioned this pull request Apr 21, 2023

Version 3.7.0 and duplicated keys in a YAML file #851

Open

hiddeco added this to the v3.9.0 milestone Jul 3, 2023

hiddeco changed the base branch from master to main July 6, 2023 20:57

felixfontein reviewed Sep 16, 2023

View reviewed changes

holiman force-pushed the fixes branch from faa520b to 32f1320 Compare September 19, 2023 12:11

holiman force-pushed the fixes branch from 32f1320 to 377b6b4 Compare September 19, 2023 13:41

hiddeco self-requested a review September 20, 2023 11:46

felixfontein reviewed Sep 20, 2023

View reviewed changes

holiman force-pushed the fixes branch from ffd644a to 8055bae Compare September 21, 2023 07:10

holiman force-pushed the fixes branch from b44f024 to 627bb97 Compare September 28, 2023 06:57

felixfontein approved these changes Sep 28, 2023

View reviewed changes

felixfontein mentioned this pull request Sep 28, 2023

Handle return values of dec.Token() to improve error messages #1307

Merged

hiddeco approved these changes Feb 9, 2024

View reviewed changes

stores/yaml/store.go Outdated Show resolved Hide resolved

felixfontein modified the milestones: v3.9.0, 3.10.0 Jun 26, 2024

stores: test for duplicate keys, reseve keyword (yaml only now)

adad27e

stores/json: use assert stores/yaml: fix failing test (empty data) stores/yaml: use assert in tests unfix error handling and ignore error Signed-off-by: Martin Holst Swende <martin@swende.se>

holiman force-pushed the fixes branch from 627bb97 to adad27e Compare September 27, 2024 13:59

felixfontein added stores/yaml area/stores enhancement labels Dec 24, 2024

Merge branch 'main' into fixes

233deec

felixfontein merged commit 2110731 into getsops:main Feb 15, 2025
15 checks passed

felixfontein mentioned this pull request Mar 31, 2025

Remove reserved keyword check from YAML store's LoadPlainFile() #1829

Merged

stores: test for duplicate keys, reserve keyword (yaml only now) #1203

stores: test for duplicate keys, reserve keyword (yaml only now) #1203

Uh oh!

Conversation

holiman commented Apr 21, 2023

Uh oh!

holiman commented Apr 21, 2023

Uh oh!

felixfontein commented Apr 21, 2023

Uh oh!

felixfontein commented Sep 16, 2023

Uh oh!

felixfontein commented Sep 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

holiman Sep 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

holiman commented Sep 19, 2023

Uh oh!

hiddeco commented Sep 19, 2023

Uh oh!

holiman commented Sep 19, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

felixfontein commented Sep 27, 2023

Uh oh!

holiman commented Sep 28, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hiddeco commented Sep 28, 2023

Uh oh!

felixfontein commented Sep 28, 2023

Uh oh!

felixfontein commented Nov 5, 2023

Uh oh!

enmanuelmoreira commented Jan 23, 2024

Uh oh!

hiddeco left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

felixfontein commented Jun 15, 2024

Uh oh!

holiman commented Sep 27, 2024

Uh oh!

Uh oh!

felixfontein commented Feb 15, 2025

Uh oh!

Uh oh!

felixfontein commented Sep 16, 2023 •

edited

Loading

holiman Sep 22, 2023 •

edited

Loading