Fixes

• [virtual-machine] Fix the regression in VM update hook introduced in #1169 by targeting the correct API resource and avoiding conflicts with KubeVirt resources. (@kvaps in #1376, backported in #1377)
• [cozy-lib] Add the missing template cozy-lib.resources.flatten. (@kvaps in #1372, backported in #1375)
• [platform] Fix a boolean override bug in Helm merge. ConfigMap values now correctly take precedence over bundle defaults. (@dyudin0821 in #1385, backported in #1388)
• [seaweedfs] Resolve connectivity issues in SeaweedFS. Increase Nginx ingress timeouts for SeaweedFS S3 endpoint. (@kvaps in #1386, backported in #1390)
• [dx] Remove the BUILDER and PLATFORM autodetect logic in Makefiles. (@kvaps in #1391, backported in #1392)

Full Changelog: v0.35.3...v0.35.4

• Updated SeaweedFS with server-side encryption support
• New images instead of deprecated bitnami
• Fix for specifying tenant quotas
• Fix for vm-update hook
• Enhanced rules for vm-routing to expose them with external IP

What's Changed

• fix seaweedfs s3 liveness probe scheme by @IvanHunters in #1368
• [docs] Changelogs for release series v0.35.x by @NickVolynkin in #1347
• [virtual-machine] Use external IP for egress traffic for PortList method too by @kvaps in #1349
• Update SeaweedFS to v3.97 to enable SSE support by @kvaps in #1373
• Fix missing cozy-lib.resources.flatten template by @kvaps in #1372
• Get rid of bitnami images by @kvaps in #1374
• [virtual-machine] Fix vm update hook by @kvaps in #1376
• fix race conditions for seaweedfs and fix tests preparing by @IvanHunters in #1371
• Release v0.36.0-alpha.2 by @cozystack-bot in #1370

Full Changelog: v0.36.0-alpha.1...v0.36.0-alpha.2

Fixes

• Add a liveness check for the SeaweedFS S3 endpoint to improve health monitoring and enable automatic recovery. (@IvanHunters in #1368)

Full Changelog: v0.35.2...v0.35.3

What's Changed

• [tenant-k8s] change coredns by @klinch0 in #1362
• [kube-ovn] Update and patch Kube-OVN by @lllamnyp in #1363
• Release v0.36.0-alpha.1 by @cozystack-bot in #1365

Full Changelog: v0.35.2...v0.36.0-alpha.1

Features and Improvements

• [talos] Add LLDPD (ghcr.io/siderolabs/lldpd) as a built-in system extension, enabling LLDP-based neighbor discovery out of the box. (@lllamnyp in #1351 and #1360)

Fixes

• [cozystack-api] Sanitize the OpenAPI v2 schema. (@kvaps in #1353)
• [seaweedfs] Fix a problem where S3 gateway would be moved to an external pod, resulting in authentication failure. (@kvaps in #1361)

Dependencies

• Update LINSTOR to v1.31.3. (@kvaps in #1358)
• Update SeaweedFS to v3.96. (@kvaps in #1361)

Full Changelog: v0.35.1...v0.35.2

Fixes

• [cozy-lib] Fix malformed retrieval of cozyConfig in the cozy-lib template. (@lllamnyp in #1348)

Full Changelog: v0.35.0...v0.35.1

Feature Highlights

External Application Sources in Cozystack

Cozystack now supports adding external application packages to the platform's application catalog. Platform administrators can include custom or third-party applications alongside built-in ones, using the Cozystack API.

Adding an application requires making an application package, similar to the ones included in Cozystack under packages/apps. Using external packages is enabled by a new CustomResourceDefinition (CRD) called CozystackResourceDefinition and a corresponding controller (reconciler) that watches for these resources.

Add your own managed application using the documentation and an example at github.com/cozystack/external-apps-example.

Cozystack API Improvements

This release brings significant improvements to the OpenAPI specs for all managed applications in Cozystack, including databases, tenant Kubernetes, virtual machines, monitoring, and others. These changes include more precise type definitions for fields that were previously defined only as generic objects, and many fields now have value constraints. Now many possible misconfigurations are detected immediately upon API request, and not later, with a failed deployment.

The Cozystack API now also displays default values for the application resources. Most other fields now have sane default values when such values are possible.

All these changes pave the road for the new Cozystack UI, which is currently under development.

Hetzner RobotLB Support

MetalLB, the default load balancer included in Cozystack, is built for bare metal and self-hosted VMs, but is not supported on most cloud providers. For example, Hetzner provides its own RobotLB service, which Cozystack now supports as an optional component.

Read the updated guide on deploying Cozystack on Hetzner.com to learn more and deploy your own Cozystack cluster on Hetzner.

S3 Service: Dedicated Clusters and Monitoring

You can now deploy dedicated Cozystack clusters to run the S3 service, powered by SeaweedFS. Thanks to the support for integration with remote filer endpoints, you can connect your primary Cozystack cluster to use S3 storage in a dedicated cluster.

For security, platform administrators can now configure the SeaweedFS application with a list of IP addresses or CIDR ranges that are allowed to access the filer service.

SeaweedFS has also been integrated into the monitoring stack and now has its own Grafana dashboard. Together, these enhancements help Cozystack users build a more reliable, scalable, and observable S3 service.

ClickHouse Keeper

The ClickHouse application now includes a ClickHouse Keeper service to improve cluster reliability and availability. This component is deployed by default with every ClickHouse cluster.

Learn more in the ClickHouse configuration reference.

Major Features and Improvements

• [platform] Enable using external application packages by adding a CozystackResourceDefinition reconciler. Read the documentation on adding external applications to Cozystack to learn more. (@klinch0 in #1313)
• [cozystack-api, apps] Add default values, clear type definitions, value constraints and other improvements to the OpenAPI specs and READMEs by migrating to cozyvalue-gen. (@kvaps and @NickVolynkin in #1216, #1314, #1316, #1321, and #1333)
• [cozystack-api] Show default values from the OpenAPI spec in the application resources. (@kvaps in #1241)
• [cozystack-api] Provide an API for administrators to define custom managed applications alongside existing managed apps. (@Klinch in #1230)
• [robotlb] Introduce the Hetzner RobotLB balancer. (@IvanHunters and @gwynbleidd2106 in #1233)
• [platform, robotlb] Autodetect if node ports should be assigned to load balancer services. (@lllamnyp in #1271)
• [seaweedfs] Enable integration with remote filer endpoints by adding new Client topology. (@kvaps in #1239)
• [seaweedfs] Add support for whitelisting and exporting via nginx-ingress. Update cosi-driver. (@kvaps in #1277)
• [monitoring, seaweedfs] Add monitoring and Grafana dashboard for SeaweedFS. (@IvanHunters in #1285)
• [clickhouse] Add the ClickHouse Keeper component. (@klinch0 in #1298 and #1320)

Security

• [keycloak] Store administrative passwords in the management cluster's secrets. (@IvanHunters in #1286)
• [keycloak] Update Keycloak client redirect URI to use HTTPS instead of HTTP. Enable cookie-secure. (@klinch0 in #1287)

Fixes

• [platform] Introduce a fixed 2-second delay at the start of reconciliation for system and tenant Helm operations. (@klinch0 in #1343)
• [kubernetes] Add dependency for snapshot CRD and migration to the latest version. (@kvaps in #1275)
• [kubernetes] Fix regression in volumesnapshotclass installation from #1203. (@kvaps in #1238)
• [kubernetes] Resolve problems with pod names exceeding allowed length by shortening the name of volume snapshot CRD from *-volumesnapshot-crd-for-tenant-k8s to *-vsnap-crd. To apply this change, update each affected tenant Kubernetes cluster after updating Cozystack. (@klinch0 in #1284)
• [kubernetes] Disable VPA for VPA in tenant Kubernetes clusters. Tenant clusters have no need for this feature, and it was not designed to work in a tenant cluster, but was enabled by mistake. (@lllamnyp in #1301 and #1318)
• [kamaji] Fix broken migration jobs originating from missing environment variables in the in-tree build. (@lllamnyp in #1338)
• [etcd] Fix the topologySpreadConstraints for etcd. (@klinch0 in #1331)
• [tenant] Fix tenant network policy to allow traffic to additional tenant-related services across namespace hierarchies. (@klinch0 in #1232)
• [tenant, monitoring] Improve the reliability of tenant monitoring by increasing the timeout and number of retries. (@IvanHunters in #1294)
• [kubevirt] Fix building KubeVirt CCM image. (@kvaps in 3c7e256)
• [virtual-machine] Fix a regression with optional=true field. (@kvaps in 01053f7)
• [virtual-machine] Enable using custom instanceType values in virtual-machine and vm-instance by disabling field validation. (@lllamnyp in #1300, backported in #1303)
• [cozystack-api] Show correct kind values of ApplicationList. (@kvaps in #1290)
• [cozystack-api] Add missing roles to allow cozystack-controller to read Kubernetes deployments. (@klinch0 in #1342)
• [linstor] Update LINSTOR monitoring configuration to use label controller_node instead of node. (@kvaps in #1326 and #1335)
• [seaweedfs] Fix SeaweedFS volume configuration. Increase the volume size limit from 100MB to 30,000MB. (@kvaps in #1328)
• [seaweedfs] Disable proxy buffering and proxy request buffering for ingress. (@kvaps in #1330)

Dependencies

• Update flux-operator to 0.28.0. (@kingdonb in #1315 and #1344)

Documentation

• Reimplement Cozystack Roadmap as a GitHub project. (@cozystack team)
• SeaweedFS Multi-DC Configuration. (@kvaps and @NickVolynkin in cozystack/website#272)
• Troubleshooting Kube-OVN. (@kvaps and @NickVolynkin in cozystack/website#273)
• Removing failed nodes from Cozystack cluster. (@kvaps and @NickVolynkin in https://g...

Major Features and Improvements

• [platform] Enable using external application packages by adding a CozystackResourceDefinition reconciler. Read the documentation on adding external applications to Cozystack to learn more. (@klinch0 in #1313)
• [robotlb] Introduce the Hetzner RobotLB balancer. (@IvanHunters and @gwynbleidd2106 in #1233)
• [platform] Autodetect if node ports should be assigned to load balancer services. (@lllamnyp in #1271)
• [seaweedfs] Enable integration with remote filer endpoints by adding new Client topology. (@kvaps in #1239)
• [cozystack-api] Show default values from the OpenAPI spec in the application resources. (@kvaps in #1241)
• [cozystack-api] Configure dynamic API using Custom Resources. (@Klinch in #1230)
• [kubernetes] Add dependency for snapshot CRD and migration to the latest version. (@kvaps in #1275)
• [seaweedfs] Add support for whitelisting and exporting via nginx-ingress. Update cosi-driver. (@kvaps in #1277)
• [monitoring, seaweedfs] Add monitoring and Grafana dashboard for SeaweedFS. (@IvanHunters in #1285)
• [apps, cozystack-api] Improve application OpenAPI specs and READMEs by switching to cozyvalue-gen. (@kvaps and @NickVolynkin in #1216, #1314, #1316, #1321, and #1333)
• [clickhouse] Add the Clickhouse Keeper component. (@klinch0 in #1298 and #1320)

Security

• [keycloak] Store administrative passwords in the management cluster's secrets. (@IvanHunters in #1286)
• [keycloak] Update Keycloak client redirect URI to use HTTPS instead of HTTP. Enable cookie-secure. (@klinch0 in #1287)

Fixes

• [kubernetes] Fix regression in volumesnapshotclass installation from #1203. (@kvaps in #1238)
• [kubernetes] Resolve problems with pod names exceeding allowed length by shortening the name of volume snapshot CRD from *-volumesnapshot-crd-for-tenant-k8s to *-vsnap-crd. To apply this change, update each affected tenant Kubernetes cluster after updating Cozystack. (@klinch0 in #1284)
• [tenant] Fix tenant network policy to allow traffic to additional tenant-related services across namespace hierarchies. (@klinch0 in #1232)
• [kubevirt] Fix building KubeVirt CCM image. (@kvaps in 3c7e256)
• [virtual-machine] Fix a regression with optional=true field. (@kvaps in 01053f7)
• [cozystack-api] Show correct kind values of ApplicationList. (@kvaps in #1290)
• [virtual-machine] Enable using custom instanceType values in virtual-machine and vm-instance by disabling field validation. (@lllamnyp in #1300, backported in #1303)
• [kubernetes] Disable VPA for VPA in tenant Kubernetes clusters. Tenant clusters have no need for this feature, and it was not designed to work in a tenant cluster, but was enabled by mistake. (@lllamnyp in #1301 and #1318)
• [tenant, monitoring] Improve the reliability of tenant monitoring by increasing the timeout and number of retries. (@IvanHunters in #1294)
• [linstor] Update LINSTOR monitoring configuration to use label controller_node instead of node. (@kvaps in #1326)
• [seaweedfs] Fix SeaweedFS volume configuration. Increase the volume size limit from 100MB to 30,000MB. (@kvaps in #1328)
• [seaweedfs] Disable proxy buffering and proxy request buffering for ingress. (@kvaps in #1330)
• [etcd] Fix the topologySpreadConstraints for etcd. (@klinch0 in #1331)

Dependencies

• Update flux-operator to 0.27.0. (@kingdonb in #1315)

Documentation

• Reimplement Cozystack Roadmap as a GitHub project. (@cozystack team)
• SeaweedFS Multi-DC Configuration. (@kvaps and @NickVolynkin in cozystack/website#272)
• Troubleshooting Kube-OVN. (@kvaps and @NickVolynkin in cozystack/website#273)
• Removing failed nodes from Cozystack cluster. (@kvaps and @NickVolynkin in cozystack/website#273)
• Installing Talos with kexec. (@kvaps and @NickVolynkin in cozystack/website#268)
• Rewrite Cozystack tutorial. (@NickVolynkin in cozystack/website#262 and cozystack/website#268)
• How to install Cozystack in Hetzner. (@NickVolynkin and @IvanHunters in cozystack/website#280)
• Adding External Applications to Cozystack Catalog. (@klinch0 and @NickVolynkin in cozystack/website#283)
• Creating and Using Named VM Images (Golden Images) (@NickVolynkin and @kvaps in cozystack/website#276)
• Creating Encrypted Storage on LINSTOR. (@kvaps and @NickVolynkin in cozystack/website#282)
• Adding and removing components on Cozystack installation using bundle-enable and bundle-disable (@NickVolynkin in cozystack/website#281)
• Restructure Cozystack documentation. Bring managed Kubernetes, managed applications, virtualization, and networking guides to the top level. (@NickVolynkin in cozystack/website#266)

Development, Testing, and CI/CD

• [tests] Add tests for S3 buckets. (@IvanHunters in #1283)
• [tests, ci] Simplify test discovery logic; run two k8s tests as separate jobs; delete Clickhouse application after a successful test. (@lllamnyp in #1236)
• [dx] When running make commands with BUILDER value specified, PLATFORM is optional. (@kvaps in #1288)
• [tests] Fix resource specification in virtual machine tests. (@IvanHunters in #1308)
• [tests] Increase available space for e2e tests. (@kvaps in 168a24f)
• [tests, ci] Continue application tests after one of them fails. (@NickVolynkin in 634b77e)
• [ci] Use a subdomain of aenix.org for Nexus service in CI. (@lllamnyp in #1322)

Full Changelog: v0.34.0...v0.35.0-beta.1
Since Last Release: v0.35.0-alpha.3...v0.35.0-beta.1