MinimalAllocationIdentity and MaximumAllocationIdentity are global
variables that can be changed at runtime by the
InitMinMaxIdentityAllocation function. Since these are dependent on
ClusterIDShift, which is no longer a constant, this commit replaces
the globals with Getter methods to compute the correct value and avoid
a race condition in which a global is accessed before it is correctly
initialized.

Signed-off-by: Tim Horner <timothy.horner@isovalent.com>

This commit adds a new flag '--max-connected-clusters' to enable
users to configure the bit allocation of cluster ID and identity
in a numeric identity to enable support for a clustermesh with
more than 255 clusters.

Currently the only supported values for this flag are 255 (default)
and 511. Increasing this value will increase the maximum number of
clusters/maximum cluster ID by reducing the maximum number of
allocatable cluster-local identities.

This feature changes the way identites are interpreted in the datapath.
It should only be used for new clusters, as there is currently
no migration path for live clusters. All clusters connected in a
clustermesh will require this flag to be set to the same value.

Signed-off-by: Tim Horner <timothy.horner@isovalent.com>

This commit adds a new flag '--max-connected-clusters' to
clustermesh-apiserver. The value will be written to the
CiliumClusterConfig and is used by remote-clusters to validate that both
clusters are running in the same identity/clusterID allocation mode
prior to synchronizing resources.

Clusters that are using different values for '--max-connected-clusters'
should never be permitted to connect to each other, as this determines
how identity is parsed by datapath BPF programs.

Signed-off-by: Tim Horner <timothy.horner@isovalent.com>

The field MaxConnectedClusters was added to CiliumClusterConfig in a
previous commit to support the extended Clustermesh feature. This value
must be written to cluster configs in the kvstore to ensure that all
clusters in a Clustermesh are using the same identity/clusterID
allocation more prior to synchronization.

Clusters that are using different values for '--max-connected-clusters'
should never be permitted to connect to each other, as this determines
how identity is parsed by datapath BPF programs.

Signed-off-by: Tim Horner <timothy.horner@isovalent.com>

With the addition of MaxConnectedClusters to CiliumClusterConfig, the
operator's KVStore watchdog must be aware of and set this value to ensure
it is not removed on update after updating the heartbeat.

Signed-off-by: Tim Horner <timothy.horner@isovalent.com>

Extended clustermesh changes the bit allocation in skb->mark to repurpose
bits from identity and grant them to cluster ID. Users can configure
this with the agent flag '--max-connected-clusters'.

This enables support for larger clustermesh/greater cluster ID but
reduces the maximum allocatable identity.

Signed-off-by: Tim Horner <timothy.horner@isovalent.com>

Since overloadable_skb.h now requires macros defined in node_config.h,
it is necessary to reorder dependencies in some tests that are using
unspec.h and including bpf_sock.c, so that when overloadable_skb.h is
included in lib/common.h, it occurs after bpf_sock.c includes
node_config.h

Signed-off-by: Tim Horner <timothy.horner@isovalent.com>

Signed-off-by: Tim Horner <timothy.horner@isovalent.com>

Set maxConnectedClusters in ci-multicluster to test extended clustermesh feature.

Signed-off-by: Tim Horner <timothy.horner@isovalent.com>

Setting Cluster ID in the upgrade-downgrade test will provide test
coverage for the updates made to BPF maps and the datapath to provide
support for extended clustermesh. These changes include expanding the
Cluster ID field on lxc, endpoint, lb4backend, and lb6backend maps, as
well as modifications to skb->mark parsing for identity.

Signed-off-by: Tim Horner <timothy.horner@isovalent.com>

New unit tests have been added to cover changes made to the skb
functions that get and set identity and cluster_id in skb->mark.

Signed-off-by: Tim Horner <timothy.horner@isovalent.com>

Remove IDENTITY_MAX from node_config.h and replace IDENTITY_LEN with
a global config variable using DECLARE_CONFIG and ASSIGN_CONFIG. This
will allow replacement at runtime in later iterations.

New helper functions now calculate the other previously defined macros
for IDENTITY_MAX, CLUSTER_ID_UPPER_MASK, and MARK_MAGIC_CLUSTER_ID_MASK,
since these all depend on IDENTITY_LEN, which could change after
compilation.

Signed-off-by: Tim Horner <timothy.horner@isovalent.com>