Skip to content

Conversation

AlexanderWert
Copy link
Member

Goal

Have a single source of truth of the relation between ECs fields and OpenTelemetry attributes and metrics

What this PR adds:

  • extended schema files to include explicit mapping from ECS attributes to OTel attributes and metrics
  • extended generation tooling to check mapping against OTel semantic conventions repository ensuring the mapping is up to date with semconv changes
  • extended generation tooling to generate additional docs:
    • for each ECS field that corresponds to an OTel attribute generate reference to the attribute and document relation type
    • Docs page explaining the relation of ECS and OTel with additional overview pages for the mapping
image image image

@AlexanderWert AlexanderWert requested a review from a team as a code owner December 12, 2024 11:13
Copy link

Documentation changes preview: https://ecs_bk_2415.docs-preview.app.elstc.co/diff

… ECS

Signed-off-by: Alexander Wert <alexander.wert@elastic.co>
Signed-off-by: Alexander Wert <alexander.wert@elastic.co>
Signed-off-by: Alexander Wert <alexander.wert@elastic.co>
Signed-off-by: Alexander Wert <alexander.wert@elastic.co>
Copy link
Member

@felixbarny felixbarny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic overview!

Signed-off-by: Alexander Wert <alexander.wert@elastic.co>
Signed-off-by: Alexander Wert <alexander.wert@elastic.co>
@@ -41,6 +42,9 @@
def main() -> None:
args = argument_parser()

if not args.semconv_version:
raise KeyError("OTel Semantic Conventions version has not been provided as a config option '--semconv-version'")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should the default be the last one? we can detect it through git and skip manually selecting it
I think for this purpose we need always to adhere to the last available version

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather keep that a manual step, or use something like dependabot to update it but not updating it automatically, that has some risks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are using there only schema, what risks do you have in mind?
we are adding a lot of new fields into each release but most probably we would need to update mapping anyway and can update the version, so this comment is not blocking.

@trisch-me trisch-me merged commit 009caf4 into elastic:main Dec 18, 2024
3 checks passed
Copy link
Contributor

@mjwolf mjwolf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great!

[[ecs-opentelemetry-relation]]
=== Relation between ECS and Semantic Conventions

The https://github.com/elastic/ecs/tree/main/schemas[ECS schema files] contain an explicit mapping between ECS fields and corresponding OTel semnatic convention attributes.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a typo here: semnatic


[%header]
|====
| ECS Field | Relation | OTel Semantic Conventions Attribute | Stability
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What Stability means could be confusing. Is it stability in ECS or otel?

This file is automatically generated. Don't edit it manually!
////
The following table gives an overview of mappings between individual ECS fields (in ECS version `9.0.0-dev`)
and corresponding OTel semantic convention attributes (in SemConv version `1.29.0`).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be good to state somewhere why all fields are not mapped yet, or given the conflict or n/a catagory. If someone looks for an ECS field, and doesn't find it, there's nowhere in these docs that states why some are missing, or what someone should do with that info.


^| image:https://img.shields.io/badge/related-efc20d?style=flat[relation,title=related,link=ecs-opentelemetry.html#ecs-opentelemetry-relation]

<| https://opentelemetry.io/docs/specs/semconv/attributes-registry/k8s/#k8s-pod-label[k8s.pod.label] +
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any way to link to a specific semconv version in the otel docs? I couldn't find a way. But if there is, it would be nice to link to the same SEMCONV_VERSION this is generated from.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants