Skip to content

feat(outputs.microsoft_fabric): Add plugin #16827

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 30 commits into from
Jun 12, 2025

Conversation

asaharn
Copy link
Contributor

@asaharn asaharn commented Apr 18, 2025

Summary

  • Added MS Fabric as output plugin

Checklist

  • No AI generated code was used in this PR

Related issues

resolves #16371

@telegraf-tiger telegraf-tiger bot added the feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin label Apr 18, 2025
@srebhan srebhan changed the title feat: Added MS Fabric plugin for telegraf feat(outputs.microsoft_fabric): Added new plugin May 5, 2025
@telegraf-tiger telegraf-tiger bot added the plugin/output 1. Request for new output plugins 2. Issues/PRs that are related to out plugins label May 5, 2025
Copy link
Member

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution @asaharn! I have some comments in the code, but I do have a general issue with understanding the purpose of this plugin... My understanding is that this is a wrapper around the azure_data_explorer plugin, named EventStream here, and the not yet existing EeventHouse plugin. So why don't you simply add a azure_eventhouse plugin instead of this plugin and get rid of all the guesswork? The user needs to know the service anyway as he/she must fill-out the config sections...

@asaharn
Copy link
Contributor Author

asaharn commented May 7, 2025

Hi @srebhan, thank you for your time. We propose using the MSFabric plugin as the output plugin, as Microsoft Fabric is positioned as a standalone product offering (distinct from Azure) and includes components like Eventhouse and Eventstream/Eventhub as part of its underlying storage components. Fabric users should not have to manage multiple plugins or complex configurations to interact with these components. Instead, they can simply provide the connection string and have data seamlessly flushed to the appropriate storage.
As the target audience consists of Fabric users, they will naturally expect plugins that are compatible with Fabric itself, rather than directly with components like Eventhouse or Eventstream.. Additionally, the plugin can be extended to support storage types introduced within the Fabric ecosystem in the future.

@srebhan srebhan self-assigned this May 12, 2025
@srebhan
Copy link
Member

srebhan commented May 15, 2025

@asaharn any update regarding what we discussed on Slack, i.e. to implicitly define the required parameters with the connection string and the plugin will then parse those out without an explicit, endpoint-specific configuration?

@srebhan srebhan added the waiting for response waiting for response from contributor label May 15, 2025
@asaharn
Copy link
Contributor Author

asaharn commented May 15, 2025

Hi @srebhan, Apologies for the delay, I will mostly complete it by the end of this week

@telegraf-tiger telegraf-tiger bot removed the waiting for response waiting for response from contributor label May 15, 2025
@asaharn
Copy link
Contributor Author

asaharn commented May 20, 2025

@srebhan have pushed the changes

Copy link
Member

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update @asaharn! The code is much better now. I have some comments in the code...

@asaharn
Copy link
Contributor Author

asaharn commented Jun 3, 2025

Hi @srebhan, updated the changes.

@asaharn asaharn requested a review from srebhan June 3, 2025 16:06
@srebhan srebhan force-pushed the feature/fabricplugin branch from 3f01e09 to e8eb802 Compare June 3, 2025 19:18
@srebhan
Copy link
Member

srebhan commented Jun 3, 2025

@asaharn I took the freedom to push some required changes. Let me know if the code is OK for you!

@srebhan srebhan changed the title feat(outputs.microsoft_fabric): Added new plugin feat(outputs.microsoft_fabric): Add plugin Jun 3, 2025
@srebhan srebhan added new plugin area/azure Azure plugins including eventhub_consumer, azure_storage_queue, azure_monitor labels Jun 3, 2025
@telegraf-tiger
Copy link
Contributor

telegraf-tiger bot commented Jun 3, 2025

@asaharn
Copy link
Contributor Author

asaharn commented Jun 4, 2025

@srebhan Thankyou. Have checked the changes, we can proceed.

@srebhan srebhan assigned skartikey and mstrandboge and unassigned srebhan Jun 5, 2025
@srebhan srebhan added the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label Jun 5, 2025
@asaharn asaharn requested a review from skartikey June 12, 2025 11:04
@asaharn
Copy link
Contributor Author

asaharn commented Jun 12, 2025

Hi @skartikey I have pushed the changes. Requesting for a re review.

Copy link
Contributor

@skartikey skartikey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asaharn Great job!

@skartikey skartikey merged commit c885603 into influxdata:master Jun 12, 2025
21 of 24 checks passed
@github-actions github-actions bot added this to the v1.35.0 milestone Jun 12, 2025
@asaharn
Copy link
Contributor Author

asaharn commented Jun 12, 2025

Thankyou @srebhan and @skartikey

skartikey pushed a commit to skartikey/telegraf that referenced this pull request Jun 12, 2025
Co-authored-by: Sven Rebhan <srebhan@influxdata.com>
skartikey pushed a commit to skartikey/telegraf that referenced this pull request Jun 13, 2025
Co-authored-by: Sven Rebhan <srebhan@influxdata.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/azure Azure plugins including eventhub_consumer, azure_storage_queue, azure_monitor feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin new plugin plugin/output 1. Request for new output plugins 2. Issues/PRs that are related to out plugins ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

A new output plugin support for Microsoft fabric EventHouse and Eventstream
4 participants