-
Notifications
You must be signed in to change notification settings - Fork 9.7k
feat(tsdb): allow using Direct IO for chunks writing #15365
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
/prombench main |
/prombench cancel |
Benchmark cancel is in progress. |
125e4a9
to
3c6fec4
Compare
/prombench main |
/prombench cancel |
Benchmark cancel is in progress. |
/prombench main |
/prombench cancel |
Benchmark cancel is in progress. |
/prombench main |
Benchmark tests are running for 3 days! If this is intended ignore this message otherwise you can cancel it by commenting: |
Benchmark tests are running for 5 days! If this is intended ignore this message otherwise you can cancel it by commenting: |
Would this require holding WAL records in memory until there is 4KB to write? |
Yes, theoretically we could omit writing up to "offsetAlign-1" of data, we could work around that, but it might result in unwanted side effects like write amplification, bloat, and switching between direct and buffered I/O... That being said, we already write in blocks of I'll need to look into this in more detail to see if it's really worth it. |
Cool. What happens if you plot just file cache memory? Another thing that DIO can buy is placing less pressure on page cache evictions and thus reducing the number of major page faults occurring in the places using mapped IO. Could be another useful metric to look at. |
There is one in here https://github.com/prometheus/proposals/pull/45/files |
ae808d1
to
60dfde2
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is a lot of deep magic here; I did not attempt to follow all of it.
Signed-off-by: machine424 <ayoubmrini424@gmail.com>
force push + rebase, the diff wouldn't be helpful :( |
without checking the conflicts, I'd say that a new feature flag was added, again. |
Signed-off-by: Ayoub Mrini <ayoubmrini424@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
See proposal prometheus/proposals#45