Skip to content

Conversation

elichai
Copy link
Contributor

@elichai elichai commented Oct 24, 2019

Utilizing C++11 and platform specific function(timegm) to replace the usage of boost in converting a timestamp in string to a unix epoch.

Moved the old function to the unit tests to check for consistency (in case there's some corner case in timezones)

Sadly C++11 doesn't have anything to handle timezones currently(C++20 seems to have heh) so I had to use timegm from POSIX and _mkgmtime from windows.

Tested on linux only.

@maflcko maflcko changed the title Removing Boost from DecodeDumpTime wallet: Remove Boost from DecodeDumpTime Oct 24, 2019
@laanwj
Copy link
Member

laanwj commented Oct 24, 2019

The function to format a ISO8601 datetime is called FormatISO8601DateTime, maybe this should be ParseISO8601DateTime?

Also: there seems to be no error handling; how does this handle invalid input?
(I guess returning 0 would be OK, but not crashing or raising an exception, though not sure, it's something to think about at least)
(nm, I missed the iss.fail() bit)

@elichai
Copy link
Contributor Author

elichai commented Oct 24, 2019

Great :) I actually looked for a header to include this so I won't need to do manual declaration in the test.

I'll rename it and move it to time.h/cpp

@laanwj
Copy link
Member

laanwj commented Oct 25, 2019

I'll rename it and move it to time.h/cpp

Yes, that'd make sense! Could also do some roundtrip tests then.

I'm a bit concerned about using timegm; from the man page:

CONFORMING TO
       These functions are nonstandard GNU extensions that are also present on the BSDs.  Avoid their use.

Such an explicit warning makes me feel bad. Seems sure this is going to cause builder complaints on some obscure or less obscure platform (does this exist on Android ?). It might be worse than just relying on boost.

I wonder if we can find an alternative to this. The old code did:

return (ptime - epoch).total_seconds();

Doesn't the same trick exist for C++11 time manipulation?

Edit: weird! travis failure looks like std::get_time isn't available on trusty. Looks like these were only added in GCC 5.

@elichai
Copy link
Contributor Author

elichai commented Oct 25, 2019

I'm a bit concerned about using timegm; from the man page:

CONFORMING TO
       These functions are nonstandard GNU extensions that are also present on the BSDs.  Avoid their use.

Such an explicit warning makes me feel bad. Seems sure this is going to cause builder complaints on some obscure or less obscure platform (does this exist on Android ?). It might be worse than just relying on boost.

I'll need to check if Android has it. I'm pretty sure glibc had it for a while so anything glibc should be fine. But I'll look into it when I'm on my laptop again.
We could copy glibc's implementation (bad idea). Or technically we could drop the string representation and just store the epoch, Altough that will be a breaking change that I'm not even sure what it'll affect exactly.

I wonder if we can find an alternative to this. The old code did:

return (ptime - epoch).total_seconds();

Doesn't the same trick exist for C++11 time manipulation?

Not that I could find. I'll try to look more. But maybe we better off leaving it with boost for now (and coming back to it when we're actually ready to get rid of boost once and for all)

Edit: weird! travis failure looks like std::get_time isn't available on trusty. Looks like these were only added in GCC 5.

That's just weird. So before gcc5 they didn't fully support c++11?

@laanwj
Copy link
Member

laanwj commented Oct 25, 2019

We could copy glibc's implementation (bad idea)

That would be a license violation (LGPL is not compatible with MIT). We could use or copy from some other library which is MIT-compatibly-licensed. But yes, probably not worth it as long as it's still using boost for so many other things.
It's surprising how involved "parse a ISO8601 datetime" is in C++ with just the standard library, even C++11.

That's just weird. So before gcc5 they didn't fully support c++11?

Apparently!

@elichai
Copy link
Contributor Author

elichai commented Oct 25, 2019

FWIU the boost trick is just because boost's time isn't unix time. So if you want to get the time since unix epoch you need to subtract unix time 0 from your time.

@elichai
Copy link
Contributor Author

elichai commented Oct 25, 2019

It's surprising how involved "parse a ISO8601 datetime" is in C++ with just the standard library, even C++11.

Yeah it seems like timezones stuff will only get in in C++20.

There's also strptime(3) which might work. And it's posix instead of a gnu extension. (but then windows https://social.msdn.microsoft.com/Forums/vstudio/en-US/199a0e75-cb51-40b8-8e3c-230f65a2f4a4/alternative-for-strptime?forum=vcgeneral)

@laanwj
Copy link
Member

laanwj commented Oct 25, 2019

FWIW strptime is disallowed by test/lint/lint-locale-dependence.sh because its date parsing can be locale-dependent (month names, for example, though not relevant for this specific case).

@elichai
Copy link
Contributor Author

elichai commented Oct 25, 2019

Right. forgot about that. (I see that getdate(3) is also there)
@laanwj do you know if the format always looks like this? 2019-10-24T23:46:06Z
i.e. YYYY-MM-DDTHH:MM:SSZ. if so I can probably write an easy parser from that to std::tm and then use mktime to get the time_t.

@laanwj
Copy link
Member

laanwj commented Oct 25, 2019

@laanwj do you know if the format always looks like this? 2019-10-24T23:46:06Z

Yes, it always looks like that.

But to be honest I think it's better to revisit this later. Likely, at the time that we're really ready to get rid of boost, trusty is no longer a relevant platform, and it can simply use std::gettime.

and then use mktime to get the time_t.

mktime uses the local timezone (and does something with daylight saving time), it's unfortunately not useful for this. This is why GNU added timegm in the first place.

@elichai
Copy link
Contributor Author

elichai commented Oct 25, 2019

But to be honest I think it's better to revisit this later. Likely, at the time that we're really ready to get rid of boost, trusty is no longer a relevant platform, and it can simply use std::gettime.

Yeah :/ I guess we'll leave this for a while and I'll move on :)

@maflcko
Copy link
Member

maflcko commented Oct 25, 2019

I guess the rename can be done today. I wasn't aware that this util forms a round trip with FormatISO8601DateTime.

@DrahtBot
Copy link
Contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #16037 (rpc: Enable wallet import on pruned nodes by promag)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@laanwj
Copy link
Member

laanwj commented Oct 26, 2019

Yes, doing the rename/move separately might be worthwhile.

maybe instead of getting rid of boost dep, new goal should be "isolate boost usage to a few places that we know are pain points" (such as date/time/sleep/filesystem handling)

@elichai
Copy link
Contributor Author

elichai commented Oct 26, 2019

Opened #17266

@elichai elichai closed this Oct 26, 2019
maflcko pushed a commit to maflcko/bitcoin-core that referenced this pull request Oct 28, 2019
e7b02b5 Add roundtrip and more tests to ParseISO8601DateTime and FormatISO8601DateTime (Elichai Turkel)
9e2c623 Rename DecodeDumpTime to ParseISO8601DateTime and move to time.cpp (Elichai Turkel)

Pull request description:

  As discussed in bitcoin#17245.

  1. Renamed the function.
  2. Moved it from `rpcdump.cpp` to `time.cpp`.
  3. Added a check if the time is less then epoch return 0 to prevent an overflow.
  4. Added more edge cases tests and a roundtrip test.

ACKs for top commit:
  laanwj:
    ACK e7b02b5
  MarcoFalke:
    ACK e7b02b5
  promag:
    Code review ACK e7b02b5. Moved code is correct, left a comment regarding the test change.

Tree-SHA512: 703c21e09b2aabc992235149e67acba63d9d77a593ec8f6d2fec3eb63a7e5c406d56cbce6c6513ab32fba43367d073d2345f3b589843e3c5fe4f55ea3e00bf29
@practicalswift
Copy link
Contributor

Now that #21110 has been merged I think it might be time to resurrect this PR :)

Context: #21110 (comment)

@maflcko
Copy link
Member

maflcko commented May 12, 2022

Maybe use https://en.cppreference.com/w/cpp/chrono/parse (C++20)?

@bitcoin bitcoin locked and limited conversation to collaborators May 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants