Skip to content

Conversation

guillaume-michel
Copy link
Contributor

@guillaume-michel guillaume-michel commented Jan 10, 2023

Specify library name and version: protobuf/3.5.1.1

This library is a dependency for Kinova robot's proprietary library.


@CLAassistant
Copy link

CLAassistant commented Jan 10, 2023

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@conan-center-bot

This comment has been minimized.

@ghost
Copy link

ghost commented Jan 10, 2023

I detected other pull requests that are modifying protobuf/all recipe:

This message is automatically generated by https://github.com/ericLemanissier/conan-center-conflicting-prs so don't hesitate to report issues/improvements there.

@conan-center-bot

This comment has been minimized.

@prince-chrismc
Copy link
Contributor

You need to wait for access request before it will be merged :)

prince-chrismc
prince-chrismc previously approved these changes Jan 11, 2023
@conan-center-bot

This comment has been minimized.

@github-actions
Copy link
Contributor

Hooks produced the following warnings for commit 00c7e93
protobuf/3.5.1.1
post_package(): WARN: [MISSING SYSTEM LIBS (KB-H043)] Library './lib/libprotobufd.so' links to system library 'm' but it is not in cpp_info.system_libs.
post_package(): WARN: [MISSING SYSTEM LIBS (KB-H043)] Library './lib/libprotocd.so' links to system library 'm' but it is not in cpp_info.system_libs.
post_package(): WARN: [MISSING SYSTEM LIBS (KB-H043)] Library './lib/libprotobuf-lited.so' links to system library 'm' but it is not in cpp_info.system_libs.

Copy link
Contributor

@jcar87 jcar87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @guillaume-michel, thanks for opening this PR.

We typically don't add older versions unless sufficiently justified, and Protobuf 3.5.1.1 is a little over 5 years old.

We have recently removed logic in the recipe to support older versions, which simplifies the logic in the recipe. This would be moving in the opposite direction, and may introduce corner cases in which certain configurations don't work.

In particular with protobuf, my understanding is that Protobuf 3 is backwards compatible and for users it should generally be safe to upgrade to newer versions without experiencing breakage - so long as they have the ability to rebuild/relink protobuf-dependent code (as there are no guarantees on binary compatibility).

Is that the case here? Would love to understand what's holding back being able to move to a newer version, so that we can take appropriate action, as we'd love to support this use case :)

@prince-chrismc
Copy link
Contributor

We can put the older version in a separate recipe folder so not complicate the newer ones

@guillaume-michel
Copy link
Contributor Author

Hi @guillaume-michel, thanks for opening this PR.

We typically don't add older versions unless sufficiently justified, and Protobuf 3.5.1.1 is a little over 5 years old.

We have recently removed logic in the recipe to support older versions, which simplifies the logic in the recipe. This would be moving in the opposite direction, and may introduce corner cases in which certain configurations don't work.

In particular with protobuf, my understanding is that Protobuf 3 is backwards compatible and for users it should generally be safe to upgrade to newer versions without experiencing breakage - so long as they have the ability to rebuild/relink protobuf-dependent code (as there are no guarantees on binary compatibility).

Is that the case here? Would love to understand what's holding back being able to move to a newer version, so that we can take appropriate action, as we'd love to support this use case :)

Hi,
Thanks for taking some time to review the PR.

We are building a medical robot which is operated via a vendor provided binary only library which exposes protobuf 3.5.1.1 in its public interface. We tried to ask for an upgraded version but did not work out in the end. Our only option right now is to use protobuf 3.5.1.1.

In general, I agree with removing old legacy code but in practice it is not always possible. I am also worried that 2 years from now I could not find the dependencies I am looking for in conan and could not build old version. I am aware that I can use my own artifactory to keep my binaries but how am I supposed to maintain recipes which disappears from conan as time goes on? My naive approach would be to keep a compulsive cumulative strategy but I know it is a large burden to maintain.

@ghost ghost mentioned this pull request Jan 17, 2023
3 tasks
@prince-chrismc
Copy link
Contributor

prince-chrismc commented Jan 17, 2023

but how am I supposed to maintain recipes which disappears from conan as time goes on?

You can see our FAQ for more details but we never delete anything from ConanCenter. That does mean version stop getting updates. Unless there's a specific use-case (which this does seem to meet) then we drop support them when they are too burdensome.

The packages are always available but the support and maintenance is not 😸 hopeful that clarifies the policy


If you contribute a working version that is not too intrusive I dont see why it could not be accepted but thank you for filling in the explanation as to why ❤️

protobuf/3.5.1.1@:
CI failed to create some packages ([All logs](https://c3i.jfrog.io/c3i/misc/summary.html?json=https://c3i.jfrog.io/c3i/misc/logs/pr/15202/3-linux-gcc/protobuf/3.5.1.1//summary.json))

Logs for packageID

@jcar87
Copy link
Contributor

jcar87 commented Jan 20, 2023

Hi @guillaume-michel, thanks for opening this PR.
We typically don't add older versions unless sufficiently justified, and Protobuf 3.5.1.1 is a little over 5 years old.
We have recently removed logic in the recipe to support older versions, which simplifies the logic in the recipe. This would be moving in the opposite direction, and may introduce corner cases in which certain configurations don't work.
In particular with protobuf, my understanding is that Protobuf 3 is backwards compatible and for users it should generally be safe to upgrade to newer versions without experiencing breakage - so long as they have the ability to rebuild/relink protobuf-dependent code (as there are no guarantees on binary compatibility).
Is that the case here? Would love to understand what's holding back being able to move to a newer version, so that we can take appropriate action, as we'd love to support this use case :)

Hi, Thanks for taking some time to review the PR.

We are building a medical robot which is operated via a vendor provided binary only library which exposes protobuf 3.5.1.1 in its public interface. We tried to ask for an upgraded version but did not work out in the end. Our only option right now is to use protobuf 3.5.1.1.

In general, I agree with removing old legacy code but in practice it is not always possible. I am also worried that 2 years from now I could not find the dependencies I am looking for in conan and could not build old version. I am aware that I can use my own artifactory to keep my binaries but how am I supposed to maintain recipes which disappears from conan as time goes on? My naive approach would be to keep a compulsive cumulative strategy but I know it is a large burden to maintain.

Hi @guillaume-michel - thanks for your reply. Indeed I can see your predicament. If you are a consuming closed-source binaries that impose a specific version, this could be a problem. Out of curiosity, are the Protobuf binaries also provided with the Kinova library? Given that Protobuf does not guarantee binary compatibility (unlike Qt for example, that has some guarantees), even if you independently build it I'm not sure it's guaranteed to work.

As @prince-chrismc points out, it should be clarified that the Conan Center remote will serve all versions that have ever been available - even if the recipe in the repository has stopped supporting older versions, so if Protobuf 3.5 were to be published, it would be available in the future even if the recipe stops supporting it.

As he mentions, if we can get this to pass our CI checks on the expected platforms, we're happy to proceed with this PR.

@guillaume-michel
Copy link
Contributor Author

I am trying to fix the CI failures for protobuf/3.5.1.1 with shared library but I don't know how to fix it.

profile failure1 is:

[settings]
arch=x86_64
build_type=Debug
compiler=gcc
compiler.libcxx=libstdc++
compiler.version=5
os=Linux
[options]
protobuf:shared=True

When I am building the package locally on my machine with the following command:
conan create . 3.5.1.1@ --build -pr=failure1 which is the command executed in the CI,
I get package ID: 75ad3442f6ede2a299883b25274d1968d8cbd66d
And I have the same failure as in the CI: protoc doesn't find libprotocd.so but it exists in the lib folder.

When I am building the package with the following command:
conan create . 3.5.1.1@ --build -pr:h=failure1 -pr:b=failure1
I get the same package ID: 75ad3442f6ede2a299883b25274d1968d8cbd66d
But this time no errors.

Repeatting the same with 3.21.9 work in both cases.

Comparing protoc for both versions shows that protoc 3.5.1.1 is missing:

 (RUNPATH)            Library runpath: [$ORIGIN/../lib]

which is there because there is the following in the RUNPATH in libprotocd.so v3.21.9:

(RUNPATH)            Library runpath: [$ORIGIN]

Any idea why the test_package fails when using a single profile?
Any idea why it is working with dual profile without the correct RUNPATH?

Thanks for your help

@guillaume-michel
Copy link
Contributor Author

In the meantime, I will have a look at the 3.5.1.1 CMakeLists.txt and try to patch the file so the RUNPATH is correctly added.

@ghost ghost mentioned this pull request Jan 27, 2023
3 tasks
@guillaume-michel
Copy link
Contributor Author

Hi @guillaume-michel, thanks for opening this PR.
We typically don't add older versions unless sufficiently justified, and Protobuf 3.5.1.1 is a little over 5 years old.
We have recently removed logic in the recipe to support older versions, which simplifies the logic in the recipe. This would be moving in the opposite direction, and may introduce corner cases in which certain configurations don't work.
In particular with protobuf, my understanding is that Protobuf 3 is backwards compatible and for users it should generally be safe to upgrade to newer versions without experiencing breakage - so long as they have the ability to rebuild/relink protobuf-dependent code (as there are no guarantees on binary compatibility).
Is that the case here? Would love to understand what's holding back being able to move to a newer version, so that we can take appropriate action, as we'd love to support this use case :)

Hi, Thanks for taking some time to review the PR.
We are building a medical robot which is operated via a vendor provided binary only library which exposes protobuf 3.5.1.1 in its public interface. We tried to ask for an upgraded version but did not work out in the end. Our only option right now is to use protobuf 3.5.1.1.
In general, I agree with removing old legacy code but in practice it is not always possible. I am also worried that 2 years from now I could not find the dependencies I am looking for in conan and could not build old version. I am aware that I can use my own artifactory to keep my binaries but how am I supposed to maintain recipes which disappears from conan as time goes on? My naive approach would be to keep a compulsive cumulative strategy but I know it is a large burden to maintain.

Hi @guillaume-michel - thanks for your reply. Indeed I can see your predicament. If you are a consuming closed-source binaries that impose a specific version, this could be a problem. Out of curiosity, are the Protobuf binaries also provided with the Kinova library? Given that Protobuf does not guarantee binary compatibility (unlike Qt for example, that has some guarantees), even if you independently build it I'm not sure it's guaranteed to work.

As @prince-chrismc points out, it should be clarified that the Conan Center remote will serve all versions that have ever been available - even if the recipe in the repository has stopped supporting older versions, so if Protobuf 3.5 were to be published, it would be available in the future even if the recipe stops supporting it.

As he mentions, if we can get this to pass our CI checks on the expected platforms, we're happy to proceed with this PR.

@jcar87 Kinova provides 2 libraries for their robot:

  • 1 with including protobuf 3.5.1.1
  • 1 not including protobuf but requires to link with an external library.

We chose to use the 1 without included protobuf because we are using protobuf elsewhere in our codebase and we don't want to link with whole kinova library for that part. Hope it clarifies the situation.

@guillaume-michel guillaume-michel requested review from prince-chrismc and jcar87 and removed request for jcar87 and prince-chrismc January 28, 2023 07:22
@conan-center-bot
Copy link
Contributor

Conan v1 pipeline ✔️

All green in build 5 (e3e509ad651018045cc2ac790dfa1db61559a9c7):

  • protobuf/3.21.9@:
    All packages built successfully! (All logs)

  • protobuf/3.21.4@:
    All packages built successfully! (All logs)

  • protobuf/3.20.0@:
    All packages built successfully! (All logs)

  • protobuf/3.18.1@:
    All packages built successfully! (All logs)

  • protobuf/3.19.6@:
    All packages built successfully! (All logs)

  • protobuf/3.19.4@:
    All packages built successfully! (All logs)

  • protobuf/3.5.1.1@:
    All packages built successfully! (All logs)


Conan v2 pipeline (informative, not required for merge) ✔️

Note: Conan v2 builds are informative and they are not required for the PR to be merged.

All green in build 4 (e3e509ad651018045cc2ac790dfa1db61559a9c7):

  • protobuf/3.20.0@:
    All packages built successfully! (All logs)

  • protobuf/3.21.9@:
    All packages built successfully! (All logs)

  • protobuf/3.19.6@:
    All packages built successfully! (All logs)

  • protobuf/3.5.1.1@:
    All packages built successfully! (All logs)

  • protobuf/3.19.4@:
    All packages built successfully! (All logs)

  • protobuf/3.21.4@:
    All packages built successfully! (All logs)

  • protobuf/3.18.1@:
    All packages built successfully! (All logs)

Comment on lines +39 to +40
--- a/src/google/protobuf/compiler/js/embed.cc
+++ /dev/null
Copy link
Contributor

@SpaceIm SpaceIm Jan 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't you keep this file if it is not used anymore? Patch would be more readable.

Comment on lines +18 to +49
diff --git a/cmake/install.cmake b/cmake/install.cmake
index 441bf55..ee17601 100644
--- a/cmake/install.cmake
+++ b/cmake/install.cmake
@@ -13,6 +13,13 @@ foreach(_library
PROPERTY INTERFACE_INCLUDE_DIRECTORIES
$<BUILD_INTERFACE:${protobuf_source_dir}/src>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>)
+ if (UNIX AND NOT APPLE)
+ set_property(TARGET ${_library}
+ PROPERTY INSTALL_RPATH "$ORIGIN")
+ elseif (APPLE)
+ set_property(TARGET ${_library}
+ PROPERTY INSTALL_RPATH "@loader_path")
+ endif()
install(TARGETS ${_library} EXPORT protobuf-targets
RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} COMPONENT ${_library}
LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} COMPONENT ${_library}
@@ -21,6 +28,13 @@ endforeach()

install(TARGETS protoc EXPORT protobuf-targets
RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR} COMPONENT protoc)
+if (UNIX AND NOT APPLE)
+ set_property(TARGET protoc
+ PROPERTY INSTALL_RPATH "$ORIGIN/../lib")
+elseif (APPLE)
+ set_property(TARGET protoc
+ PROPERTY INSTALL_RPATH "@loader_path/../lib")
+endif()

install(FILES ${CMAKE_CURRENT_BINARY_DIR}/protobuf.pc ${CMAKE_CURRENT_BINARY_DIR}/protobuf-lite.pc DESTINATION "${CMAKE_INSTALL_LIBDIR}/pkgconfig")

Copy link
Contributor

@SpaceIm SpaceIm Jan 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be avoided, conanfile.py of test_v1_package should be fixed instead:

from conans import ConanFile, CMake, tools
import os


class TestPackageConan(ConanFile):
    settings = "os", "arch", "compiler", "build_type"
    generators = "cmake", "cmake_find_package_multi"
    test_type = "explicit"

    def requirements(self):
        self.requires(self.tested_reference_str)

    def build_requirements(self):
        if hasattr(self, "settings_build"):
            self.build_requires(self.tested_reference_str)

    def build(self):
        with tools.no_op() if hasattr(self, "settings_build") else tools.run_environment(self):
            cmake = CMake(self)
            cmake.definitions["protobuf_LITE"] = self.options["protobuf"].lite
            cmake.configure()
            cmake.build()

    def test(self):
        if not tools.cross_building(self):
            self.run("protoc --version", run_environment=True)
            self.run(os.path.join("bin", "test_package"), run_environment=True)

@stale
Copy link

stale bot commented May 8, 2023

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale
Copy link

stale bot commented Jul 15, 2023

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale
Copy link

stale bot commented Oct 15, 2023

This pull request has been automatically closed because it has not had recent activity. Thank you for your contributions.

@stale stale bot closed this Oct 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants