Skip to content

Conversation

vszakats
Copy link
Member

@vszakats vszakats commented Apr 27, 2023

Add macOS CI jobs, both cmake and autotools for all supported crypto
backends (except BoringSSL), with debug, zlib enabled. Without running
tests. It also introduces OpenSSL 1.1 into the CI with a non-MSVC
compiler.

Credits to curl's macos.yml, that I used as a base.

Fix these issues uncovered by the new tests:

  • openssl: fix warning when built with wolfSSL, or OpenSSL 1.1 and
    earlier. CI missed it because apparently the only OpenSSL 1.1 test
    we had used MSVC, which did not complain.

    ../src/openssl.c:3852:19: error: variable 'sslError' set but not used [-Werror,-Wunused-but-set-variable]
        unsigned long sslError;
                      ^
    

    Regression from 097c8f0

  • pem: add hack to build without MD5 crypto-backend support.

    The Homebrew wolfSSL build comes with MD5 support disabled. We can
    expect this becoming the norm. FIPS also requires MD5 disabled.

    We deleted the same hack from hostkey.c a month ago:
    ad6aae3

    A better fix would be to guard the MD5 logic with our LIBSSH2_MD5
    macro.

    pem.c:214:32: error: use of undeclared identifier 'MD5_DIGEST_LENGTH'; did you mean 'SHA_DIGEST_LENGTH'?
            unsigned char secret[2*MD5_DIGEST_LENGTH];
                                   ^~~~~~~~~~~~~~~~~
                                   SHA_DIGEST_LENGTH
    

    Regression from 386e012

  • configure.ac: add crypto libs late.

    Fix it by adding crypto libs to LIBS at the end of the configuration
    process.

    Otherwise configure links crypto libs while doing feature tests,
    which can cause unwanted detections. For example LibreSSL publishes
    the function explicit_bzero(), which masks the system alternative,
    e.g. memset_s() on macOS. Then when trying to compile libssh2, its
    declaration is missing:

    bcrypt_pbkdf.c:93:5: error: implicit declaration of function 'explicit_bzero' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        _libssh2_explicit_zero(ciphertext, sizeof(ciphertext));
        ^
    ../src/misc.h:50:43: note: expanded from macro '_libssh2_explicit_zero'
                                              ^
    

    Regression from 4f0f4bf

  • cmake: fix to list our own include directory before the crypto libs',
    when building tests.

    Otherwise a global crypto header path, such as /usr/local/include,
    containing an external libssh2.h of a different version, could cause
    weird errors:

    cc -DHAVE_CONFIG_H -DLIBSSH2_LIBGCRYPT \
      -I../src -I../../src -I/usr/local/include -I[...]/libssh2/include \
      -g -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk  \
      -mmacosx-version-min=12.6 -MD -MT  \
      tests/CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o \
      -MF CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o.d  \
      -o CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o -c \
      [...]/libssh2/tests/test_aa_warmup.c
    
    [ 62%] Building C object tests/CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o
    In file included from /Users/runner/work/libssh2/libssh2/tests/test_aa_warmup.c:4:
    In file included from /Users/runner/work/libssh2/libssh2/tests/runner.h:42:
    In file included from /Users/runner/work/libssh2/libssh2/tests/session_fixture.h:43:
    /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:649:5: error: type name requires a specifier or qualifier
        LIBSSH2_AUTHAGENT_FUNC((*authagent));
        ^
    /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:649:30: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int]
        LIBSSH2_AUTHAGENT_FUNC((*authagent));
                                 ^
    /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:650:5: error: type name requires a specifier or qualifier
        LIBSSH2_ADD_IDENTITIES_FUNC((*addLocalIdentities));
        ^
    /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:650:35: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int]
        LIBSSH2_ADD_IDENTITIES_FUNC((*addLocalIdentities));
                                      ^
    /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:651:5: error: type name requires a specifier or qualifier
        LIBSSH2_AUTHAGENT_SIGN_FUNC((*agentSignCallback));
        ^
    /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:651:35: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int]
        LIBSSH2_AUTHAGENT_SIGN_FUNC((*agentSignCallback));
                                      ^
    6 errors generated.
    
  • tests/session_fixture.h: delete duplicate libssh2.h,
    libssh2_priv.h already includes it.

    Follow-up to a683133

CI logs with these errors:
https://github.com/libssh2/libssh2/actions/runs/4824079094
https://github.com/libssh2/libssh2/actions/runs/4824270819

curl's macos.yml: https://github.com/curl/curl/blob/da2470de96e94e1c8d276b9ae6e4c97c2cf54239/.github/workflows/macos.yml

Tidying-up while here:

  • tests/session_fixture.h: delete duplicate libssh2.h.
    libssh2_priv.h includes it already.

    Follow-up to a683133

  • ci.yml: yamllint warnings and formatting.

  • ci.yml: msvc section formatting and step-naming sync with macOS.

    Follow-up to f4a4c05

  • ci.yml: enable --enable-werror for msys2 jobs.

    Follow-up to 71cae94

  • appveyor.yml: show OpenSSL versions, link to image content.

Closes #1013

@vszakats vszakats added the build label Apr 27, 2023
@vszakats vszakats force-pushed the gha-macos branch 2 times, most recently from a45d105 to f316b2d Compare April 27, 2023 23:53
@vszakats vszakats changed the title ci: try adding macOS CI jobs ci: adding macOS CI jobs Apr 28, 2023
when built with wolfSSL. Not sure why it didn't come up
with OpenSSL 1.x.

```
../src/openssl.c:3852:19: error: variable 'sslError' set but not used [-Werror,-Wunused-but-set-variable]
    unsigned long sslError;
                  ^
```

Regression from 097c8f0
The Homebrew wolfSSL build comes with MD5 support disabled. This is
expected to become the norm with other builds. FIPS also requires MD5
disabled.

The same hack was deleted recently from hostkey.c: ad6aae3

A better fix would be to guard the MD5 logic with our LIBSSH2_MD5 macro.

```
pem.c:214:32: error: use of undeclared identifier 'MD5_DIGEST_LENGTH'; did you mean 'SHA_DIGEST_LENGTH'?
        unsigned char secret[2*MD5_DIGEST_LENGTH];
                               ^~~~~~~~~~~~~~~~~
                               SHA_DIGEST_LENGTH
```

Regression from 386e012
Otherwise crypto libs are linked for feature tests, which may
cause unwanted detections. For example LibreSSL publishes the
function `explicit_bzero()`, which masks the system alternative,
e.g. memset_s() on macOS. Then when trying to compile libssh2,
the declaration is missing:

```
bcrypt_pbkdf.c:93:5: error: implicit declaration of function 'explicit_bzero' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    _libssh2_explicit_zero(ciphertext, sizeof(ciphertext));
    ^
../src/misc.h:50:43: note: expanded from macro '_libssh2_explicit_zero'
                                          ^
```

Fix this by adding crypto libs to `LIBS` at the very end of the
configuration process.

Regression from 4f0f4bf
Otherwise it was possible that the detected crypto header was generic
/usr/local/include directory which may contain an external libssh2.h,
resulting in bizarre errors, like this one:

In this particular case the crypto lib was libgcrypt on macOS Homebrew.

```
cc -DHAVE_CONFIG_H -DLIBSSH2_LIBGCRYPT \
  -I../src -I../../src -I/usr/local/include -I[...]/libssh2/include \
  -g -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk  \
  -mmacosx-version-min=12.6 -MD -MT  \
  tests/CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o \
  -MF CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o.d  \
  -o CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o -c \
  [...]/libssh2/tests/test_aa_warmup.c
```

```
[ 62%] Building C object tests/CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o
In file included from /Users/runner/work/libssh2/libssh2/tests/test_aa_warmup.c:4:
In file included from /Users/runner/work/libssh2/libssh2/tests/runner.h:42:
In file included from /Users/runner/work/libssh2/libssh2/tests/session_fixture.h:43:
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:649:5: error: type name requires a specifier or qualifier
    LIBSSH2_AUTHAGENT_FUNC((*authagent));
    ^
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:649:30: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int]
    LIBSSH2_AUTHAGENT_FUNC((*authagent));
                             ^
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:650:5: error: type name requires a specifier or qualifier
    LIBSSH2_ADD_IDENTITIES_FUNC((*addLocalIdentities));
    ^
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:650:35: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int]
    LIBSSH2_ADD_IDENTITIES_FUNC((*addLocalIdentities));
                                  ^
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:651:5: error: type name requires a specifier or qualifier
    LIBSSH2_AUTHAGENT_SIGN_FUNC((*agentSignCallback));
    ^
/Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:651:35: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int]
    LIBSSH2_AUTHAGENT_SIGN_FUNC((*agentSignCallback));
                                  ^
6 errors generated.
```

Ref for all the errors fixed here:
https://github.com/libssh2/libssh2/actions/runs/4824079094
https://github.com/libssh2/libssh2/actions/runs/4824270819
@vszakats vszakats marked this pull request as draft April 28, 2023 00:49
@vszakats vszakats changed the title ci: adding macOS CI jobs ci: add macOS CI jobs + fix issues revealed Apr 28, 2023
@vszakats vszakats marked this pull request as ready for review April 28, 2023 09:04
@vszakats vszakats closed this in d93ccf4 Apr 28, 2023
@vszakats vszakats deleted the gha-macos branch April 28, 2023 14:01
vszakats added a commit to vszakats/libssh2 that referenced this pull request Jun 1, 2023
We repositioned crypto libs in 4f0f4bf
d93ccf4 libssh2#1013.

This patch also moves libz accordingly.

Reported-by: Kenneth Davidson
Fixes libssh2#1075
Closes #xxxx
vszakats added a commit to vszakats/libssh2 that referenced this pull request Jun 1, 2023
We repositioned crypto libs in 4f0f4bf
via libssh2#941 and subsequently in d4f58f0
from d93ccf4 via libssh2#1013.

This patch also moves libz accordingly, to unbreak certain build
scenarios.

Reported-by: Kenneth Davidson
Fixes libssh2#1075
Closes #xxxx
vszakats added a commit that referenced this pull request Jun 1, 2023
We repositioned crypto libs in 4f0f4bf
via #941 and subsequently in d4f58f0
from d93ccf4 via #1013.

This patch moves libz accordingly, to unbreak certain build scenarios.

Reported-by: Kenneth Davidson
Regression from 4f0f4bf #941
Fixes #1075
Closes #1077
lampmanyao pushed a commit to lampmanyao/libssh2 that referenced this pull request Jul 16, 2023
We repositioned crypto libs in 4f0f4bf
via libssh2#941 and subsequently in d4f58f0
from d93ccf4 via libssh2#1013.

This patch moves libz accordingly, to unbreak certain build scenarios.

Reported-by: Kenneth Davidson
Regression from 4f0f4bf libssh2#941
Fixes libssh2#1075
Closes libssh2#1077
agreppin pushed a commit to agreppin/libssh2 that referenced this pull request Jul 14, 2024
Add macOS CI jobs, both cmake and autotools for all supported crypto
backends (except BoringSSL), with debug, zlib enabled. Without running
tests. It also introduces OpenSSL 1.1 into the CI with a non-MSVC
compiler.

Credits to curl's `macos.yml`, that I used as a base.

Fix these issues uncovered by the new tests:

- openssl: fix warning when built with wolfSSL, or OpenSSL 1.1 and
  earlier. CI missed it because apparently the only OpenSSL 1.1 test
  we had used MSVC, which did not complain.

  ```
  ../src/openssl.c:3852:19: error: variable 'sslError' set but not used [-Werror,-Wunused-but-set-variable]
      unsigned long sslError;
                    ^
  ```

  Regression from 097c8f0

- pem: add hack to build without MD5 crypto-backend support.

  The Homebrew wolfSSL build comes with MD5 support disabled. We can
  expect this becoming the norm. FIPS also requires MD5 disabled.

  We deleted the same hack from `hostkey.c` a month ago:
  ad6aae3

  A better fix would be to guard the MD5 logic with our `LIBSSH2_MD5`
  macro.

  ```
  pem.c:214:32: error: use of undeclared identifier 'MD5_DIGEST_LENGTH'; did you mean 'SHA_DIGEST_LENGTH'?
          unsigned char secret[2*MD5_DIGEST_LENGTH];
                                 ^~~~~~~~~~~~~~~~~
                                 SHA_DIGEST_LENGTH
  ```

  Regression from 386e012

- `configure.ac`: add crypto libs late.

  Fix it by adding crypto libs to `LIBS` at the end of the configuration
  process.

  Otherwise `configure` links crypto libs while doing feature tests,
  which can cause unwanted detections. For example LibreSSL publishes
  the function `explicit_bzero()`, which masks the system alternative,
  e.g. `memset_s()` on macOS. Then when trying to compile libssh2, its
  declaration is missing:

  ```
  bcrypt_pbkdf.c:93:5: error: implicit declaration of function 'explicit_bzero' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
      _libssh2_explicit_zero(ciphertext, sizeof(ciphertext));
      ^
  ../src/misc.h:50:43: note: expanded from macro '_libssh2_explicit_zero'
                                            ^
  ```

  Regression from 4f0f4bf

- cmake: fix to list our own include directory before the crypto libs',
  when building tests.

  Otherwise a global crypto header path, such as `/usr/local/include`,
  containing an external `libssh2.h` of a different version, could cause
  weird errors:

  ```
  cc -DHAVE_CONFIG_H -DLIBSSH2_LIBGCRYPT \
    -I../src -I../../src -I/usr/local/include -I[...]/libssh2/include \
    -g -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk  \
    -mmacosx-version-min=12.6 -MD -MT  \
    tests/CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o \
    -MF CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o.d  \
    -o CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o -c \
    [...]/libssh2/tests/test_aa_warmup.c
  ```

  ```
  [ 62%] Building C object tests/CMakeFiles/test_aa_warmup.dir/test_aa_warmup.c.o
  In file included from /Users/runner/work/libssh2/libssh2/tests/test_aa_warmup.c:4:
  In file included from /Users/runner/work/libssh2/libssh2/tests/runner.h:42:
  In file included from /Users/runner/work/libssh2/libssh2/tests/session_fixture.h:43:
  /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:649:5: error: type name requires a specifier or qualifier
      LIBSSH2_AUTHAGENT_FUNC((*authagent));
      ^
  /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:649:30: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int]
      LIBSSH2_AUTHAGENT_FUNC((*authagent));
                               ^
  /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:650:5: error: type name requires a specifier or qualifier
      LIBSSH2_ADD_IDENTITIES_FUNC((*addLocalIdentities));
      ^
  /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:650:35: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int]
      LIBSSH2_ADD_IDENTITIES_FUNC((*addLocalIdentities));
                                    ^
  /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:651:5: error: type name requires a specifier or qualifier
      LIBSSH2_AUTHAGENT_SIGN_FUNC((*agentSignCallback));
      ^
  /Users/runner/work/libssh2/libssh2/tests/../src/libssh2_priv.h:651:35: error: type specifier missing, defaults to 'int' [-Werror,-Wimplicit-int]
      LIBSSH2_AUTHAGENT_SIGN_FUNC((*agentSignCallback));
                                    ^
  6 errors generated.
  ```

- `tests/session_fixture.h`: delete duplicate `libssh2.h`,
  `libssh2_priv.h` already includes it.

  Follow-up to a683133

CI logs with these errors:
https://github.com/libssh2/libssh2/actions/runs/4824079094
https://github.com/libssh2/libssh2/actions/runs/4824270819

curl's `macos.yml`: https://github.com/curl/curl/blob/da2470de96e94e1c8d276b9ae6e4c97c2cf54239/.github/workflows/macos.yml

Tidying-up while here:

- tests/session_fixture.h: delete duplicate `libssh2.h`.
  `libssh2_priv.h` includes it already.

  Follow-up to a683133

- ci.yml: yamllint warnings and formatting.

- ci.yml: msvc section formatting and step-naming sync with macOS.

  Follow-up to f4a4c05

- ci.yml: enable `--enable-werror` for msys2 jobs.

  Follow-up to 71cae94

- appveyor.yml: show OpenSSL versions, link to image content.

Closes libssh2#1013
agreppin pushed a commit to agreppin/libssh2 that referenced this pull request Jul 14, 2024
We repositioned crypto libs in 4f0f4bf
via libssh2#941 and subsequently in d4f58f0
from d93ccf4 via libssh2#1013.

This patch moves libz accordingly, to unbreak certain build scenarios.

Reported-by: Kenneth Davidson
Regression from 4f0f4bf libssh2#941
Fixes libssh2#1075
Closes libssh2#1077
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant