Add new MongoDownloader tool #109

0xced · 2021-01-20T11:09:39Z

This tool automates the download of latest MongoDB binaries (MongoDB Community Server and MongoDB Database Tools) for Linux, macOS and Windows.

MongoDB Community Server contains the mongod binary.
MongoDB Database Tools contains the mongoexport and mongoimport binaries.

Then it merges the binaries into a bin directory for each platform and moves other files into community-server and database-tools directories.

Here is how the layout of the tools directory looks like after downlading and extracting files for MongoDB Community Server 4.4.3 and MongoDB Database Tools 100.2.1:

.
├── README.md
├── mongodb-linux-4.4.3-database-tools-100.2.1
│   ├── bin
│   │   ├── mongod
│   │   ├── mongoexport
│   │   └── mongoimport
│   ├── community-server
│   │   ├── LICENSE-Community.txt
│   │   ├── MPL-2
│   │   ├── README
│   │   └── THIRD-PARTY-NOTICES
│   └── database-tools
│       ├── LICENSE.md
│       ├── README.md
│       └── THIRD-PARTY-NOTICES
├── mongodb-macos-4.4.3-database-tools-100.2.1
│   ├── bin
│   │   ├── mongod
│   │   ├── mongoexport
│   │   └── mongoimport
│   ├── community-server
│   │   ├── LICENSE-Community.txt
│   │   ├── MPL-2
│   │   ├── README
│   │   └── THIRD-PARTY-NOTICES
│   └── database-tools
│       ├── LICENSE.md
│       ├── README.md
│       └── THIRD-PARTY-NOTICES
└── mongodb-windows-4.4.3-database-tools-100.2.1
    ├── bin
    │   ├── mongod.exe
    │   ├── mongoexport.exe
    │   └── mongoimport.exe
    ├── community-server
    │   ├── LICENSE-Community.txt
    │   ├── MPL-2
    │   ├── README
    │   └── THIRD-PARTY-NOTICES
    └── database-tools
        ├── LICENSE.md
        ├── README.md
        └── THIRD-PARTY-NOTICES

Fixes #82

0xced · 2021-01-22T19:13:10Z

I have tested the downloader on both on macOS and Windows so I'm pretty confident it works well.

I tried to run the tests with the new binaries and I encountered a problem: the logs are now in JSON format and the ReplicaSetReadyIdentifier log (i.e. transition to primary complete; database writes are now permitted) is not printed anymore resulting in this exceptions:

System.TimeoutException
Replica set initialization took longer than the specified timeout of 10 seconds. Please consider increasing the value of singleNodeReplSetWaitTimeout.

Some more work needs to be done for the tests to pass.

This tool automates the download of latest MongoDB binaries (_MongoDB Community Server_ and _MongoDB Database Tools_) for Linux, macOS and Windows. * _MongoDB Community Server_ contains the `mongod` binary. * _MongoDB Database Tools_ contains the `mongoexport` and `mongoimport` binaries. Then it merges the binaries into a `bin` directory for each platform and moves other files into `community-server` and `database-tools` directories. Here is how the layout of the `tools` directory looks like after downlading and extracting files for _MongoDB Community Server_ 4.4.3 and _MongoDB Database Tools_ 100.2.1: ``` . ├── README.md ├── mongodb-linux-4.4.3-database-tools-100.2.1 │ ├── bin │ │ ├── mongod │ │ ├── mongoexport │ │ └── mongoimport │ ├── community-server │ │ ├── LICENSE-Community.txt │ │ ├── MPL-2 │ │ ├── README │ │ └── THIRD-PARTY-NOTICES │ └── database-tools │ ├── LICENSE.md │ ├── README.md │ └── THIRD-PARTY-NOTICES ├── mongodb-macos-4.4.3-database-tools-100.2.1 │ ├── bin │ │ ├── mongod │ │ ├── mongoexport │ │ └── mongoimport │ ├── community-server │ │ ├── LICENSE-Community.txt │ │ ├── MPL-2 │ │ ├── README │ │ └── THIRD-PARTY-NOTICES │ └── database-tools │ ├── LICENSE.md │ ├── README.md │ └── THIRD-PARTY-NOTICES └── mongodb-windows-4.4.3-database-tools-100.2.1 ├── bin │ ├── mongod.exe │ ├── mongoexport.exe │ └── mongoimport.exe ├── community-server │ ├── LICENSE-Community.txt │ ├── MPL-2 │ ├── README │ └── THIRD-PARTY-NOTICES └── database-tools ├── LICENSE.md ├── README.md └── THIRD-PARTY-NOTICES ``` Fixes Mongo2Go#82

Also remove the AssemblyInfo.cs file and have everything in the csproj instead.

The `transition to primary complete` message in recent mongod versions now starts with a capital `T`.

* MongoDB Community Server 4.4.4 * MongoDB Database Tools 100.3.0 This is required for the tests to pass since the search patterns in the `MongoBinaryLocator` have been updated to match the new MongoDB naming scheme.

0xced · 2021-03-10T07:29:58Z

All the tests are now passing with the latest MongoDB server (4.4.4) on macOS. On Windows, I get the following error in when using commands that create console output → should not timeout:

MongoDB.Driver.MongoCommandException: Command dropIndexes failed: cannot perform operation: a background operation is currently running for collection IntegrationTest.TestCollection.

According to https://jira.mongodb.org/browse/SERVER-37726 this issue should be fixed in MongoDB 4.3.4 so maybe that's an issue with the C# driver?

To verify that the binaries committed in a55bd18 are the original ones, delete the 3 directories in the tools directory then download them with the new downloader tool:

cd src/MongoDownloader
dotnet run

You should see that the git repository has no diff if the latest versions are still MongoDB Community Server 4.4.4 and MongoDB Database Tools 100.3.0.

JohannesHoppe · 2021-03-10T16:53:04Z

Awesome work!!! =)

According to https://jira.mongodb.org/browse/SERVER-37726 this issue should be fixed in MongoDB 4.3.4 so maybe that's an issue with the C# driver?

I don't see any issue in updating the C# driver, if this fixes the tests. The nuget package itself has no external dependencies (including the c# driver) at all.

tools/README.md

0xced · 2021-03-11T21:47:36Z

I don't see any issue in updating the C# driver, if this fixes the tests.

The thing is, the tests are not fixed on Windows. 🙁 They pass 100% only on macOS (I haven't tested on Linux). Can you try yourself on Windows? Do yo get the same MongoDB.Driver.MongoCommandException in when_using_commands_that_create_console_output or do all the tests pass for you?

0xced · 2021-03-26T16:16:37Z

Have you had a chance to test with the latest version of MongoDB?

JohannesHoppe · 2021-03-26T23:43:33Z

Same for me. 😞

100% green on macOS
Mongo2GoTests.Runner.when_using_commands_that_create_console_output.should_not_timeout fails on Windows 10

JohannesHoppe · 2021-03-26T23:54:55Z

Fixed! 🎉

JohannesHoppe · 2021-03-27T06:59:12Z

@0xced I pushed v3 to nuget. Please give it a try! Many thanks for your support! 🙂👍

0xced · 2021-03-28T14:33:50Z

Awesome, thanks for merging and releasing a new version on NuGet, it works like a charm. 🎉

JohannesHoppe · 2021-04-01T15:33:34Z

I thank you for making it all possible. Great job! 👍

0xced force-pushed the downloader branch 3 times, most recently from cdeef71 to e075c62 Compare January 21, 2021 16:09

0xced force-pushed the downloader branch 3 times, most recently from 7b7aeef to 0fb3008 Compare March 9, 2021 10:46

0xced added 2 commits March 9, 2021 13:24

Update AppVeyor image to Visual Studio 2019

fdd77c1

0xced force-pushed the downloader branch 2 times, most recently from f50947a to 96c4997 Compare March 9, 2021 20:08

0xced added 6 commits March 9, 2021 22:22

Update tests

1c3fe1c

Target .NET Standard 2.1

bc68c46

Also remove the AssemblyInfo.cs file and have everything in the csproj instead.

Update MongoDB.Driver to latest version (2.12.0)

8395772

Compare mongod messages ignoring case

61951f6

The `transition to primary complete` message in recent mongod versions now starts with a capital `T`.

Update MongoDB Community Server and Database Tools

a55bd18

* MongoDB Community Server 4.4.4 * MongoDB Database Tools 100.3.0 This is required for the tests to pass since the search patterns in the `MongoBinaryLocator` have been updated to match the new MongoDB naming scheme.

Update Travis CI to use the latest .NET SDK

f56fd10

0xced force-pushed the downloader branch from 96c4997 to f56fd10 Compare March 9, 2021 21:26

JohannesHoppe approved these changes Mar 10, 2021

View reviewed changes

tools/README.md Show resolved Hide resolved

JohannesHoppe added enhancement high prio labels Mar 10, 2021

JohannesHoppe mentioned this pull request Mar 10, 2021

Add support for NUGET_PACKAGES env var #110

Merged

JohannesHoppe merged commit 1736fe0 into Mongo2Go:master Mar 26, 2021

JohannesHoppe mentioned this pull request Mar 26, 2021

Creating collections (and views) in transaction #112

Closed

0xced deleted the downloader branch March 28, 2021 14:32

korzanov mentioned this pull request Apr 28, 2024

Missing libcrypto.so.1.1 in mcr.microsoft.com/dotnet/sdk:8.0 #149

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add new MongoDownloader tool #109

Add new MongoDownloader tool #109

Uh oh!

0xced commented Jan 20, 2021

Uh oh!

0xced commented Jan 22, 2021 •

edited

Loading

Uh oh!

0xced commented Mar 10, 2021

Uh oh!

JohannesHoppe commented Mar 10, 2021

Uh oh!

Uh oh!

0xced commented Mar 11, 2021

Uh oh!

0xced commented Mar 26, 2021

Uh oh!

JohannesHoppe commented Mar 26, 2021

Uh oh!

JohannesHoppe commented Mar 26, 2021

Uh oh!

JohannesHoppe commented Mar 27, 2021

Uh oh!

0xced commented Mar 28, 2021

Uh oh!

JohannesHoppe commented Apr 1, 2021

Uh oh!

Uh oh!

Add new MongoDownloader tool #109

Add new MongoDownloader tool #109

Uh oh!

Conversation

0xced commented Jan 20, 2021

Uh oh!

0xced commented Jan 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

0xced commented Mar 10, 2021

Uh oh!

JohannesHoppe commented Mar 10, 2021

Uh oh!

Uh oh!

0xced commented Mar 11, 2021

Uh oh!

0xced commented Mar 26, 2021

Uh oh!

JohannesHoppe commented Mar 26, 2021

Uh oh!

JohannesHoppe commented Mar 26, 2021

Uh oh!

JohannesHoppe commented Mar 27, 2021

Uh oh!

0xced commented Mar 28, 2021

Uh oh!

JohannesHoppe commented Apr 1, 2021

Uh oh!

Uh oh!

0xced commented Jan 22, 2021 •

edited

Loading