Upgrade janet to 1.27, add mingw workflow build #2155

AlecTroemel · 2023-03-22T02:13:57Z

Includes support for mingw builds, however we may need to additionally make some edits to the CMakeList.txt.

Hopefully will address #2154

includes support for mingw builds, along with sandbox functionality needed to lock down io

sogaiu · 2023-03-22T05:10:37Z

After cloning and checking out an appropriate branch, I did something like the following (via an msys2 mingw64 terminal -- trying via an "ordinary" console arrangement ended in failure for me but using a msys2 mingw64 shell / terminal seemed to help):

cd TIC-80

# edit CMakeFiles.txt to add bit about MINGW and using ming32-make:
#
#    elseif(MINGW)
#        add_custom_command(
#            OUTPUT ${THIRDPARTY_DIR}/janet/build/c/janet.c
#            COMMAND mingw32-make build/c/janet.c
#            WORKING_DIRECTORY ${THIRDPARTY_DIR}/janet/
#        )   

# updates vendor/janet
git submodule update

# regenerate amalgamated janet.c
cd vendor/janet
mingw32-make

# off to generate build files
cd ../..
cd build
cmake -G "MinGW Makefiles" ..

# build -- avoid using -j when testing so output easier to grok
mingw32-make

# try out the result
./bin/tic80.exe

That seemed to work here.

Not sure if editing CMakeFiles.txt is really necessary. Will try without editing too.

I also edited janetconf.h to make version numbers more up-to-date, but I don't know if this was necessary.

sogaiu · 2023-03-22T05:32:12Z

Not sure if editing CMakeFiles.txt is really necessary. Will try without editing too.

It didn't appear to be necessary.

Removed the changes to CMakeFiles.txt, regenerated the build files with cmake, removed bin/tic80.exe, and tried again. Result appeared to be a success.

Note that I already had what seemed to be the prerequisites listed in the build instructions from previous activities so I'm fuzzy on exactly what those bits are and how they got installed. I think many pieces were in place via scoop. Regarding MinGW -- I've never really understood exactly what that points at, but I think I have some setup that's based on MSYS2.

AlecTroemel · 2023-03-22T13:00:32Z

ah good call on the janetconf.h, totally forgot about that. Ya Im not entirely what mingw does and where it sits.. I think its a windows implementation of common GNU library's, so you can build "as if" its on linux? If thats the case it would make intuitive sense that changes to the CMakeFiles.txt would not be necessary.

I'll go through the janetconf and get that up to date

AlecTroemel · 2023-03-22T13:01:02Z

I may also try my hand at adding a mingw build step to the github actions pipeline

NPO-197 · 2023-03-22T13:33:53Z

Hmm... I think a change to the config might be necessary after all, I still get an error when trying to build the Janet library while leaving the CMakelists.txt file "as is".

NPO-197 · 2023-03-22T13:35:07Z

Currently trying to build it on my end using mingw, should be possible on this branch with Janet being updated, sorry my laptop is not very fast.

sogaiu · 2023-03-22T13:35:20Z

Possibly off-topic, but:

Regarding MinGW -- I've never really understood exactly what that points at,

My confusion about this comes from statements such as:

MinGW - "formerly mingw32, is a free and open source software development environment to create Microsoft Windows applications"
Mingw-w64 - "is a free and open source software development environment to create (cross-compile) Microsoft Windows PE applications. It was forked in 2005–2010 from MinGW (Minimalist GNU for Windows)"

I think what I've been using is the latter and the build instructions mention mingw-w64 (though in CMakeLists.txt I see MinGW) and to make things more fun, some of the commands we use are named like mingw32-make.

Quite confusing.

sogaiu · 2023-03-22T13:36:53Z

@NPO-197 Did you make the changes mentioned here? Without them (or something close to them), I didn't have any luck.

NPO-197 · 2023-03-22T13:37:38Z

@sogaiu That is what I am trying now.

NPO-197 · 2023-03-22T13:39:50Z

Possibly off-topic, but:

Regarding MinGW -- I've never really understood exactly what that points at,

My confusion about this comes from statements such as:
* [MinGW](https://en.wikipedia.org/wiki/MinGW) - "formerly mingw32, is a free and open source software development environment to create Microsoft Windows applications"

* [Mingw-w64](https://en.wikipedia.org/wiki/Mingw-w64) - "is a free and open source software development environment to create (cross-compile) Microsoft Windows PE applications. It was forked in 2005–2010 from MinGW (Minimalist GNU for Windows)"
I think what I've been using is the latter and the build instructions mention mingw-w64 (though in CMakeLists.txt I see MinGW) and to make things more fun, some of the commands we use are named like mingw32-make.

Quite confusing.

My understanding is that even if you have ming-w64 you still use the command mingw32-make to build a 64 bit binary which is about as clear as mud to me.

sogaiu · 2023-03-22T13:48:34Z

My understanding is that even if you have ming-w64 you still use the command mingw32-make to build a 64 bit binary which is about as clear as mud to me.

Yeah, I agree.

NPO-197 · 2023-03-22T13:58:17Z

Looks like that did the trick!
I added the following condition for MINGW at the start of the if else block under Janet

################################
# Janet
################################

if(MINGW)
    add_custom_command(
        OUTPUT ${THIRDPARTY_DIR}/janet/build/c/janet.c
        COMMAND mingw32-make build/c/janet.c
        WORKING_DIRECTORY ${THIRDPARTY_DIR}/janet/
    )
elseif(WIN32)
    add_custom_command(
        OUTPUT ${THIRDPARTY_DIR}/janet/build/c/janet.c
        COMMAND ./build_win.bat
        WORKING_DIRECTORY ${THIRDPARTY_DIR}/janet/
    )
else()
    add_custom_command(
        OUTPUT ${THIRDPARTY_DIR}/janet/build/c/janet.c
        COMMAND make build/c/janet.c
        WORKING_DIRECTORY ${THIRDPARTY_DIR}/janet/
    )
endif()

NPO-197 · 2023-03-22T14:03:38Z

So can confirm a change to the cmake file is necessary.
Might also be helpful to change in the readme
from:

run following commands in terminal

to

run following commands in mingw64

sogaiu · 2023-03-22T14:12:10Z

@NPO-197 Hmm, I thought it wasn't necessary here.

I'll check again.

NPO-197 · 2023-03-22T14:15:33Z

Did you delete the libjanet.a file before rebuilding?

sogaiu · 2023-03-22T14:18:00Z

I didn't verify, but I did run mingw32-make clean.

Ah you know, there is a /usr/bin/make in my MSYS2 / MinGW64 environment. I wonder if that's related.

NPO-197 · 2023-03-22T14:20:28Z

Ahh, I don't have that must be why we are getting different errors.

sogaiu · 2023-03-22T14:23:16Z

Then may be it's better to modify CMakeFiles.txt, as you mentioned it works for you.

According to pkgfile output, /usr/bin/make.exe is part of the msys/make package. It's possible it's not part of a default install and that I installed it at some point.

NPO-197 · 2023-03-22T14:36:44Z

I got my distribution of mingw64 from the msys2 pakage which apparently doesn't have msys/make by default. 🤔

sogaiu · 2023-03-22T14:40:31Z

Might also be helpful to change in the readme
from:

run following commands in terminal

to

run following commands in mingw64

Yeah, I think "terminal" can definitely lead to problems. I'm not sure what would be good to replace it with. In my case I used MSYS2's MingW64 shell, but may be it's possible to end up with a MingW64 environment some other way?

I gave up trying to find a way via https://www.mingw-w64.org/, but may be there is one.

NPO-197 · 2023-03-22T14:41:09Z

Oh but looks like the w64devkit one dose! So this would be on me for not using the distribution of mingw64 linked in the read me!

NPO-197 · 2023-03-22T14:48:30Z

Yeah It honestly might be better to just link to https://www.msys2.org/ in the readme instead of to https://www.mingw-w64.org/ and change the cmake for Janet to use mingw32-make on MINGW.

sogaiu · 2023-03-22T15:00:50Z

I agree it is kind of vague as it stands.

I haven't been following TIC-80 for very long and I don't know what else depends on MINGW in detail, nor how other folks install the related bits. May be other folks who have been around longer have a more informed view of the matter.

AlecTroemel · 2023-03-24T14:41:49Z

for the readme wording, maybe we could say

run following commands within a `mingw64` context, for example within a MingW64 shell

sogaiu · 2023-03-24T14:44:33Z

Best idea so far :)

AlecTroemel · 2023-03-30T00:33:51Z

any thoughts on whats going wrong with the workflow here? https://github.com/nesbox/TIC-80/actions/runs/4559234069/jobs/8042966444?pr=2155#step:5:36

I'll admit that I havent spun up a windows vm and tried building locally with mingw.. should probably try that, thought this seems like a ci/cd problem

sogaiu · 2023-03-30T01:30:25Z

any thoughts on whats going wrong with the workflow here? https://github.com/nesbox/TIC-80/actions/runs/4559234069/jobs/8042966444?pr=2155#step:5:36

Puzzling.

Is it correct that some of the output is a result of invoking mingw32-make -j4 (line 4)?

A long shot, but perhaps output is intermingled by parallelization making interpretation harder?

Possibly not using -j might clear things up a bit?

Also did a search for "cmake not able to compile a simple test program" and got hits, but not sure any were helpful.

sogaiu · 2023-03-30T02:16:56Z

I looked at PATH and there an awful lot of things there. Is it perhaps possible that an inappropriate toolchain / set of binaries is being picked up?

Possibly similar phenomena mentioned here: microsoft/LightGBM#3469 -- lots of theories in that issue.

AlecTroemel · 2023-03-30T02:30:25Z

this seems pretty close to what im seeing https://stackoverflow.com/questions/59355908/mingw-c-compiler-not-able-to-compile-a-simple-test-program

sogaiu · 2023-03-30T02:31:48Z

Chocolatey [1] was mentioned in the issue I linked above too.

For your viewing pleasure, below is the PATH, made more readable...

C:\Program Files\MongoDB\Server\5.0\bin
C:\aliyun-cli
C:\vcpkg
C:\cf-cli
C:\Program Files (x86)\NSIS\
C:\tools\zstd
C:\Program Files\Mercurial\
C:\hostedtoolcache\windows\stack\2.9.3\x64
C:\cabal\bin
C:\\ghcup\bin
C:\Program Files\dotnet
C:\mysql\bin
C:\Program Files\R\R-4.2.2\bin\x64
C:\SeleniumWebDrivers\GeckoDriver
C:\Program Files (x86)\sbt\bin
C:\Program Files (x86)\GitHub CLI
C:\Program Files\Git\bin
C:\Program Files (x86)\pipx_bin
C:\npm\prefix
C:\hostedtoolcache\windows\go\1.17.13\x64\bin
C:\hostedtoolcache\windows\Python\3.7.9\x64\Scripts
C:\hostedtoolcache\windows\Python\3.7.9\x64
C:\tools\kotlinc\bin
C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\8.0.362-9\x64\bin
C:\Program Files\ImageMagick-7.1.1-Q16-HDRI
C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin
C:\ProgramData\kind
C:\Program Files\Eclipse Foundation\jdk-8.0.302.8-hotspot\bin
C:\Windows\system32
C:\Windows
C:\Windows\System32\Wbem
C:\Windows\System32\WindowsPowerShell\v1.0\
C:\Windows\System32\OpenSSH\
C:\ProgramData\Chocolatey\bin
C:\Program Files\PowerShell\7\
C:\Program Files\Microsoft\Web Platform Installer\
C:\Program Files\Microsoft SQL Server\130\Tools\Binn\
C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn\
C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\
C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn\
C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn\
C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn\
C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn\
C:\Program Files (x86)\Microsoft SQL Server\150\DTS\Binn\
C:\Program Files (x86)\Microsoft SQL Server\160\DTS\Binn\
C:\Program Files\OpenSSL\bin
C:\Strawberry\c\bin
C:\Strawberry\perl\site\bin
C:\Strawberry\perl\bin
C:\ProgramData\chocolatey\lib\pulumi\tools\Pulumi\bin
C:\Program Files\TortoiseSVN\bin
C:\Program Files\CMake\bin
C:\ProgramData\chocolatey\lib\maven\apache-maven-3.8.7\bin
C:\Program Files\Microsoft Service Fabric\bin\Fabric\Fabric.Code
C:\Program Files\Microsoft SDKs\Service Fabric\Tools\ServiceFabricLocalClusterManager
C:\Program Files\nodejs\
C:\Program Files\Git\cmd
C:\Program Files\Git\mingw64\bin
C:\Program Files\Git\usr\bin
C:\Program Files\GitHub CLI\
c:\tools\php
C:\Program Files (x86)\sbt\bin
C:\SeleniumWebDrivers\ChromeDriver\
C:\SeleniumWebDrivers\EdgeDriver\
C:\Program Files\Amazon\AWSCLIV2\
C:\Program Files\Amazon\SessionManagerPlugin\bin\
C:\Program Files\Amazon\AWSSAMCLI\bin\
C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\bin
C:\Program Files (x86)\Microsoft BizTalk Server\
C:\Program Files\LLVM\bin
C:\Users\runneradmin\.dotnet\tools
C:\Users\runneradmin\.cargo\bin
C:\Users\runneradmin\AppData\Local\Microsoft\WindowsApps

[1] Though it does have some things that scoop doesn't, I have felt the quality of Chocolatey stuff has been too variable for my taste and I have found scoop to have met my needs better. Perhaps there is no choice in this situation though?

sogaiu · 2023-03-30T02:42:40Z

I noticed (lines 36-38):

The C compiler

"C:/ProgramData/chocolatey/lib/mingw/tools/install/mingw32/bin/cc.exe"

I looked in the Windows environment I had success building with and I think there the path has mingw64 in it, not mingw32. Not sure whether this is relevant though as I find the whole mingw* naming quite confusing.

… image?

AlecTroemel · 2023-03-31T18:52:41Z

OK! I've finally got what I think is the pipeline building using mingw, or at least mingw64 under msys2. This required way too many triggers of the pipeline, and stealing from the sdl2 github workflow config. The largest deviation is using the cmake msys makefiles generator instead of the mingw one.. though I think thats ok

https://cmake.org/cmake/help/latest/generator/MSYS%20Makefiles.html#generator:MSYS%20Makefiles

lock into janet 1.27

d959c78

includes support for mingw builds, along with sandbox functionality needed to lock down io

AlecTroemel added 3 commits March 24, 2023 09:23

update janet conf version

5354809

special case the mingw build for janet

18deff0

check for mingw before win32

3c7c275

update verbage for mingw part of readme

7616d14

AlecTroemel added 2 commits March 24, 2023 12:11

try using 32-bit binaries for mingw workflow.

d0990e0

try sdl off

3ba6065

no parallel to help debug

9ac5198

AlecTroemel added 14 commits March 31, 2023 11:03

dont explicitly setup mingw, since it seems to already be in the base…

9522579

… image?

use mysys shell?

69c3c41

try out sys3 setup action

c479438

disable all the other jobs, try different path config

d716aa7

try scrict path type?

7e4a9f6

try manually editing path

ee9f643

no more msys2

25c139a

use msys2 setup and model after their example

0ed2f7a

install some mingw-w64 packages

6d03470

split out make from build

6abb9fd

dont set -G ... just like sdl2?

6bc4657

use msys makefiles generator

1c1d9bd

include sdl

bee3a3f

add the other builds back in

48b8138

AlecTroemel marked this pull request as ready for review March 31, 2023 18:50

nesbox approved these changes Apr 3, 2023

View reviewed changes

nesbox merged commit 31ea041 into nesbox:main Apr 3, 2023

nesbox assigned AlecTroemel Apr 3, 2023

nesbox linked an issue Apr 3, 2023 that may be closed by this pull request

Build with Janet breaks compiling on windows without Visual Studio #2154

Closed

AlecTroemel mentioned this pull request Apr 3, 2023

Build with Janet breaks compiling on windows without Visual Studio #2154

Closed

Uh oh!

Upgrade janet to 1.27, add mingw workflow build #2155

Upgrade janet to 1.27, add mingw workflow build #2155

Uh oh!

Conversation

AlecTroemel commented Mar 22, 2023

Uh oh!

sogaiu commented Mar 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sogaiu commented Mar 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlecTroemel commented Mar 22, 2023

Uh oh!

AlecTroemel commented Mar 22, 2023

Uh oh!

NPO-197 commented Mar 22, 2023

Uh oh!

NPO-197 commented Mar 22, 2023

Uh oh!

sogaiu commented Mar 22, 2023

Uh oh!

sogaiu commented Mar 22, 2023

Uh oh!

NPO-197 commented Mar 22, 2023

Uh oh!

NPO-197 commented Mar 22, 2023

Uh oh!

sogaiu commented Mar 22, 2023

Uh oh!

NPO-197 commented Mar 22, 2023

Uh oh!

NPO-197 commented Mar 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sogaiu commented Mar 22, 2023

Uh oh!

NPO-197 commented Mar 22, 2023

Uh oh!

sogaiu commented Mar 22, 2023

Uh oh!

NPO-197 commented Mar 22, 2023

Uh oh!

sogaiu commented Mar 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NPO-197 commented Mar 22, 2023

Uh oh!

sogaiu commented Mar 22, 2023

Uh oh!

NPO-197 commented Mar 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NPO-197 commented Mar 22, 2023

Uh oh!

sogaiu commented Mar 22, 2023

Uh oh!

AlecTroemel commented Mar 24, 2023

Uh oh!

sogaiu commented Mar 24, 2023

Uh oh!

AlecTroemel commented Mar 30, 2023

Uh oh!

sogaiu commented Mar 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sogaiu commented Mar 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlecTroemel commented Mar 30, 2023

Uh oh!

sogaiu commented Mar 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sogaiu commented Mar 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlecTroemel commented Mar 31, 2023

Uh oh!

sogaiu commented Mar 22, 2023 •

edited

Loading

sogaiu commented Mar 22, 2023 •

edited

Loading

NPO-197 commented Mar 22, 2023 •

edited

Loading

sogaiu commented Mar 22, 2023 •

edited

Loading

NPO-197 commented Mar 22, 2023 •

edited

Loading

sogaiu commented Mar 30, 2023 •

edited

Loading

sogaiu commented Mar 30, 2023 •

edited

Loading

sogaiu commented Mar 30, 2023 •

edited

Loading

sogaiu commented Mar 30, 2023 •

edited

Loading