Skip to content

Conversation

iofq
Copy link
Contributor

@iofq iofq commented May 19, 2025

Adding katrain as it's becoming difficult to build on a modern Linux system such as ubuntu and nix could help :)

To do so, I had to add 2 new python modules, kivymd and ffpyplayer. ffpyplayer was difficult because the project needs ffmpeg C bindings; after troubleshooting for a few hours, I still wasn't able to get it to compile against any version of ffmpeg, and fell back to the wheel from pypi (which appears to be built with ffmpeg 6).

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • ** Only tested x86_64, do not have a darwin machine **
  • Nixpkgs 25.11 Release Notes (or backporting 24.11 and 25.05 Nixpkgs Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
  • NixOS 25.11 Release Notes (or backporting 24.11 and 25.05 NixOS Release notes)
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added 6.topic: python Python is a high-level, general-purpose programming language. 8.has: maintainer-list (update) This PR changes `maintainers/maintainer-list.nix` labels May 19, 2025
@nix-owners nix-owners bot requested a review from natsukium May 19, 2025 04:17
@github-actions github-actions bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels May 19, 2025
Copy link
Contributor

@pilz0 pilz0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks okay to me

x86_64-linux

✅ 10 packages built:
  • katrain
  • katrain.dist
  • python312Packages.ffpyplayer
  • python312Packages.ffpyplayer.dist
  • python312Packages.kivymd
  • python312Packages.kivymd.dist
  • python313Packages.ffpyplayer
  • python313Packages.ffpyplayer.dist
  • python313Packages.kivymd
  • python313Packages.kivymd.dist

version = "v4.5.2";
format = "wheel";

src = fetchurl {

This comment was marked as resolved.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense, easier to bump versions in the future. updated + linted commits

else
"sha256-+6aibWImMvNe6GNP/JSos0QQJq8NZ7X4v+MwpC4BPFw=";
};

Copy link
Contributor

@pilz0 pilz0 May 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im pretty sure it would fail to build on linux/aarch64, so it would be good to build in support for arm64, with something like this. I will later test if it builds on aarch64.

    url =
      if stdenv.isDarwin then
        "https://github.com/matham/ffpyplayer/releases/download/v${version}/ffpyplayer-${version}-cp313-cp313-macosx_10_13_universal2.whl"
      if stdenv.isAarch64 then
        "/link/to/arch64.whl"
      else
        "https://github.com/matham/ffpyplayer/releases/download/v${version}/ffpyplayer-${version}-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl";

    hash =
      if stdenv.isDarwin then
        "sha256-mPvggPi8KbOCuLqHxxHPcvqSt3BGWVgsLlTIFU8mlsU="
      if stdenv.isAarch64 then
        "sha256-Foooooooooooooooooooooooooooooooooooooooo="

      else
        "sha256-+6aibWImMvNe6GNP/JSos0QQJq8NZ7X4v+MwpC4BPFw=";
  };

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfortunately the project doesn't offer linux arm64 wheels, and they have a custom CI setup to build deps that I haven't had luck replicating in nix and would rather not chase. I could open an issue over there to start supporting arm64 wheels and circle back to nixpkgs when that's complete, if that's an acceptable workaround?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm working on a PR to add arm64 wheels over at ffpyplayer and will circle back here when those are available.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pilz0 I've added ffpyplayer arm64 wheels upstream and referenced them in this PR, would you like to take another look?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, thanks for adding arm wheel builds upstream, im going to to take a look at the PR later today. Im also pretty new to nixpkgs so i cant find all issues.

@wegank wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jun 9, 2025
@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jun 11, 2025
@iofq iofq force-pushed the init_katrain branch 3 times, most recently from 64c7c9c to 9bd2692 Compare June 11, 2025 07:46
@pilz0
Copy link
Contributor

pilz0 commented Jun 11, 2025

Hey, ive looked over the PR again, and at least for me KivyMD doesnt work (amd64). ffpyplayer seems to work on both arm and amd64. Otherwise i haved noticed any issues :)

@iofq
Copy link
Contributor Author

iofq commented Jun 12, 2025

Hey, ive looked over the PR again, and at least for me KivyMD doesnt work (amd64). ffpyplayer seems to work on both arm and amd64. Otherwise i haved noticed any issues :)

Hey, can you describe what exactly is not working for kivymd? For me it builds and katrain can use it fine.

@pilz0
Copy link
Contributor

pilz0 commented Jun 13, 2025

Hey, ive looked over the PR again, and at least for me KivyMD doesnt work (amd64). ffpyplayer seems to work on both arm and amd64. Otherwise i haved noticed any issues :)

Hey, can you describe what exactly is not working for kivymd? For me it builds and katrain can use it fine.

Im getting this error when im running katrain via nixpkgs-review

[ERROR  ] [Clock       ] Unable to import kivy._clock. Have you perhaps forgotten to compile kivy? Kivy contains Cython code which needs to be compiled. A missing kivy._clock often indicates the Cython code has not been compiled. Please follow the installation instructions and make sure to compile Kivy
 Traceback (most recent call last):
   File "/nix/store/f61x1pl8rhhwv10ddj5c2jcn864k1dik-katrain-v1.16/bin/.katrain-wrapped", line 6, in <module>
     from katrain.__main__ import run_app
   File "/nix/store/f61x1pl8rhhwv10ddj5c2jcn864k1dik-katrain-v1.16/lib/python3.12/site-packages/katrain/__main__.py", line 49, in <module>
     from kivy.base import ExceptionHandler, ExceptionManager
   File "/nix/store/g5na9p8x5r0qfyz98rq1n3rbgbfw5xxf-python3.13-kivy-2.3.1/lib/python3.13/site-packages/kivy/base.py", line 28, in <module>
     from kivy.clock import Clock
   File "/nix/store/g5na9p8x5r0qfyz98rq1n3rbgbfw5xxf-python3.13-kivy-2.3.1/lib/python3.13/site-packages/kivy/clock.py", line 466, in <module>
     from kivy._clock import CyClockBase, ClockEvent, FreeClockEvent, \
 ModuleNotFoundError: No module named 'kivy._clock

@pilz0
Copy link
Contributor

pilz0 commented Jun 13, 2025

Python gives me this error:

Python 3.12.10 (main, Apr  8 2025, 11:35:47) [GCC 14.2.1 20250322] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import kivymd
[INFO   ] [Logger      ] Record log in /home/marie/.kivy/logs/kivy_25-06-13_1.txt
[INFO   ] [Kivy        ] v2.3.1
[INFO   ] [Kivy        ] Installed at "/nix/store/g5na9p8x5r0qfyz98rq1n3rbgbfw5xxf-python3.13-kivy-2.3.1/lib/python3.13/site-packages/kivy/__init__.py"
[INFO   ] [Python      ] v3.12.10 (main, Apr  8 2025, 11:35:47) [GCC 14.2.1 20250322]
[INFO   ] [Python      ] Interpreter at "/nix/store/8w718rm43x7z73xhw9d6vh8s4snrq67h-python3-3.12.10/bin/python"
[INFO   ] [Logger      ] Purge log fired. Processing...
[INFO   ] [Logger      ] Purge finished!
[INFO   ] [KivyMD      ] 1.1.1, git-Unknown, 2022-10-13 (installed at "/nix/store/5prvh59y4rp41bsvqyv3h0kdrpi2i5by-python3.13-kivymd-1.1.1/lib/python3.13/site-packages/kivymd/__init__.py")
[INFO   ] [Factory     ] 195 symbols loaded
 Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "/nix/store/5prvh59y4rp41bsvqyv3h0kdrpi2i5by-python3.13-kivymd-1.1.1/lib/python3.13/site-packages/kivymd/__init__.py", line 66, in <module>
     import kivymd.font_definitions  # NOQA
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   File "/nix/store/5prvh59y4rp41bsvqyv3h0kdrpi2i5by-python3.13-kivymd-1.1.1/lib/python3.13/site-packages/kivymd/font_definitions.py", line 10, in <module>
     from kivy.core.text import LabelBase
   File "/nix/store/g5na9p8x5r0qfyz98rq1n3rbgbfw5xxf-python3.13-kivy-2.3.1/lib/python3.13/site-packages/kivy/core/text/__init__.py", line 85, in <module>
     from kivy.graphics.texture import Texture
   File "/nix/store/g5na9p8x5r0qfyz98rq1n3rbgbfw5xxf-python3.13-kivy-2.3.1/lib/python3.13/site-packages/kivy/graphics/__init__.py", line 89, in <module>
     from kivy.graphics.instructions import Callback, Canvas, CanvasBase, \
 ModuleNotFoundError: No module named 'kivy.graphics.instructions'

@iofq iofq force-pushed the init_katrain branch 2 times, most recently from 41a2cbc to e041be4 Compare June 17, 2025 05:03
@iofq
Copy link
Contributor Author

iofq commented Jun 17, 2025

  • Fixed kivymd dependencies (needed kivy)
  • Added new python packages asyncgui and asynckivy for kivymd dependencies
  • Added optional Darwin dependencies to katrain (to match their pyproject.toml)
  • Added tests/pythonImportsCheck where appropriate. Many tests involving kivy do not run due to lack of video output
  • Added katrain nixos module to handle patching the katrain source with a nix store path to katago. There are multiple katago packages available (CPU, OpenCL, etc.) so it made sense to make it configurable.
  • Updated commits to match nixpkgs convention

@github-actions github-actions bot added 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` and removed 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Jun 17, 2025
@iofq iofq force-pushed the init_katrain branch 3 times, most recently from 4947c5f to 194b639 Compare June 17, 2025 05:49
@iofq iofq requested a review from pilz0 June 20, 2025 01:01
@pilz0
Copy link
Contributor

pilz0 commented Jun 20, 2025

Katrain currently fails to build :/ (https://github.com/NixOS/nixpkgs/pull/408590/checks?check_run_id=44305902174)

@iofq iofq force-pushed the init_katrain branch 2 times, most recently from e5548e1 to 728884f Compare June 21, 2025 02:13
@iofq
Copy link
Contributor Author

iofq commented Jun 21, 2025

Katrain currently fails to build :/ (https://github.com/NixOS/nixpkgs/pull/408590/checks?check_run_id=44305902174)

Sorry about that! All is well in nixpkgs-review now

@EinEtw4s
Copy link

EinEtw4s commented Jun 21, 2025

builds for me now, too, without error

the import error as seen in #408590 (comment) does still persist, though

@iofq iofq force-pushed the init_katrain branch 3 times, most recently from d93302f to 47b8ae6 Compare July 22, 2025 07:02
@iofq iofq force-pushed the init_katrain branch 3 times, most recently from 9352efb to 9f4ed0e Compare July 23, 2025 02:35
iofq added 2 commits July 22, 2025 21:45
Adds a module for pkgs.katrain to govern katago patching behavior and additional dependencies
@iofq
Copy link
Contributor Author

iofq commented Jul 23, 2025

Hello @EinEtw4s @pilz0,

Can you describe how you're reaching the import error in #408590 (comment)?

I'm using the following flake to test importing each python package without issue:

{
  inputs = {
    nixpkgs.url = "github:iofq/nixpkgs/init_katrain";
  };

  outputs = {
    self,
    nixpkgs,
  }: {
    devShells.x86_64-linux.default = let
      pkgs = import nixpkgs {
        system = "x86_64-linux";
      };
    in
      pkgs.mkShell {
        packages = [
          pkgs.katrain
          (
            pkgs.python3.withPackages
            (ps:
              with ps; [
                kivymd
                asyncgui
                asynckivy
                ffpyplayer
              ])
          )
        ];
      };
  };
}
iofq@nix ~/d/nixreview> nix develop --command python -c "import asyncgui,asynckivy,kivymd,ffpyplayer"
[INFO   ] [Kivy        ] v2.3.1
[INFO   ] [Kivy        ] Installed at "/nix/store/a0y8hih1b2c2n0jpy7agk3kxrbfdikgk-python3.13-kivy-2.3.1/lib/python3.13/site-packages/
kivy/__init__.py"
[INFO   ] [Python      ] v3.13.5 (main, Jun 11 2025, 15:36:57) [GCC 14.3.0]
[INFO   ] [Python      ] Interpreter at "/nix/store/djck7mx6jad1w0yy6zings96dyxanls6-python3-3.13.5/bin/python"
[INFO   ] [Logger      ] Purge log fired. Processing...
[INFO   ] [Logger      ] Purge finished!
[INFO   ] [Image       ] Providers: img_tex, img_dds, img_sdl2, img_pil (img_ffpyplayer ignored)
[INFO   ] [Factory     ] 195 symbols loaded
[INFO   ] [Window      ] Provider: sdl2
[INFO   ] [GL          ] Using the "OpenGL" graphics system
[INFO   ] [GL          ] Backend used <sdl2>
[INFO   ] [GL          ] OpenGL version <b'4.6 (Compatibility Profile) Mesa 25.1.1'>
[INFO   ] [GL          ] OpenGL vendor <b'AMD'>
[INFO   ] [GL          ] OpenGL renderer <b'AMD Radeon Graphics (radeonsi, renoir, ACO, DRM 3.63, 6.15.0)'>
[INFO   ] [GL          ] OpenGL parsed version: 4, 6
[INFO   ] [GL          ] Shading version <b'4.60'>
[INFO   ] [GL          ] Texture max size <16384>
[INFO   ] [GL          ] Texture max units <32>
[INFO   ] [Window      ] auto add sdl2 input provider
[INFO   ] [Window      ] virtual keyboard not allowed, single mode, not docked
[INFO   ] [KivyMD      ] v0.104.1

@nixpkgs-ci nixpkgs-ci bot added 8.has: changelog This PR adds or changes release notes 8.has: documentation This PR adds or changes documentation labels Jul 23, 2025
@iofq iofq requested a review from EinEtw4s August 9, 2025 21:06
@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Aug 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.status: merge conflict This PR has merge conflicts with the target branch 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: python Python is a high-level, general-purpose programming language. 8.has: changelog This PR adds or changes release notes 8.has: documentation This PR adds or changes documentation 8.has: maintainer-list (update) This PR changes `maintainers/maintainer-list.nix` 8.has: module (update) This PR changes an existing module in `nixos/` 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants