I don't think this is going to be sufficient - the mime-types gem appears to only support lookups based on file extension.

I don't think this is going to be sufficient - the mime-types gem appears to only support lookups based on file extension.

Which is the only functionality used from searching the code.
Couldn't find any other references to the original gem.
So might work?

See #41750 (comment) for at least one person using that feature.

Which is the only functionality used from searching the code.
Couldn't find any other references to the original gem.
So might work?

The mimemagic gem is also a dependency of marcel which is used by activestorage.

Which is the only functionality used from searching the code.
Couldn't find any other references to the original gem.
So might work?

The mimemagic gem is also a dependency of marcel which itself is used by activestorage.

I saw, the issues tracked rails/marcel#23
So, might need a fix from marcel as well, and then pull that inside this PR?
Cause I don't want to replace every dependency that needs this either :/

So managed to throw out the requirement on mimemagic but this does not resolve the marcel dependency, which also requires the gem :/

Which is the only functionality used from searching the code.
Couldn't find any other references to the original gem.
So might work?

the marcel gem is based on mimemagic (marcel is just a thin shim on top really), and I believe is typing files based on "magic bytes" using mimemagic.

the "magic byte" database is basically what mimemagic provides.

It's not actually safe to type files just based on extension, which may be maliciously changed. There were a whole class of security vulnerabilities involving this a couple years ago, some involving imagemagick.

it actually is pretty important to use "magic byte". But it's unclear if there is an open source non-GPL source of "magic bytes".

This is a pretty big disruptive problem actually.

Yes I looked into it, hence why I've updated the main description of this pull request.
This approach will only work if marcel finds a suitable solution to the problem at hand.

The documetnation for this part of the shrine gem usefully gives us a list of some alternatives.

https://shrinerb.com/docs/plugins/determine_mime_type

There aren't any great ones. One interesting one is:

https://github.com/blackwinter/ruby-filemagic

It also has a "magic byte" database. But it's marked as "no longer maintained", and it has no license information at all, and I'm not sure where it's original magic byte database came from, it's possible it is also unlabelled GPL. Oops, it also has some native C parts, it's not pure ruby, so that doesn't work great either.

Really, the main problem is figuring out if there's a non-viral-open-source source of "magic byte" information at all.

Agreed, and I would love to get some involvement from the Rails core team or Basecamp to find a suitable way to proceed with this. I'm more than happy to make the required pull requests for the involved gems where needed, but I'd like someone to steer this.

it actually is pretty important to use "magic byte". But it's unclear if there is an open source non-GPL source of "magic bytes".

BSD's file(1) uses a BSD-licensed database, I believe

Hi. Maintainer mime-types and mime-types-data chiming in:

• mime-types could use some performance love, but I’m mostly not using Rails these days. There are several proposals out there and @jeremyevans did some work with the prior version to make it better. Because it’s surrounding mime-types-data, it’s more intended to be complete rather than fast.
• mime-types-data can easily be extended to include magic data from a PD, MIT, or BSD source; it would be the work of a couple of days to extend the model (in mime-types) and then we could start accepting PRs that include magic instructions for given mime-types. A little more work could even read something like the BSD data source (like mime-types-data tooling currently reads the IANA MIME database) to incorporate the magic data directly.

I don’t really have the time or expertise to do this, but I can highly recommend building this into mime-types-data as available data and building something on top of this. The real jewel of mime-types is mime-types-data, which can and should be used across the Ruby ecosystem (I’m going to be doing a little work to also make it possible to use this data in the Elixir ecosystem).

Hi. Maintainer mime-types and mime-types-data chiming in:

* mime-types could use some performance love, but I’m mostly not using Rails these days. There are several proposals out there and @jeremyevans did some work with the prior version to make it _better_. Because it’s surrounding `mime-types-data`, it’s more intended to be _complete_ rather than _fast_.

* mime-types-data can _easily_ be extended to include magic data from a PD, MIT, or BSD source; it would be the work of a couple of days to extend the model (in `mime-types`) and then we could start accepting PRs that include magic instructions for given mime-types. A little more work could even read something like the BSD data source (like mime-types-data tooling currently reads the IANA MIME database) to incorporate the magic data directly.

I don’t really have the time or expertise to do this, but I can highly recommend building this into mime-types-data as available data and building something on top of this. The real jewel of mime-types is mime-types-data, which can and should be used across the Ruby ecosystem (I’m going to be doing a little work to also make it possible to use this data in the Elixir ecosystem).

I like this idea and approach actually.
But would require quite some time for me to contribute to it, so need to find a way to allocate time for this :/
Still would like some feedback from the Rails team if this is a path worth going forward it to avoid potential future issues.

Thank you for the pull request. We are working in a solution as we speak. We are taking a different direction. See #41750 (comment).

Thanks for getting in touch.
Looking forward to seeing the solution.

Still, thanks a lot for trying to find a fix @coding-bunny, I hope the whole world appreciates it 😄

I updated mimemagic then issue gone, may be this is temporary fix but i don't want stuck ;) . 'bundle update mimemagic'

I updated mimemagic then issue gone, may be this is temporary fix but i don't want stuck ;) . 'bundle update mimemagic'

Read the discussion and Pull request please. Upgrading to a GPL license is not possible for everyone

Thank you for the pull request. We are working in a solution as we speak. We are taking a different direction. See #41750 (comment).

Thanks for the update @rafaelfranca - will this solution be backwards compatible to older Rails versions also?

Thanks for the update @rafaelfranca - will this solution be backwards compatible to older Rails versions also?

Yes.