Skip to content

ExtensionInstaller #498

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Conversation

igorw
Copy link
Contributor

@igorw igorw commented Mar 25, 2012

Add an ExtensionInstaller for installing PHP C extensions from source

This PR simply copies the compiled extensions into vendor/ext. You can then set the extension_dir directive in php.ini to point to that.

It makes a few assumptions about the extensions' package:

  • user has phpize and make in the PATH
  • you can cd into the directory and run phpize && ./configure && make
  • the compiled extensions go into modules (afaik standard for php extensions)

Sample composer.json

{
    "repositories": [
        {
            "type": "package",
            "package": {
                "name": "mkoppanen/php-zmq",
                "type": "extension",
                "version": "1.0.0",
                "source": {
                    "url": "https://github.com/mkoppanen/php-zmq",
                    "type": "git",
                    "reference": "master"
                }
            }
        },
        {
            "packagist": false
        }
    ],
    "require": {
        "mkoppanen/php-zmq": "1.0.0"
    }

TODO

  • windows support
  • docs
  • discuss php.ini generation

@beberlei
Copy link
Contributor

More todos:

  • On multi-php linux setups you need to specify php-config and path to phpize + configure. That should be configurable via composer extra and ENV Variable
  • I think we should at least generate a php.ini which contains all extension= lines. And then you can either symlink that into your php binary config or do whatever in your setup. We should be unopiniated on how this works for clients, but maybe try to find best practices on how to do it.

@wilmoore
Copy link

wilmoore commented May 2, 2012

This could get a bit tricky if you have specified a "--with-config-file-scan-dir"

php --ini
Configuration File (php.ini) Path: /Users/wilmoore/local/php/versions/5.4.0/etc
Loaded Configuration File:         /Users/wilmoore/local/php/versions/5.4.0/etc/php.ini
Scan for additional .ini files in: /Users/wilmoore/local/php/versions/5.4.0/etc/conf.d
Additional .ini files parsed:      /Users/wilmoore/local/php/versions/5.4.0/etc/conf.d/date.timezone.ini,
/Users/wilmoore/local/php/versions/5.4.0/etc/conf.d/pdo_pgsql.ini,
/Users/wilmoore/local/php/versions/5.4.0/etc/conf.d/xdebug.ini

I think we would want to parse all of those files for extension= lines (i.e. not just php.ini)

@bergie
Copy link

bergie commented May 11, 2012

It seems people still find C extensions difficult. In Node.js this isn't a problem as NPM installs both C++ and JavaScript modules in the same way, making the ecosystem a lot healthier.

There are many good reasons to serve something as an extension instead of doing it in PHP, especially when providing bindings to C libraries, or for better performance.

Difficulty of installing C extensions has always held Midgard back, and clearly makes our PHPCR work nearly irrelevant. So if I can help somehow with making extensions installable with Composer, I'd be happy to do things.

@stealth35
Copy link

About Windows, @pierrejoye (link) and I (link) have a lot of PECL, but most of time the extensions has not been testing on Windows and the compilation don't pass (thinking about all C89 standard)
I think we need to create an alternative to phpize, this is an old thing, and some systems like CMake or Waf (or maybe another custom made) could be much simplier

@pierrejoye
Copy link

phpize won't be replaced. We can't create another set of scripts to build PHP extensions.

I have began to port phpize to Windows, it is part of the PHP SDK now.

However, we could easily get the pecl installer code and do a port for composer. Or just write a wrapper for phpize&configure in composer. But we then have to see if the composer json formats is enough (it is not :) for extensions.

About extensions not compiling on Windows, please send patches over :)

Also I really don't want to rely on git repo here for dependencies (like in the example here), it is not sane and should not be done. It is very easy and straightforward to rely on pecl.php.net for that. I'm ready to add any kind of meta data we may need to make composer works with pecl extensions.

@pierrejoye
Copy link

Also it is important to keep in mind that binaries will be available on pecl.php.net (for windows). On unix, it could be done the same way as it is now, using phpize&co wrapper (or pecl install... in the meantime).

@pierrejoye
Copy link

And finally, binary support, on all platforms, would rock. It could be done in a very simple manner using the respective package managers. 90% of the users will rely on that anyway :)

@stealth35
Copy link

About extensions not compiling on Windows, please send patches over :)

Yeah I try, but we are few people who care about this

that binaries will be available on pecl.php.net (for windows).

I'm here, if you need help

@pierrejoye
Copy link

@stealth35 I do care about this, so does my team, send them over :-)

@igorw
Copy link
Contributor Author

igorw commented Jul 4, 2012

@pierrejoye Thanks for getting involved with this ;-)

Also I really don't want to rely on git repo here for dependencies (like in the example here), it is not sane and should not be done. It is very easy and straightforward to rely on pecl.php.net for that. I'm ready to add any kind of meta data we may need to make composer works with pecl extensions.

What is the problem with allowing extensions to be installed from all sources supported by composer? There are already a number of PHP extensions exclusively available on GitHub already. It would make distribution of alpha quality packages a lot easier.

@pierrejoye
Copy link

@igorw impact of unstable, not reviewed, etc. exts are so much higher than PHP codes. This is really not an option for extensions. Also

pecl will provide snapshots support too (any contributors welcome btw :-). That should cover developments needs. Well, writing that down, I realize that there is no big difference between snaps and repo installation :-). However, to define statically a dep against a repo sounds wrong. I would rather define a dep against, say pecl.php.net/someext and then define the min version.

pecl.php.net (or other) can then define the repo if snapshot/repo usage is requested (to test a fix for example). But I fear that it will bring us to what we have now in the pecl installer (very small subset of pear).

@Seldaek
Copy link
Member

Seldaek commented Jul 4, 2012

@pierrejoye regarding the PECL vs GitHub issue. I think there are a few issues that can make this irrelevant. If you look at what we did for packagist, submitting is so easy and maintenance so low that the incentive for people to do it is very high. Obviously given there are way less extensions you may want to keep the manual submission/acceptance loop, but I think it should still be doable to just tag a release on github and have it show up & build on pecl automatically. I also think that composer should still support building straight from github if anyone feels like this is a good idea. It might be for closed source extensions for example that can not be on pecl.

One more issue regarding PECL would be to have an easy list of all packages/versions.

Regarding windows support etc, I discussed this a while back with @bergie and we found the best way would probably just to merge this PR (in some form or another, I haven't looked at the code in a while), which would provide support for boxes that are properly setup, and if the install fails or it's not supported for any reason (windows or missing phpize etc), then it just tells the user they should add the extension in their php themselves, which is more or less what we have now.

In any case, supporting this everywhere is going to be extremely difficult I think. And I think the best we can do is to just compile and drop the extensions in the extension_dir, then add to php.ini if we have rights, or ask the user to do it. Using dl() is simply not an option as @pierrejoye told me, so extensions will have to be system-wide. It's a shame but so it goes.

@pierrejoye
Copy link

@Seldaek There is no pecl vs github, actually there is no <(d)vcs) vs issue per se. However pecl support is a must, I won't and can't help with extension support in composer without pecl (site) support (in one way or another). github for me is only a set of tools, like sf.net or other, pecl is the most widely used site to fetch pecl extensions and where most of the extensions are released. It is a also trusted src by many companies.

One more issue regarding PECL would be to have an easy list of all packages/versions.

Where is the issue? It is already easy to do.

About how this could be done. Well, mergin the PR sounds like a good idea, but I won't do it until the format of the JSON is well defined. I first saw this PR today when @stealth35 mentioned me in his comment, so I will need some time to see if it covers what we need for extensions support.

@stealth35
Copy link

Beyond the github link, there is the PEAR repo no ?

Dummy example :

{
    "repositories": [
        {
            "type": "pear",
            "url": "http://pecl.php.net"
        }
    ],
    "require": {
        "pear-pecl/apc":"3.1.10"
    }
}

@pierrejoye
Copy link

Btw, Pear != pecl Not the same hosts, meta may differ too.
On Jul 4, 2012 5:32 PM, "stealth35" <
reply@reply.github.com>
wrote:

Beyond the github link, there is the PEAR repo no ?

Dummy example :

{
    "repositories": [
        {
            "type": "pear",
            "url": "http://pecl.php.net"
        }
    ],
    "require": {
        "pear-pecl/apc":"3.1.10"
    }
}

Reply to this email directly or view it on GitHub:
#498 (comment)

@stealth35
Copy link

We just need a pecl type repository ;)

{
    "repositories": [
        {
            "type": "pecl",
            "url": "http://pecl.php.net"
        }
    ],
    "require": {
        "pecl-pecl/apc":"3.1.10"
    }
}

@Seldaek
Copy link
Member

Seldaek commented Jul 5, 2012

@pierrejoye another thing for windows, either we need binaries that are built reliably, or we need some one-click-install thing that can be downloaded and that is then able to build binaries. I don't know which is easier to do, but IMO that's the only way to unlock the situation on windows. If more people start using them on windows, we can only hope that authors will start caring a bit more.

@pierrejoye
Copy link

@Seldaek yes, and it is on its way. I'd to work on that once I'm at work. The binaries will be avaible via pecl.php.net directly.

We have to define a format to define the deps (DLL and other exts), along what is proposed here, to make it work with composer smoothly.

@stof
Copy link
Contributor

stof commented Sep 16, 2012

@pierrejoye any news about this ?

@ivank
Copy link

ivank commented Nov 19, 2013

This seems incredibly promising, are there any news about this or something similar?

@Seldaek
Copy link
Member

Seldaek commented Nov 20, 2013

So there is a good news here:

Windows DLLs are now built automatically for every recent PHP version/variant and hosted on pecl, e.g. http://pecl.php.net/package/APCu/4.0.2/windows

That means all we need is to have two install paths: windows and others. The windows one is missing right now, and should just download the dlls into the extension dir. The other platforms are covered as good as we can with the current code I guess.

We could try to add the extension=... line to php.ini as well on all platforms, and if it fails due to permissions warn the user about it. Similarly when downloading the dll if the permissions don't allow it we can just output a warning to the user with the URL to the dll and the place where it should go (don't think we can easily ask for elevated permissions within a php script, but maybe @pierrejoye has a clue there). On unix as well I guess if the permissions are wrong we could output a small one liner that can be executed as root or something like that.

As for removal, I think extensions should be a one way install only thing, because since many projects can require the same on one machine and they can only be installed globally (dl() isn't gonna happen I hear), it means a project can not uninstall one safely.

Another limitation is that the windows support will only work for extensions hosted on PECL. Another point is that probably we would need a PeclRepositroy to be able to scan packages available on Pecl, and then maybe we build that into packagist, or better would be to maybe add a composer compatible repo to the pecl site itself. We could add the repos directly from github to packagist but then we lose the pecl binding and the windows download.. Maybe this can be worked around with an extra value or something, there could be a link to download the binaries in the extra section of the package, and this could optionally be a pecl.php.net/packages/ link where we could infer the rest. This part is not so clear and we need to see what's possible/ideal. Just throwing ideas out there.

@pierrejoye
Copy link

On Wed, Nov 20, 2013 at 12:21 PM, Jordi Boggiano
notifications@github.comwrote:

So there is a good news here:

Windows DLLs are now built automatically for every recent PHP
version/variant and hosted on pecl, e.g.
http://pecl.php.net/package/APCu/4.0.2/windows

That means all we need is to have two install paths: windows and others.
The windows one is missing right now, and should just download the dlls
into the extension dir. The other platforms are covered as good as we can
with the current code I guess.

We could try to add the extension=... line to php.ini as well on all
platforms, and if it fails due to permissions warn the user about it.
Similarly when downloading the dll if the permissions don't allow it we can
just output a warning to the user with the URL to the dll and the place
where it should go (don't think we can easily ask for elevated permissions
within a php script, but maybe @pierrejoye https://github.com/pierrejoyehas a clue there). On unix as well I guess if the permissions are wrong we
could output a small one liner that can be executed as root or something
like that.

Yes, extension_dir should be the right place.

As for removal, I think extensions should be a one way install only thing,
because since many projects can require the same on one machine and they
can only be installed globally (dl() isn't gonna happen I hear), it means
a project can not uninstall one safely.

For both install and uninstall php process(es) have to be restarted. On
update it has to be stopped, replace the DLLs (or so) and started again.
That's one of the reasons why the pecl command is called with "php -n"
(pecl is the pecl installer, part of pear).

Another limitation is that the windows support will only work for
extensions hosted on PECL. Another point is that probably we would need a
PeclRepositroy to be able to scan packages available on Pecl, and then
maybe we build that into packagist, or better would be to maybe add a
composer compatible repo to the pecl site itself. We could add the repos
directly from github to packagist but then we lose the pecl binding and the
windows download.. Maybe this can be worked around with an extra value or
something, there could be a link to download the binaries in the extra
section of the package, and this could optionally be a
pecl.php.net/packages/ link where we could infer the rest. This part is
not so clear and we need to see what's possible/ideal. Just throwing ideas
out there.

About pecl hosted extensions, it only has to be released there, the code
can be everywhere else.

Pierre

@pierrejoye | http://www.libgd.org

@stof
Copy link
Contributor

stof commented Nov 20, 2013

Another limitation is that the windows support will only work for extensions hosted on PECL. Another point is that probably we would need a PeclRepositroy to be able to scan packages available on Pecl, and then maybe we build that into packagist, or better would be to maybe add a composer compatible repo to the pecl site itself.

I would vote for having the composer repository for pecl packages either in the pecl website itself (if the pecl maintainers agree) or as a separate project than Packagist. The logic to get package infos will be different than for the existing Packagist project. the best solution would probably be to have it in pecl itself, so that it does not have a delay due to crawling pecl to update a separate repo. I don't think pecl has a way to register hooks being triggered each time a new pecl release is done

@pierrejoye
Copy link

On Wed, Nov 20, 2013 at 2:43 PM, Christophe Coevoet <
notifications@github.com> wrote:

Another limitation is that the windows support will only work for
extensions hosted on PECL. Another point is that probably we would need a
PeclRepositroy to be able to scan packages available on Pecl, and then
maybe we build that into packagist, or better would be to maybe add a
composer compatible repo to the pecl site itself.

I would vote for having the composer repository for pecl packages either
in the pecl website itself (if the pecl maintainers agree) or as a separate
project than Packagist. The logic to get package infos will be different
than for the existing Packagist project. the best solution would probably
be to have it in pecl itself, so that it does not have a delay due to
crawling pecl to update a separate repo. I don't think pecl has a way to
register hooks being triggered each time a new pecl release is done

Adding such a hook would not be a problem, the code is in our git repo, I
can give some support if anyone is willing to add it (Pierre on freenode).

Pierre

@pierrejoye | http://www.libgd.org

@naderman
Copy link
Member

Closing this in favour of #2898

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants