TL;DR: Check the list of what is being updated before you let an OS automatically update its programs.
We have a home automation / MythTV / ZoneMinder server with automatic updates disabled. In the process of updating OpenHAB to OpenHAB2, Scott suggested we update everything else while we’re at it. No big, did a quick “dnf update” … got a gig of packages downloaded, waiting for >1400 packages to install, and rebooted.
PHP could not talk to MySQL. At all. ZoneMinder just threw an error saying we didn’t have the PHP MySQL module installed (it worked half an hour ago, so it is INSTALLED). MythWeb completely failed to load – just a white screen. The quick web view of OpenHAB persistence history threw a class not found error.
I checked to see if the extensions were loaded (use the command “print_r(get_loaded_extensions());” in a PHP page) – huh, a LOT of my modules were missing. But there weren’t any useful errors anywhere indicating why.
I modified the php.ini file to show startup errors.
[root@fedora01 conf.modules.d]# grep display_startup_errors /etc/php.ini
; display_startup_errors
display_startup_errors = On
Oooooh, now there are errors! A lot of them. Not particularly useful, but at least a good clue that this isn’t going to go so well for me:
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/pdo.so’ – /usr/lib64/php/modules/pdo.so: undefined symbol: zend_ce_exception in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/mysqlnd.so’ – /usr/lib64/php/modules/mysqlnd.so: undefined symbol: zend_hash_str_del in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/bcmath.so’ – /usr/lib64/php/modules/bcmath.so: undefined symbol: _emalloc_16 in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/bz2.so’ – /usr/lib64/php/modules/bz2.so: undefined symbol: zend_fetch_resource2_ex in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/calendar.so’ – /usr/lib64/php/modules/calendar.so: undefined symbol: _emalloc_32 in Unknown on line 0
PHP Warning: PHP Startup: ctype: Unable to initialize module\nModule compiled with module API=20151012\nPHP compiled with module API=20131226\nThese options need to match\n in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/curl.so’ – /usr/lib64/php/modules/curl.so: undefined symbol: zend_list_close in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/dom.so’ – /usr/lib64/php/modules/dom.so: undefined symbol: zend_ce_exception in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/exif.so’ – /usr/lib64/php/modules/exif.so: undefined symbol: zend_hash_str_exists in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/fileinfo.so’ – /usr/lib64/php/modules/fileinfo.so: undefined symbol: zend_list_close in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/ftp.so’ – /usr/lib64/php/modules/ftp.so: undefined symbol: zend_fetch_resource2 in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/gd.so’ – /usr/lib64/php/modules/gd.so: undefined symbol: zend_list_close in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/gettext.so’ – /usr/lib64/php/modules/gettext.so: undefined symbol: zend_parse_arg_str_slow in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/iconv.so’ – /usr/lib64/php/modules/iconv.so: undefined symbol: _zval_get_string_func in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/json.so’ – /usr/lib64/php/modules/json.so: undefined symbol: _emalloc_56 in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/mbstring.so’ – /usr/lib64/php/modules/mbstring.so: undefined symbol: zend_hash_str_del in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/mysqlnd.so’ – /usr/lib64/php/modules/mysqlnd.so: undefined symbol: zend_hash_str_del in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/phar.so’ – /usr/lib64/php/modules/phar.so: undefined symbol: zend_sort in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/posix.so’ – /usr/lib64/php/modules/posix.so: undefined symbol: _zend_hash_str_update in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/shmop.so’ – /usr/lib64/php/modules/shmop.so: undefined symbol: zend_list_close in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/simplexml.so’ – /usr/lib64/php/modules/simplexml.so: undefined symbol: zend_ce_exception in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/sockets.so’ – /usr/lib64/php/modules/sockets.so: undefined symbol: zend_hash_str_del in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/sqlite3.so’ – /usr/lib64/php/modules/sqlite3.so: undefined symbol: zend_ce_exception in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/sysvmsg.so’ – /usr/lib64/php/modules/sysvmsg.so: undefined symbol: _emalloc_64 in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/sysvsem.so’ – /usr/lib64/php/modules/sysvsem.so: undefined symbol: _emalloc_24 in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/sysvshm.so’ – /usr/lib64/php/modules/sysvshm.so: undefined symbol: zend_list_close in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/tidy.so’ – /usr/lib64/php/modules/tidy.so: undefined symbol: _zend_hash_str_update in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/tokenizer.so’ – /usr/lib64/php/modules/tokenizer.so: undefined symbol: _emalloc_large in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/xml.so’ – /usr/lib64/php/modules/xml.so: undefined symbol: _zend_hash_str_add in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/xmlwriter.so’ – /usr/lib64/php/modules/xmlwriter.so: undefined symbol: _emalloc_16 in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/xsl.so’ – /usr/lib64/php/modules/xsl.so: undefined symbol: dom_node_class_entry in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/mysql.so’ – /usr/lib64/php/modules/mysql.so: undefined symbol: mysqlnd_connect in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/mysqli.so’ – /usr/lib64/php/modules/mysqli.so: undefined symbol: zend_ce_exception in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/pdo_mysql.so’ – /usr/lib64/php/modules/pdo_mysql.so: undefined symbol: mysqlnd_allocator in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/pdo_sqlite.so’ – /usr/lib64/php/modules/pdo_sqlite.so: undefined symbol: php_pdo_unregister_driver in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/wddx.so’ – /usr/lib64/php/modules/wddx.so: undefined symbol: zend_list_close in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/xmlreader.so’ – /usr/lib64/php/modules/xmlreader.so: undefined symbol: dom_node_class_entry in Unknown on line 0
PHP Warning: PHP Startup: Unable to load dynamic library ‘/usr/lib64/php/modules/json.so’ – /usr/lib64/php/modules/json.so: undefined symbol: _emalloc_56 in Unknown on line 0
Turns out DNF installed PHP 7, but didn’t do anything to remove the PHP 5 modules from my Apache configuration:
[root@fedora01 tmp]# cd /etc/httpd/modules
[root@fedora01 modules]# grep php *
Binary file libphp5.so matches
Binary file libphp5-zts.so matches
Binary file libphp7.so matches
Binary file libphp7-zts.so matches
[root@fedora01 modules]# mkdir /tmp/oldphp
[root@fedora01 modules]# mv libphp5* /tmp/oldphp
And remove them from the conf.modules.d too (if you just remove the module files but try to load them in the conf.modules.d … Apache will just fail to load. You could remove them from conf.modules.d … but I don’t want a lot of no-longer-used files sitting there to confuse me in a year or two!)
[root@fedora01 modules]# cd /etc/httpd/conf.modules.d/
[root@fedora01 conf.modules.d]# grep php *
10-php.conf: LoadModule php5_module modules/libphp5.so
10-php.conf: LoadModule php5_module modules/libphp5-zts.so
15-php.conf:# Cannot load both php5 and php7 modules
15-php.conf:<IfModule !mod_php5.c>
15-php.conf: LoadModule php7_module modules/libphp7.so
15-php.conf:<IfModule !mod_php5.c>
15-php.conf: LoadModule php7_module modules/libphp7-zts.so
[root@fedora01 conf.modules.d]# mv 10-php.conf /tmp/oldphp/
Then restart Apache without PHP 5:
root@fedora01 conf.modules.d]# service httpd start
Redirecting to /bin/systemctl start httpd.service
Voila, perfectly functioning web sites. And, yeah, I should probably check the list of “what will be updated” when I update a server. Would save HOURS of reading through strace output to find out old versions were still hanging about.