Мы используем apache http server 2.2.29 в Linux. Ниже приведена информация о версии Linux (находится в файле / proc / version). Linux version 2.6.32-573.el6.x86_64 (mockbuild@x86-027.build.eng.bos.redhat.com) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-16) (GCC) ) #1 SMP Wed Jul 1 18:23:37 EDT 2015
Мы заметили, что apache создает множество дампов ядра и регулярно дает сбой. Когда он генерирует дамп ядра, в журнале ошибок http печатается следующее:
[Mon Jun 20 01:28:09 2016] [notice] child pid 6520 exit signal Segmentation fault (11), possible coredump in /app/http_instance_name
[Mon Jun 20 03:27:54 2016] [notice] child pid 6462 exit signal Segmentation fault (11), possible coredump in /app/http_instance_name
Ниже приведены файлы ядра, созданные при этих ошибках сегментации.
-rw------- 1 http http 65130496 Jun 20 01:28 core.6520
-rw------- 1 http http 66134016 Jun 20 03:27 core.6462
Мы работаем с http-версией Apache ниже:
Server version: Apache/2.2.29 (Unix)
Server built: Mar 4 2016 15:21:54
Я попытался проанализировать основные файлы с помощью инструмента gdb, однако я не смог многое из этого понять?
Пожалуйста, дайте мне знать, если потребуется дополнительная информация, которая поможет лучше разобраться в этом вопросе.
может это быть связано с некоторыми модулями, используемыми в файле httpd.conf? Мы наблюдаем эту проблему во всех наших средах.
ОБНОВЛЕНИЕ: Общий шаблон
Наиболее распространенный паттерн, который я наблюдаю при возникновении ошибки сегментации, заключается в том, что приведенная ниже ошибка возникает в большинстве случаев непосредственно перед появлением ошибки сегментации.
[Mon May 16 16:08:02 2016] [error] [client 10.10.12.113] (-2)Unknown error 18446744073709551614: proxy: error reading status line from remote server rxxxx.xxxx.net:8543, referer: https://sit1-xxxxxx.co.uk:8443/CommonWeb/pages/common/jsp/login.jsp
[Mon May 16 16:08:02 2016] [error] [client 10.10.12.113] proxy: Error reading from remote server returned by /CommonWeb/pages/common/includes/js/dojo.js, referer: https://sit1.xxxxxx:8443/CommonWeb/pages/common/jsp/login.jsp
[Mon May 16 16:08:02 2016] [error] [client 10.10.12.113] (-2)Unknown error 18446744073709551614: proxy: error reading status line from remote server rpovassxxxx.srxxx.net:8543, referer: https://sit1-****:8443/CommonWeb/pages/common/jsp/login.jsp
[Mon May 16 16:08:02 2016] [error] [client 10.10.12.113] proxy: Error reading from remote server returned by /CommonWeb/pages/common/includes/js/resize.js, referer: https://xxxx-xxxx:8443/CommonWeb/pages/common/jsp/login.jsp
[Mon May 16 16:08:03 2016] [notice] child pid 28308 exit signal Segmentation fault (11), possible coredump in /app/
Ниже показан результат "gdb full" во время отладки:
#0 0x0000003704a0e7dd in read () from /lib64/libpthread.so.0
No symbol table info available.
#1 0x000000000044f8b7 in ap_mpm_pod_check (pod=<value optimized out>) at pod.c:54
c = <value optimized out>
fd = 6
rc = <value optimized out>
#2 0x000000000044de74 in child_main (child_num_arg=1) at worker.c:1259
threads = 0xc2bab0
rv = <value optimized out>
ts = 0xc079f8
thread_attr = 0xc07a18
start_thread_id = <value optimized out>
#3 0x000000000044e0dd in make_child (s=0xb44870, slot=1) at worker.c:1342
pid = 0
#4 0x000000000044efc0 in perform_idle_server_maintenance (_pconf=<value optimized out>, plog=<value optimized out>, s=<value optimized out>)
at worker.c:1568
j = <value optimized out>
free_slots = {1, 5, 6, 7, 11817336, 0, 11784504, 0, -1236161560, 11137, 11817336, 0, 11800920, 0, 11815024, 0, 11784504, 0, -1361903789, 11137,
4337480, 0, -1238262800, 11137, 29, 0, 12768544, 0, 11815024, 0, 12489224, 0}
total_non_dead = <value optimized out>
active_thread_count = <value optimized out>
idle_thread_count = <value optimized out>
free_length = 1
i = <value optimized out>
ps = <value optimized out>
totally_free_length = <value optimized out>
last_non_dead = <value optimized out>
#5 server_main_loop (_pconf=<value optimized out>, plog=<value optimized out>, s=<value optimized out>) at worker.c:1680
status = 11
pid = {pid = -1, in = 0xb45178, out = 0xb41158, err = 0xb3d138}
i = <value optimized out>
child_slot = <value optimized out>
exitwhy = 6
processed_status = <value optimized out>
#6 ap_mpm_run (_pconf=<value optimized out>, plog=<value optimized out>, s=<value optimized out>) at worker.c:1782
remaining_children_to_start = <value optimized out>
rv = <value optimized out>
#7 0x0000000000425e34 in main (argc=3, argv=0x7ffcafb44eb8) at main.c:753
c = 0 '\000'
configtestonly = <value optimized out>
confname = 0x458a64 "conf/httpd.conf"
def_server_root = 0x458a4f "/app/***"
temp_error_log = 0x0
error = <value optimized out>
process = 0xb44870
server_conf = 0xb44870
pglobal = 0xb3b128
pconf = 0xb3d138
plog = 0xb45178
ptemp = 0xb41158
pcommands = 0xb3f148
opt = 0xb3f238
rv = <value optimized out>
mod = <value optimized out>
optarg = 0x0
signal_server = <value optimized out>'
В файле httpd.conf используются следующие модули.
`cat httpd.conf | grep LoadMo
# необходимо разместить соответствующие LoadModule' lines at this location so the
# LoadModule foo_module modules/mod_foo.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authn_dbm_module modules/mod_authn_dbm.so
LoadModule authn_anon_module modules/mod_authn_anon.so
LoadModule authn_dbd_module modules/mod_authn_dbd.so
LoadModule authn_default_module modules/mod_authn_default.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_dbm_module modules/mod_authz_dbm.so
LoadModule authz_owner_module modules/mod_authz_owner.so
LoadModule authz_default_module modules/mod_authz_default.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule auth_digest_module modules/mod_auth_digest.so
LoadModule dbd_module modules/mod_dbd.so
LoadModule dumpio_module modules/mod_dumpio.so
LoadModule reqtimeout_module modules/mod_reqtimeout.so
LoadModule ext_filter_module modules/mod_ext_filter.so
LoadModule include_module modules/mod_include.so
LoadModule filter_module modules/mod_filter.so
LoadModule substitute_module modules/mod_substitute.so
LoadModule deflate_module modules/mod_deflate.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule log_forensic_module modules/mod_log_forensic.so
LoadModule logio_module modules/mod_logio.so
LoadModule mime_magic_module modules/mod_mime_magic.so
LoadModule cern_meta_module modules/mod_cern_meta.so
LoadModule expires_module modules/mod_expires.so
LoadModule headers_module modules/mod_headers.so
LoadModule ident_module modules/mod_ident.so
LoadModule usertrack_module modules/mod_usertrack.so
LoadModule unique_id_module modules/mod_unique_id.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule version_module modules/mod_version.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_scgi_module modules/mod_proxy_scgi.so
LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
#LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule ssl_module modules/mod_ssl.so
LoadModule mime_module modules/mod_mime.so
LoadModule dav_module modules/mod_dav.so
LoadModule status_module modules/mod_status.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule info_module modules/mod_info.so
LoadModule cgid_module modules/mod_cgid.so
LoadModule cgi_module modules/mod_cgi.so
LoadModule dav_fs_module modules/mod_dav_fs.so
LoadModule vhost_alias_module modules/mod_vhost_alias.so
LoadModule negotiation_module modules/mod_negotiation.so
LoadModule dir_module modules/mod_dir.so
LoadModule imagemap_module modules/mod_imagemap.so
LoadModule actions_module modules/mod_actions.so
LoadModule speling_module modules/mod_speling.so
LoadModule userdir_module modules/mod_userdir.so
LoadModule alias_module modules/mod_alias.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule auth_kerb_module modules/mod_auth_kerb.so
LoadModule manager_module modules/mod_manager.so
LoadModule slotmem_module modules/mod_slotmem.so
LoadModule proxy_cluster_module modules/mod_proxy_cluster.so
LoadModule advertise_module modules/mod_advertise.so
Ниже приведен вывод httpd -V (заглавная V)
./httpd -V
Server version: Apache/2.2.29 (Unix)
Server built: Jan 13 2016 15:10:57
Server's Module Magic Number: 20051115:36
Server loaded: APR 1.5.1, APR-Util 1.5.3
Compiled using: APR 1.5.1, APR-Util 1.5.3
Architecture: 64-bit
Server MPM: Worker
threaded: yes (fixed thread count)
forked: yes (variable process count)
Server compiled with....
-D APACHE_MPM_DIR="server/mpm/worker"
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_SYSVSEM_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=128
-D HTTPD_ROOT="/app/****"
-D SUEXEC_BIN="/app/****/bin/suexec"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"
** ВЫВОД файла * **
httpd.exp: script text executable for .
mod_actions.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_advertise.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_alias.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_auth_basic.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_auth_digest.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_auth_kerb.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_authn_anon.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_authn_dbd.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_authn_dbm.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_authn_default.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_authn_file.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_authz_dbm.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_authz_default.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_authz_groupfile.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_authz_host.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_authz_owner.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_authz_user.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_autoindex.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_cern_meta.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_cgid.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_cgi.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_dav_fs.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_dav.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_dbd.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_deflate.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_dir.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_dumpio.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_expires.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_ext_filter.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_filter.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_headers.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_ident.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_imagemap.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_include.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_info.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_log_config.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_log_forensic.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_logio.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_manager.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_mime_magic.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_mime.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_negotiation.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_proxy_ajp.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_proxy_balancer.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_proxy_cluster.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_proxy_connect.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_proxy_ftp.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_proxy_http.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_proxy_scgi.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_proxy.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_reqtimeout.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_rewrite.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_setenvif.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_slotmem.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_speling.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_ssl.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_status.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_substitute.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_unique_id.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_userdir.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_usertrack.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_version.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
mod_vhost_alias.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, not stripped
Для использования GDB https://httpd.apache.org/dev/debugging.html.
Вы сможете отследить истинную причину ошибки.
Если сбой трудно воспроизвести, может быть хорошей идеей настроить Apache на использование только одного дочернего процесса для обработки запросов. Конфигурация примерно такая:
StartServers 1
MinSpareServers 1
MaxSpareServers 1
Проверьте загруженный файл конфигурации - параметры php.ini. Настройте его в соответствии с конфигурацией сервера (H / W ресурсы)
Проверьте, используется ли какой-либо метод кэширования страниц, например APC или memcached, если да, то выделите ресурсы соответственно.