Назад | Перейти на главную страницу

Gearman слишком много процессов проблема

Я использую Net_Gearman из PECL, Gearmand 1.1.11 и Gearman Manager. Каждый раз, когда я добавляю фоновое задание, я вижу нового воркера без функции или идентификатора в Ggearman-Monitor:

Если я добавлю много сообщений в цикл bash, через некоторое время он станет очень медленным.

for i in $(seq 0 9999); do php Client.php && echo $i; done

Вчера ситуация была еще хуже - в журнале Gearmand было много сообщений об ошибках, касающихся Too many open files и как только я добавил --file-descriptors=49152 как вариант и переключился на 1.1.11 с 1.0.6, эти ошибки исчезли.

Вот это lsof -p $(cat /var/run/gearman/gearmand.pid) вывод:

COMMAND   PID    USER   FD   TYPE             DEVICE SIZE/OFF     NODE NAME
gearmand 2020 gearman  cwd    DIR                8,2     4096        2 /
gearmand 2020 gearman  rtd    DIR                8,2     4096        2 /
gearmand 2020 gearman  txt    REG                8,2  3852472  3672962 /opt/sbin/gearmand
gearmand 2020 gearman  mem    REG                8,2    52120  9961752 /lib/x86_64-linux-gnu/libnss_files-2.15.so
gearmand 2020 gearman  mem    REG                8,2    47680  9961756 /lib/x86_64-linux-gnu/libnss_nis-2.15.so
gearmand 2020 gearman  mem    REG                8,2    97248  9961768 /lib/x86_64-linux-gnu/libnsl-2.15.so
gearmand 2020 gearman  mem    REG                8,2    35680  9961750 /lib/x86_64-linux-gnu/libnss_compat-2.15.so
gearmand 2020 gearman  mem    REG                8,2    92720  9964871 /lib/x86_64-linux-gnu/libz.so.1.2.3.4
gearmand 2020 gearman  mem    REG                8,2   109288 11014600 /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
gearmand 2020 gearman  mem    REG                8,2  1030512  9961759 /lib/x86_64-linux-gnu/libm-2.15.so
gearmand 2020 gearman  mem    REG                8,2  1930616  9964982 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
gearmand 2020 gearman  mem    REG                8,2   382896  9964977 /lib/x86_64-linux-gnu/libssl.so.1.0.0
gearmand 2020 gearman  mem    REG                8,2  1815224  9961748 /lib/x86_64-linux-gnu/libc-2.15.so
gearmand 2020 gearman  mem    REG                8,2    88384  9964865 /lib/x86_64-linux-gnu/libgcc_s.so.1
gearmand 2020 gearman  mem    REG                8,2   962656 11014043 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16
gearmand 2020 gearman  mem    REG                8,2   199600 11016157 /usr/lib/x86_64-linux-gnu/libmemcached.so.11.0.0
gearmand 2020 gearman  mem    REG                8,2    31752  9961755 /lib/x86_64-linux-gnu/librt-2.15.so
gearmand 2020 gearman  mem    REG                8,2    14768  9961763 /lib/x86_64-linux-gnu/libdl-2.15.so
gearmand 2020 gearman  mem    REG                8,2   414280  9183971 /usr/lib/libboost_program_options.so.1.46.1
gearmand 2020 gearman  mem    REG                8,2   283832  9183656 /usr/lib/libevent-2.0.so.5.1.4
gearmand 2020 gearman  mem    REG                8,2   664504 11014432 /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6
gearmand 2020 gearman  mem    REG                8,2   135366  9961757 /lib/x86_64-linux-gnu/libpthread-2.15.so
gearmand 2020 gearman  mem    REG                8,2  3534240  9175810 /usr/lib/libmysqlclient.so.18.1.0
gearmand 2020 gearman  mem    REG                8,2   149280  9961760 /lib/x86_64-linux-gnu/ld-2.15.so
gearmand 2020 gearman    0u   CHR                1,3      0t0     1029 /dev/null
gearmand 2020 gearman    1u   CHR                1,3      0t0     1029 /dev/null
gearmand 2020 gearman    2u   CHR                1,3      0t0     1029 /dev/null
gearmand 2020 gearman    3w   REG                8,2  9381897  3409366 /var/log/gearman-job-server/gearman.log
gearmand 2020 gearman    4r  FIFO                0,8      0t0 38869143 pipe
gearmand 2020 gearman    5w  FIFO                0,8      0t0 38869143 pipe
gearmand 2020 gearman    6u  0000                0,9        0     6826 anon_inode
gearmand 2020 gearman    7u  unix 0xffff880230fdf500      0t0 38869144 socket
gearmand 2020 gearman    8u  unix 0xffff880230fdde40      0t0 38869145 socket
gearmand 2020 gearman    9u  IPv4           38869146      0t0      TCP localhost:4730 (LISTEN)
gearmand 2020 gearman   10r  FIFO                0,8      0t0 38869147 pipe
gearmand 2020 gearman   11w  FIFO                0,8      0t0 38869147 pipe
gearmand 2020 gearman   12u  0000                0,9        0     6826 anon_inode
gearmand 2020 gearman   13u  unix 0xffff880230fde4c0      0t0 38869148 socket
gearmand 2020 gearman   14u  unix 0xffff880230fdeb40      0t0 38869149 socket
gearmand 2020 gearman   15r  FIFO                0,8      0t0 38869150 pipe
gearmand 2020 gearman   16w  FIFO                0,8      0t0 38869150 pipe
gearmand 2020 gearman   17u  0000                0,9        0     6826 anon_inode
gearmand 2020 gearman   18u  0000                0,9        0     6826 anon_inode
gearmand 2020 gearman   19u  unix 0xffff880230fdb400      0t0 38869151 socket
gearmand 2020 gearman   20u  unix 0xffff880230fdaa40      0t0 38869152 socket
gearmand 2020 gearman   21r  FIFO                0,8      0t0 38869153 pipe
gearmand 2020 gearman   22w  FIFO                0,8      0t0 38869153 pipe
gearmand 2020 gearman   23u  unix 0xffff880203cfce00      0t0 38868290 socket
gearmand 2020 gearman   24u  unix 0xffff880203cfdb00      0t0 38868291 socket
gearmand 2020 gearman   25r  FIFO                0,8      0t0 38868292 pipe
gearmand 2020 gearman   26w  FIFO                0,8      0t0 38868292 pipe
gearmand 2020 gearman   27u  0000                0,9        0     6826 anon_inode
gearmand 2020 gearman   28u  unix 0xffff880203cf9040      0t0 38868293 socket
gearmand 2020 gearman   29u  unix 0xffff880203cfaa40      0t0 38868294 socket
gearmand 2020 gearman   30r  FIFO                0,8      0t0 38868295 pipe
gearmand 2020 gearman   31w  FIFO                0,8      0t0 38868295 pipe
gearmand 2020 gearman   32u  IPv4           38868324      0t0      TCP localhost:4730->localhost:57954 (ESTABLISHED)
gearmand 2020 gearman   33u  IPv4           38868325      0t0      TCP localhost:4730->localhost:57955 (ESTABLISHED)
gearmand 2020 gearman   34u  IPv4           38901247      0t0      TCP localhost:4730->localhost:38594 (ESTABLISHED)
gearmand 2020 gearman   35u  IPv4           38868327      0t0      TCP localhost:4730->localhost:57957 (ESTABLISHED)
gearmand 2020 gearman   36u  IPv4           38867483      0t0      TCP localhost:4730->localhost:57959 (ESTABLISHED)
gearmand 2020 gearman   37u  IPv4           38867484      0t0      TCP localhost:4730->localhost:57958 (ESTABLISHED)
gearmand 2020 gearman   38u  IPv4           38901248      0t0      TCP localhost:4730->localhost:38595 (CLOSE_WAIT)
gearmand 2020 gearman   39u  IPv4           38901249      0t0      TCP localhost:4730->localhost:38597 (ESTABLISHED)
gearmand 2020 gearman   40u  IPv4           38869201      0t0      TCP localhost:4730->localhost:57979 (ESTABLISHED)
gearmand 2020 gearman   41u  IPv4           38900437      0t0      TCP localhost:4730->localhost:38599 (ESTABLISHED)
gearmand 2020 gearman   42u  IPv4           38900438      0t0      TCP localhost:4730->localhost:38602 (ESTABLISHED)
gearmand 2020 gearman   43u  IPv4           38868375      0t0      TCP localhost:4730->localhost:57987 (ESTABLISHED)
gearmand 2020 gearman   44u  IPv4           38900468      0t0      TCP localhost:4730->localhost:38606 (CLOSE_WAIT)
gearmand 2020 gearman   45u  IPv4           38868381      0t0      TCP localhost:4730->localhost:57999 (ESTABLISHED)
gearmand 2020 gearman   46u  IPv4           38868388      0t0      TCP localhost:4730->localhost:58007 (ESTABLISHED)
gearmand 2020 gearman   47u  IPv4           38868393      0t0      TCP localhost:4730->localhost:58011 (ESTABLISHED)
gearmand 2020 gearman   48u  IPv4           38903950      0t0      TCP localhost:4730->localhost:38609 (ESTABLISHED)
gearmand 2020 gearman   49u  IPv4           38870276      0t0      TCP localhost:4730->localhost:58019 (ESTABLISHED)
gearmand 2020 gearman   50u  IPv4           38903955      0t0      TCP localhost:4730->localhost:38613 (ESTABLISHED)
gearmand 2020 gearman   51u  IPv4           38900477      0t0      TCP localhost:4730->localhost:38617 (CLOSE_WAIT)
gearmand 2020 gearman   52u  IPv4           38867630      0t0      TCP localhost:4730->localhost:58031 (ESTABLISHED)
gearmand 2020 gearman   53u  IPv4           38867633      0t0      TCP localhost:4730->localhost:58035 (ESTABLISHED)
gearmand 2020 gearman   54u  IPv4           38867636      0t0      TCP localhost:4730->localhost:58039 (ESTABLISHED)
gearmand 2020 gearman   55u  IPv4           38900536      0t0      TCP localhost:4730->localhost:38619 (ESTABLISHED)
gearmand 2020 gearman   56u  IPv4           38868419      0t0      TCP localhost:4730->localhost:58047 (ESTABLISHED)
gearmand 2020 gearman   57u  IPv4           38869263      0t0      TCP localhost:4730->localhost:58051 (ESTABLISHED)
gearmand 2020 gearman   58u  IPv4           38900537      0t0      TCP localhost:4730->localhost:38621 (ESTABLISHED)
gearmand 2020 gearman   59u  IPv4           38869271      0t0      TCP localhost:4730->localhost:58059 (ESTABLISHED)
gearmand 2020 gearman   60u  IPv4           38900538      0t0      TCP localhost:4730->localhost:38623 (ESTABLISHED)
gearmand 2020 gearman   61u  IPv4           38870319      0t0      TCP localhost:4730->localhost:58067 (ESTABLISHED)
gearmand 2020 gearman   62u  IPv4           38900540      0t0      TCP localhost:4730->localhost:38628 (ESTABLISHED)
gearmand 2020 gearman   63u  IPv4           38869289      0t0      TCP localhost:4730->localhost:58075 (ESTABLISHED)
...
gearmand 2020 gearman 2229u  IPv4           38903885      0t0      TCP localhost:4730->localhost:38572 (ESTABLISHED)
gearmand 2020 gearman 2230u  IPv4           38901211      0t0      TCP localhost:4730->localhost:38576 (ESTABLISHED)
gearmand 2020 gearman 2234u  IPv4           38901237      0t0      TCP localhost:4730->localhost:38588 (ESTABLISHED)

ubuntu gearman