Как я могу дополнительно проанализировать использование плунжера при возможной утечке? [debian @ HyperV + Kafka + Zookeeper]
У нас есть кластер из трех блоков Kafka + Zookeeper в виртуализированной среде (в Windows 2019 HyperV).
В конфигурации машин динамическая память отключена, и у каждой из них есть ОЗУ 4G (я знаю, что это очень мало для такого типа кластера, но использование очень легкое)
- Гостевой SO - это Debian stretch 9.3, ядро 4.9.0-5-amd64
- Кафка - это 2.11-2.3.0
- Zookeeper - 3.4.9
После запуска кластера или перезапуска узла использование оперативной памяти становится нормальным:
# free -m
total used free shared buff/cache available
Mem: 3937 2200 198 40 1538 1462
Swap: 1903 0 1903
Через неделю использование плунжера нормализуется (использование плунжера 50%). Но через 2 недели барабан начинает потихоньку сливаться. До тех пор, пока несколько дней спустя, когда процент физического свободного доступа не достигнет 15, он резко (и быстро) упадет, начав съедать своп, пока он не будет полностью выделен.
# free -m
total used free shared buff/cache available
Mem: 3937 3744 117 3 76 39
Swap: 1903 1197 706
Я установил smem, и здесь вы можете увидеть разницу между двумя ситуациями:
(НОТА: правильно и неправильно - разные машины, но конфигурация и поведение во времени такие же)
RIGHT:
# smem -tw
Area Used Cache Noncache
firmware/hardware 0 0 0
kernel image 0 0 0
kernel dynamic memory 2777144 1983260 793884
userspace memory 1027116 44548 982568
free memory 228016 228016 0
----------------------------------------------------------
4032276 2255824 1776452
WRONG:
# smem -tw
Area Used Cache Noncache
firmware/hardware 0 0 0
kernel image 0 0 0
kernel dynamic memory 2763772 64968 2698804
userspace memory 1138972 35928 1103044
free memory 129532 129532 0
----------------------------------------------------------
4032276 230428 3801848
Большая часть памяти находится в динамической памяти ядра / Noncache, я не уверен, что это означает и по какой причине.
В ходе мониторинга мы увидели, что (в / etc / meminfo):
- DirectMap4k растет
- MemFree резко убывает, когда начинает жрать своп
- MemAvailable медленно уменьшается
- Большинство других уменьшаются
Вот фото нашего инструмента мониторинга:
Добавление Ram / Swap для всех процессов:
RIGHT:
# for file in /proc/*/status ; do awk -F ' ' '/VmRSS/ {print $2}' $file; done | awk '{sum += $1} END {print sum/1024 "Mb"}'
1053.5Mb
# for file in /proc/*/status ; do awk -F ' ' '/VmSwap/ {print $2}' $file; done | awk '{sum += $1} END {print sum/1024 "Mb"}'
0Mb
WRONG:
# for file in /proc/*/status ; do awk -F ' ' '/VmRSS/ {print $2}' $file; done | awk '{sum += $1} END {print sum/1024 "Mb"}'
308.941Mb
# for file in /proc/*/status ; do awk -F ' ' '/VmSwap/ {print $2}' $file; done | awk '{sum += $1} END {print sum/1024 "Mb"}'
1147.54Mb
top -c, отсортировано по Mem:
RIGHT:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
674 kafka 20 0 17.229g 818152 19956 S 12.1 20.3 3304:07 java -Xmx2048m -Xms512m -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesCo+
734 zookeep+ 20 0 4838708 172708 16736 S 0.0 4.3 81:41.62 java -Dzookeeper.log.dir=/var/log/zookeeper -Dzookeeper.root.logger=INFO,CONSOLE -cp /opt/zookeeper/bin/../build/classes:/opt/zooke+
676 Debian-+ 20 0 58368 10636 6236 S 0.0 0.3 16:08.80 /usr/sbin/snmpd -LSwd -Lf /dev/null -u Debian-snmp -g Debian-snmp -p /run/snmpd.pid -f
1 root 20 0 57124 6900 5320 S 0.0 0.2 34:29.32 /sbin/init
116398 root 20 0 95168 6812 5840 S 0.3 0.2 0:00.34 sshd: root@pts/0
WRONG:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2732 kafka 20 0 17.269g 235004 0 S 6.9 5.8 1259:50 java -Xmx2048m -Xms512m -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesCo+
707 root 20 0 1955516 20720 6896 S 2.8 0.5 52:04.15 /usr/share/filebeat/bin/filebeat -environment systemd -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /et+
2537 zookeep+ 20 0 4437272 8944 0 S 0.0 0.2 20:29.38 java -Dzookeeper.log.dir=/var/log/zookeeper -Dzookeeper.root.logger=INFO,CONSOLE -cp /opt/zookeeper/bin/../build/classes:/opt/zooke+
1 root 20 0 57292 3688 2704 S 0.0 0.1 29:28.41 /sbin/init
При этом я был бы признателен, если бы кто-нибудь мог указать мне, как получить больше доказательств, чтобы определить первопричину этого.
Спасибо.
Изменить (2020/06/26 Добавить несколько выходов):
RIGHT:
# cat /proc/meminfo
MemTotal: 4032276 kB
MemFree: 155480 kB
MemAvailable: 1925864 kB
Buffers: 679696 kB
Cached: 1293012 kB
SwapCached: 0 kB
Active: 2013308 kB
Inactive: 985748 kB
Active(anon): 867464 kB
Inactive(anon): 200204 kB
Active(file): 1145844 kB
Inactive(file): 785544 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 1949692 kB
SwapFree: 1949692 kB
Dirty: 160 kB
Writeback: 0 kB
AnonPages: 1026380 kB
Mapped: 49320 kB
Shmem: 41332 kB
Slab: 129624 kB
SReclaimable: 81024 kB
SUnreclaim: 48600 kB
KernelStack: 6592 kB
PageTables: 5936 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 3965828 kB
Committed_AS: 1151652 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
HardwareCorrupted: 0 kB
AnonHugePages: 0 kB
ShmemHugePages: 0 kB
ShmemPmdMapped: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 1368000 kB
DirectMap2M: 2826240 kB
DirectMap1G: 0 kB
# ps -ef | grep java
kafka 674 1 10 Jun03 ? 2-09:20:08 java -Xmx2048m -Xms512m -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -Djava.awt.headless=true -Xloggc:/opt/kafka/bin/../logs/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dkafka.logs.dir=/opt/kafka/bin/../logs -Dlog4j.configuration=file:/opt/kafka/bin/../config/log4j.properties -cp /opt/kafka/bin/../libs/activation-1.1.1.jar:/opt/kafka/bin/../libs/aopalliance-repackaged-2.5.0.jar:/opt/kafka/bin/../libs/argparse4j-0.7.0.jar:/opt/kafka/bin/../libs/audience-annotations-0.5.0.jar:/opt/kafka/bin/../libs/commons-lang3-3.8.1.jar:/opt/kafka/bin/../libs/connect-api-2.3.0.jar:/opt/kafka/bin/../libs/connect-basic-auth-extension-2.3.0.jar:/opt/kafka/bin/../libs/connect-file-2.3.0.jar:/opt/kafka/bin/../libs/connect-json-2.3.0.jar:/opt/kafka/bin/../libs/connect-runtime-2.3.0.jar:/opt/kafka/bin/../libs/connect-transforms-2.3.0.jar:/opt/kafka/bin/../libs/guava-20.0.jar:/opt/kafka/bin/../libs/hk2-api-2.5.0.jar:/opt/kafka/bin/../libs/hk2-locator-2.5.0.jar:/opt/kafka/bin/../libs/hk2-utils-2.5.0.jar:/opt/kafka/bin/../libs/jackson-annotations-2.9.9.jar:/opt/kafka/bin/../libs/jackson-core-2.9.9.jar:/opt/kafka/bin/../libs/jackson-databind-2.9.9.jar:/opt/kafka/bin/../libs/jackson-dataformat-csv-2.9.9.jar:/opt/kafka/bin/../libs/jackson-datatype-jdk8-2.9.9.jar:/opt/kafka/bin/../libs/jackson-jaxrs-base-2.9.9.jar:/opt/kafka/bin/../libs/jackson-jaxrs-json-provider-2.9.9.jar:/opt/kafka/bin/../libs/jackson-module-jaxb-annotations-2.9.9.jar:/opt/kafka/bin/../libs/jackson-module-paranamer-2.9.9.jar:/opt/kafka/bin/../libs/jackson-module-scala_2.11-2.9.9.jar:/opt/kafka/bin/../libs/jakarta.annotation-api-1.3.4.jar:/opt/kafka/bin/../libs/jakarta.inject-2.5.0.jar:/opt/kafka/bin/../libs/jakarta.ws.rs-api-2.1.5.jar:/opt/kafka/bin/../libs/javassist-3.22.0-CR2.jar:/opt/kafka/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/kafka/bin/../libs/javax.ws.rs-api-2.1.1.jar:/opt/kafka/bin/../libs/jaxb-api-2.3.0.jar:/opt/kafka/bin/../libs/jersey-client-2.28.jar:/opt/kafka/bin/../libs/jersey-common-2.28.jar:/opt/kafka/bin/../libs/jersey-container-servlet-2.28.jar:/opt/kafka/bin/../libs/jersey-container-servlet-core-2.28.jar:/opt/kafka/bin/../libs/jersey-hk2-2.28.jar:/opt/kafka/bin/../libs/jersey-media-jaxb-2.28.jar:/opt/kafka/bin/../libs/jersey-server-2.28.jar:/opt/kafka/bin/../libs/jetty-client-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jetty-continuation-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jetty-http-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jetty-io-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jetty-security-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jetty-server-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jetty-servlet-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jetty-servlets-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jetty-util-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jopt-simple-5.0.4.jar:/opt/kafka/bin/../libs/jsr305-3.0.2.jar:/opt/kafka/bin/../libs/kafka_2.11-2.3.0.jar:/opt/kafka/bin/../libs/kafka_2.11-2.3.0-sources.jar:/opt/kafka/bin/../libs/kafka-clients-2.3.0.jar:/opt/kafka/bin/../libs/kafka-log4j-appender-2.3.0.jar:/opt/kafka/bin/../libs/kafka-streams-2.3.0.jar:/opt/kafka/bin/../libs/kafka-streams-examples-2.3.0.jar:/opt/kafka/bin/../libs/kafka-streams-scala_2.11-2.3.0.jar:/opt/kafka/bin/../libs/kafka-streams-test-utils-2.3.0.jar:/opt/kafka/bin/../libs/kafka-tools-2.3.0.jar:/opt/kafka/bin/../libs/log4j-1.2.17.jar:/opt/kafka/bin/../libs/lz4-java-1.6.0.jar:/opt/kafka/bin/../libs/maven-artifact-3.6.1.jar:/opt/kafka/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka/bin/../libs/osgi-resource-locator-1.0.1.jar:/opt/kafka/bin/../libs/paranamer-2.8.jar:/opt/kafka/bin/../libs/plexus-utils-3.2.0.jar:/opt/kafka/bin/../libs/reflections-0.9.11.jar:/opt/kafka/bin/../libs/rocksdbjni-5.18.3.jar:/opt/kafka/bin/../libs/scala-library-2.11.12.jar:/opt/kafka/bin/../libs/scala-logging_2.11-3.9.0.jar:/opt/kafka/bin/../libs/scala-reflect-2.11.12.jar:/opt/kafka/bin/../libs/slf4j-api-1.7.26.jar:/opt/kafka/bin/../libs/slf4j-log4j12-1.7.26.jar:/opt/kafka/bin/../libs/snappy-java-1.1.7.3.jar:/opt/kafka/bin/../libs/spotbugs-annotations-3.1.9.jar:/opt/kafka/bin/../libs/validation-api-2.0.1.Final.jar:/opt/kafka/bin/../libs/zkclient-0.11.jar:/opt/kafka/bin/../libs/zookeeper-3.4.14.jar:/opt/kafka/bin/../libs/zstd-jni-1.4.0-1.jar kafka.Kafka /opt/kafka/config/server.properties
zookeep+ 734 1 0 Jun03 ? 01:28:30 java -Dzookeeper.log.dir=/var/log/zookeeper -Dzookeeper.root.logger=INFO,CONSOLE -cp /opt/zookeeper/bin/../build/classes:/opt/zookeeper/bin/../build/lib/*.jar:/opt/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/opt/zookeeper/bin/../lib/netty-3.10.5.Final.jar:/opt/zookeeper/bin/../lib/log4j-1.2.16.jar:/opt/zookeeper/bin/../lib/jline-0.9.94.jar:/opt/zookeeper/bin/../zookeeper-3.4.9.jar:/opt/zookeeper/bin/../src/java/lib/*.jar:/opt/zookeeper/bin/../conf: -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=false org.apache.zookeeper.server.quorum.QuorumPeerMain /opt/zookeeper/conf/zoo.cfg
WRONG:
# cat /proc/meminfo
MemTotal: 4032276 kB
MemFree: 141820 kB
MemAvailable: 170960 kB
Buffers: 104896 kB
Cached: 150488 kB
SwapCached: 129780 kB
Active: 1872244 kB
Inactive: 798372 kB
Active(anon): 1709660 kB
Inactive(anon): 746052 kB
Active(file): 162584 kB
Inactive(file): 52320 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 1949692 kB
SwapFree: 1412724 kB
Dirty: 72 kB
Writeback: 0 kB
AnonPages: 2286048 kB
Mapped: 39720 kB
Shmem: 40468 kB
Slab: 72456 kB
SReclaimable: 31544 kB
SUnreclaim: 40912 kB
KernelStack: 8360 kB
PageTables: 9968 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 3965828 kB
Committed_AS: 4008680 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
HardwareCorrupted: 0 kB
AnonHugePages: 0 kB
ShmemHugePages: 0 kB
ShmemPmdMapped: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 3798976 kB
DirectMap2M: 395264 kB
DirectMap1G: 0 kB
# ps -ef | grep java
zookeep+ 2537 1 0 Jun08 ? 00:23:35 java -Dzookeeper.log.dir=/var/log/zookeeper -Dzookeeper.root.logger=INFO,CONSOLE -cp /opt/zookeeper/bin/../build/classes:/opt/zookeeper/bin/../build/lib/*.jar:/opt/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/opt/zookeeper/bin/../lib/netty-3.10.5.Final.jar:/opt/zookeeper/bin/../lib/log4j-1.2.16.jar:/opt/zookeeper/bin/../lib/jline-0.9.94.jar:/opt/zookeeper/bin/../zookeeper-3.4.9.jar:/opt/zookeeper/bin/../src/java/lib/*.jar:/opt/zookeeper/bin/../conf: -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=false org.apache.zookeeper.server.quorum.QuorumPeerMain /opt/zookeeper/conf/zoo.cfg
kafka 2732 1 5 Jun08 ? 22:06:19 java -Xmx2048m -Xms512m -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -Djava.awt.headless=true -Xloggc:/opt/kafka/bin/../logs/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dkafka.logs.dir=/opt/kafka/bin/../logs -Dlog4j.configuration=file:/opt/kafka/bin/../config/log4j.properties -cp /opt/kafka/bin/../libs/activation-1.1.1.jar:/opt/kafka/bin/../libs/aopalliance-repackaged-2.5.0.jar:/opt/kafka/bin/../libs/argparse4j-0.7.0.jar:/opt/kafka/bin/../libs/audience-annotations-0.5.0.jar:/opt/kafka/bin/../libs/commons-lang3-3.8.1.jar:/opt/kafka/bin/../libs/connect-api-2.3.0.jar:/opt/kafka/bin/../libs/connect-basic-auth-extension-2.3.0.jar:/opt/kafka/bin/../libs/connect-file-2.3.0.jar:/opt/kafka/bin/../libs/connect-json-2.3.0.jar:/opt/kafka/bin/../libs/connect-runtime-2.3.0.jar:/opt/kafka/bin/../libs/connect-transforms-2.3.0.jar:/opt/kafka/bin/../libs/guava-20.0.jar:/opt/kafka/bin/../libs/hk2-api-2.5.0.jar:/opt/kafka/bin/../libs/hk2-locator-2.5.0.jar:/opt/kafka/bin/../libs/hk2-utils-2.5.0.jar:/opt/kafka/bin/../libs/jackson-annotations-2.9.9.jar:/opt/kafka/bin/../libs/jackson-core-2.9.9.jar:/opt/kafka/bin/../libs/jackson-databind-2.9.9.jar:/opt/kafka/bin/../libs/jackson-dataformat-csv-2.9.9.jar:/opt/kafka/bin/../libs/jackson-datatype-jdk8-2.9.9.jar:/opt/kafka/bin/../libs/jackson-jaxrs-base-2.9.9.jar:/opt/kafka/bin/../libs/jackson-jaxrs-json-provider-2.9.9.jar:/opt/kafka/bin/../libs/jackson-module-jaxb-annotations-2.9.9.jar:/opt/kafka/bin/../libs/jackson-module-paranamer-2.9.9.jar:/opt/kafka/bin/../libs/jackson-module-scala_2.11-2.9.9.jar:/opt/kafka/bin/../libs/jakarta.annotation-api-1.3.4.jar:/opt/kafka/bin/../libs/jakarta.inject-2.5.0.jar:/opt/kafka/bin/../libs/jakarta.ws.rs-api-2.1.5.jar:/opt/kafka/bin/../libs/javassist-3.22.0-CR2.jar:/opt/kafka/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/kafka/bin/../libs/javax.ws.rs-api-2.1.1.jar:/opt/kafka/bin/../libs/jaxb-api-2.3.0.jar:/opt/kafka/bin/../libs/jersey-client-2.28.jar:/opt/kafka/bin/../libs/jersey-common-2.28.jar:/opt/kafka/bin/../libs/jersey-container-servlet-2.28.jar:/opt/kafka/bin/../libs/jersey-container-servlet-core-2.28.jar:/opt/kafka/bin/../libs/jersey-hk2-2.28.jar:/opt/kafka/bin/../libs/jersey-media-jaxb-2.28.jar:/opt/kafka/bin/../libs/jersey-server-2.28.jar:/opt/kafka/bin/../libs/jetty-client-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jetty-continuation-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jetty-http-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jetty-io-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jetty-security-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jetty-server-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jetty-servlet-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jetty-servlets-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jetty-util-9.4.18.v20190429.jar:/opt/kafka/bin/../libs/jopt-simple-5.0.4.jar:/opt/kafka/bin/../libs/jsr305-3.0.2.jar:/opt/kafka/bin/../libs/kafka_2.11-2.3.0.jar:/opt/kafka/bin/../libs/kafka_2.11-2.3.0-sources.jar:/opt/kafka/bin/../libs/kafka-clients-2.3.0.jar:/opt/kafka/bin/../libs/kafka-log4j-appender-2.3.0.jar:/opt/kafka/bin/../libs/kafka-streams-2.3.0.jar:/opt/kafka/bin/../libs/kafka-streams-examples-2.3.0.jar:/opt/kafka/bin/../libs/kafka-streams-scala_2.11-2.3.0.jar:/opt/kafka/bin/../libs/kafka-streams-test-utils-2.3.0.jar:/opt/kafka/bin/../libs/kafka-tools-2.3.0.jar:/opt/kafka/bin/../libs/log4j-1.2.17.jar:/opt/kafka/bin/../libs/lz4-java-1.6.0.jar:/opt/kafka/bin/../libs/maven-artifact-3.6.1.jar:/opt/kafka/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka/bin/../libs/osgi-resource-locator-1.0.1.jar:/opt/kafka/bin/../libs/paranamer-2.8.jar:/opt/kafka/bin/../libs/plexus-utils-3.2.0.jar:/opt/kafka/bin/../libs/reflections-0.9.11.jar:/opt/kafka/bin/../libs/rocksdbjni-5.18.3.jar:/opt/kafka/bin/../libs/scala-library-2.11.12.jar:/opt/kafka/bin/../libs/scala-logging_2.11-3.9.0.jar:/opt/kafka/bin/../libs/scala-reflect-2.11.12.jar:/opt/kafka/bin/../libs/slf4j-api-1.7.26.jar:/opt/kafka/bin/../libs/slf4j-log4j12-1.7.26.jar:/opt/kafka/bin/../libs/snappy-java-1.1.7.3.jar:/opt/kafka/bin/../libs/spotbugs-annotations-3.1.9.jar:/opt/kafka/bin/../libs/validation-api-2.0.1.Final.jar:/opt/kafka/bin/../libs/zkclient-0.11.jar:/opt/kafka/bin/../libs/zookeeper-3.4.14.jar:/opt/kafka/bin/../libs/zstd-jni-1.4.0-1.jar kafka.Kafka /opt/kafka/config/server.properties
For ease of view (kafka's params) :
-Xmx2048m
-Xms512m
-XX:+UseG1GC
-XX:MaxGCPauseMillis=20
-XX:InitiatingHeapOccupancyPercent=35
-XX:+ExplicitGCInvokesConcurrent
-Djava.awt.headless=true
-Xloggc:/opt/kafka/bin/../logs/kafkaServer-gc.log
-verbose:gc
-XX:+PrintGCDetails
-XX:+PrintGCDateStamps
-XX:+PrintGCTimeStamps
-XX:+UseGCLogFileRotation
-XX:NumberOfGCLogFiles=10
-XX:GCLogFileSize=100M
Я не уверен, что значения Java mem оптимальны для этого сценария, но я попытался настроить их немного сдержанно.