一jvm問題分析--cpu高すぎる分析、jstack使用
5680 ワード
問題の説明
1台のサーバーを測定して、cpuはずっと高くなって、問題のスレッドを調べます
運用環境
k 8 s,javaサービス
ステップまず虚機に上陸し、k 8 s上のpod位置 を表示する.問題pod(k 8 sの基本概念に詳しくなければ、先に学ぶことができる) は対応podサービスに入り、 を命令する.
4.対応するサービスプロセスを見つけるコマンドtop-HP pidクエリはcpu最大スレッド を占めます.対応スレッドidを16進数 に変換する.対応するスレッド情報 を検索する.問題のまとめ:上jstackログから、cpuの損失の主な位置がわかり、これによって私たちの問題 を解決します.
1台のサーバーを測定して、cpuはずっと高くなって、問題のスレッドを調べます
運用環境
k 8 s,javaサービス
ステップ
[root@kube-node01 ~]# kubectl get namespace
NAME STATUS AGE
default Active 102d
yy Active 34d
[root@kube-node01 ~]# kubectl get pods --namespace= yy |grep 'nihao'
nihao-6dc768ffdb-f7smm 2/2 Running 0 40m
kubectl exec -it $PODname --namespace=$ -- /bin/bash
[root@kube-node01 ~]# kubectl exec -it nihao-hessian-6dc768ffdb-f7smm --namespace=yy -- /bin/bash
Defaulting container name to nihao-hessian.
Use 'kubectl describe pod/nihao hessian-6dc768ffdb-f7smm -n yc' to see all of the containers in this pod.
[root@ nihao-hessian-6dc768ffdb-f7smm /]#
4.対応するサービスプロセスを見つける
[root@ nihao-hessian-6dc768ffdb-f7smm /]#top
top - 19:38:31 up 100 days, 2:45, 0 users, load average: 5.18, 3.37, 2.64
Tasks: 5 total, 1 running, 4 sleeping, 0 stopped, 0 zombie
%Cpu(s): 6.9 us, 0.7 sy, 0.0 ni, 92.1 id, 0.0 wa, 0.0 hi, 0.2 si, 0.0 st
KiB Mem : 13174908+total, 862080 free, 75972832 used, 54914184 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 46853388 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
30 root 20 0 18.8g 2.8g 15984 S 201.3 2.2 89:26.16 java
1 root 20 0 15264 1700 1328 S 0.0 0.0 0:00.04 entrypoint.sh
15 root 20 0 15260 1764 1360 S 0.0 0.0 0:00.00 startup.sh
419 root 20 0 15392 2196 1636 S 0.0 0.0 0:00.02 bash
437 root 20 0 59576 2116 1508 R 0.0 0.0 0:00.03 top
top - 19:52:56 up 100 days, 2:59, 0 users, load average: 5.08, 2.12, 2.08
Threads: 241 total, 28 running, 213 sleeping, 0 stopped, 0 zombie
%Cpu(s): 5.5 us, 0.8 sy, 0.0 ni, 93.5 id, 0.0 wa, 0.0 hi, 0.2 si, 0.0 st
KiB Mem : 13174908+total, 789344 free, 75998560 used, 54961192 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 46816480 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
294 root 20 0 18.9g 2.8g 15988 S 12.1 2.2 5:23.91 java
361 root 20 0 18.9g 2.8g 15988 S 12.1 2.2 5:18.12 java
301 root 20 0 18.9g 2.8g 15988 S 10.8 2.2 5:19.84 java
[root@ nihao-hessian-6dc768ffdb-f7smm /]# printf "%x
" 294
126
[root@nihao-hessian-6dc768ffdb-f7smm /]# jstack 30 | grep 0x126 --color -ab30
165950:"http-nio-8080-exec-1" #204 daemon prio=5 os_prio=0 tid=0x00007f88c61fe000 nid=0x126 runnable [0x00007f87a4837000]
166065- java.lang.Thread.State: RUNNABLE
166101- at java.lang.Class.getDeclaredFields0(Native Method)
166155- at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
166217- at java.lang.Class.getDeclaredField(Class.java:2068)
166271- at jdk.nashorn.internal.runtime.Context$ContextCodeInstaller$1.run(Context.java:209)
166357- at jdk.nashorn.internal.runtime.Context$ContextCodeInstaller$1.run(Context.java:204)
166443- at java.security.AccessController.doPrivileged(Native Method)
166506- at jdk.nashorn.internal.runtime.Context$ContextCodeInstaller.initialize(Context.java:204)
166597- at jdk.nashorn.internal.codegen.CompilationPhase$InstallPhase.transform(CompilationPhase.java:508)
166697- at jdk.nashorn.internal.codegen.CompilationPhase.apply(CompilationPhase.java:624)
166780- at jdk.nashorn.internal.codegen.Compiler.compile(Compiler.java:655)
166849- at jdk.nashorn.internal.runtime.Context.compile(Context.java:1317)
166917- - locked <0x00000000e9c00368> (a jdk.nashorn.internal.runtime.Context)
166989- at jdk.nashorn.internal.runtime.Context.compileScript(Context.java:1251)
167063- at jdk.nashorn.internal.runtime.Context.compileScript(Context.java:627)
167136- at jdk.nashorn.api.scripting.NashornScriptEngine.compileImpl(NashornScriptEngine.java:535)
167228- at jdk.nashorn.api.scripting.NashornScriptEngine.compileImpl(NashornScriptEngine.java:524)
167320- at jdk.nashorn.api.scripting.NashornScriptEngine.evalImpl(NashornScriptEngine.java:402)
167409- at jdk.nashorn.api.scripting.NashornScriptEngine.eval(NashornScriptEngine.java:155)
167494- at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)
167568- at com.yeepay.yjf.core.aop.CacheHanlerAop.getSelDefKey(CacheHanlerAop.java:104)
167649- at com.yeepay.yjf.core.aop.CacheHanlerAop.cacheAop(CacheHanlerAop.java:52)
167725- at sun.reflect.GeneratedMethodAccessor274.invoke(Unknown Source)
167791- at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
167881- at java.lang.reflect.Method.invoke(Method.java:498)
167934- at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
168056- at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
168165- at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
168257- at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168)
168367- at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)