root@Ubuntu18-JumpBox:~# hostname
Ubuntu18-JumpBox
root@Ubuntu18-JumpBox:~# cat /etc/os-release | grep -i version
VERSION="18.04.6 LTS (Bionic Beaver)"
VERSION_ID="18.04"
VERSION_CODENAME=bionic
root@Ubuntu18-JumpBox:~#
root@Ubuntu18-JumpBox:~# nproc
2
We have multiple commands to check the top consuming process of CPU in Linux.
Most of time we use following commands to check the CPU or Performance of Linux VM
1) top
2) sar
3) iostat
4) mpstat
5) commands/scripts
We will understand which one give more accurate information in terms of CPU process usage in Linux
simple top command output below
top - 13:47:09 up 3:08, 5 users, load average: 1.00, 1.01, 1.17
Tasks: 171 total, 2 running, 100 sleeping, 3 stopped, 0 zombie
%Cpu(s): 15.0 us, 37.4 sy, 0.0 ni, 47.5 id, 0.0 wa, 0.0 hi, 0.2 si, 0.0 st
KiB Mem : 8148440 total, 6489492 free, 440468 used, 1218480 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 7435128 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
14741 root 20 0 7928 832 768 R 99.7 0.0 124:40.17 yes
5512 root 20 0 0 0 0 I 0.3 0.0 0:00.02 kworker/0:0-eve
25517 root 20 0 44544 4232 3524 R 0.3 0.1 0:06.94 top
1 root 20 0 159964 9236 6704 S 0.0 0.1 0:03.34 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 rcu_gp
4 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 rcu_par_gp
6 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 kworker/0:0H-ev
root@Ubuntu18-JumpBox:~# nproc
2
The VM is having 2 procs , which means 2 process , that means each process can be utilized for 100% percentage in top command.
Many Linux Admins get confused here , the above 100% utilization is for only 1 proc and not for entire VM, that means out of 2 CPU the VM is completely utilizing 100% of one process and we have another process which can be taken care for next tasks.
In above top command output the below one is important to understand
%Cpu(s): 15.0 us, 37.4 sy, 0.0 ni, 47.5 id, 0.0 wa, 0.0 hi, 0.2 si, 0.0 st
15.0% CPU using by USER
37.4% CPU using by SYSTEM
47.5 % CPU is IDLE
=================================================
99.9 % TOTAL CPU of VM (either 2 core or 4 core or 8 core etc)
=================================================
1) TOP
Command to check the TOP consuming process
root@Ubuntu18-JumpBox:~# ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10
The command output and top command output shows the same, that means the command shows the accurate information.
2) SAR command output
root@Ubuntu18-JumpBox:~# sar -u 1 5
Linux 5.4.0-1091-azure (Ubuntu18-JumpBox) 10/01/22 _x86_64_ (2 CPU)
14:22:23 CPU %user %nice %system %iowait %steal %idle
14:22:24 all 12.06 0.00 38.69 0.00 0.00 49.25
14:22:25 all 14.07 0.00 38.19 0.00 0.00 47.74
14:22:26 all 12.56 0.00 40.20 0.00 0.00 47.24
14:22:27 all 14.43 0.00 36.32 0.00 0.00 49.25
14:22:28 all 14.00 0.00 37.00 0.00 0.00 49.00
Average: all 13.43 0.00 38.08 0.00 0.00 48.50
The sar output also matches the current scenario of CPU utilization with TOP command of VM.
Conclusion : That means TOP and SAR and above commands are shows the same output in real time scenario.
The next question is which commands are not showing same outputs when compared to TOP/SAR
3) IOSTAT & MPSTAT
Now we are using 100% of CPU for both cores , lets see whether IOSTAT or MPSTAT shows the correct output or not
TOP - 100% usage of 2 cores output below
MPSTAT and IOSTAT outputs below
We can understand that mpstat and iostat outputs are both are same but not accurate with TOP command when comparing with USER , SYSTEM and IDLE percentages.
Now Question is : Then both command outputs are wrong ??? answer is NO
Yes , both commands mpstat and iostat shows the statistics of average CPU usage of VM since VM boots/start , it wont shows the live statistics & we should not use those commands outputs for live troubleshooting of performance issues.
Then is there any way to check live usage of CPU with MPSTAT and IOSTAT ....... Yes we can ... Solution is below
You may want to use following command, which gives you 2 outputs every 3 seconds (as previous command gives information since the last reboot):
root@Ubuntu18-JumpBox:~# iostat -xtc 3 2
Note: We have to look the SECOND output for LIVE statistics and First output shows the AVG CPU utilization since reboot of VM.
root@Ubuntu18-JumpBox:~# mpstat -P ALL 2 3
How to create 100% CPU load in Linux
root@Ubuntu18-JumpBox:~# yes > /dev/null &
root@Ubuntu18-JumpBox:~# yes > /dev/null &
If the VM is having 4 CPU's then execute above commands 4 times for utilizing the all 4 CPU's
To stop it, execute the command killall yes as root OR kill -9 PID
Conclusion : for live performance issues always use TOP/SAR/COMMAND instead of mpstat or iostat
Post a Comment