7

I wonder on my staging server, whenever I run ps aux, it'll stuck. It outputs a list of process and stop responding. I checked there is enough RAM (1GB).

When I run the top command, it looks ok to me, but I wonder there is one zombie process. What's that? Anyone can explain?

top - 11:00:29 up  3:53,  2 users,  load average: 51.75, 50.52, 45.38
Tasks:  79 total,   1 running,  77 sleeping,   0 stopped,   1 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1747660k total,   603572k used,  1144088k free,    12644k buffers
Swap:   917496k total,        0k used,   917496k free,    97732k cached

Thanks

Chamnap
  • 805
  • Where does it get stuck? Before outputting any output at all or a bit further down? If it stops halfway through the output UID resolution could be screwed up, are you using LDAP/NIS or anything like that? – HampusLi May 19 '11 at 10:22

6 Answers6

7

If on Linux, run strace on ps to see what system calls it calls, and you may potentially see on which one it's hanging on:

$ strace ps aux

If you're on a different Unix-y system you'd use truss or dtruss.

4

Considering that you have a load of over 50, but the CPU is 100% idle, I would look at i/o. This looks like the computer is waiting for either the disks or the network to return data before it can proceed.

Try using iotop to see what's blocking this. It could be a drive on its way out.

Similar behaviour is also possible of your machine is configured to do name/group resolution and authentication against an external resource and that resource is not available. In that case it would help to check you PAM configuration and any relevant services, such as DNS, LDAP, NIS.

wolfgangsz
  • 8,917
1

Wikipedia explains it in great detail.

Sven
  • 99,533
  • 15
  • 182
  • 228
1

Um...your load average is showing something is REALLY eating i/o or CPU. Is your hard disk thrashing? Is it sluggish in response? A load average of 51 is really not all that normal.

Zombie - if a process is spawned from another process and ends, it's supposed to be reaped by the parent process again. If the parent crashed or disappeared, the orphaned process turns into a zombie. It is essentially a process in the task table that isn't taking resources or doing anything, but now the init process has to take care of it. There's really no way to clear them without a reboot, but they usually don't hurt anything. Now, if you have a LOT of zombies, you have a problem; a process with a bug, a resource issue, something isn't running properly. One or two zombies do not a Romero movie make, though.

Clarification-I used the term orphan, but the pedantic side would point out there is a difference between an orphaned process and a zombie. Orphans are still running, while the zombie is doing nothing, it's not taking resources other than a process table entry. And I'm not sure about init taking the zombie process. I may be wrong on that (Init is supposed to take care of orphan processes). Either way, unless you have many zombies appearing, one or two on a system isn't a problem normally.

Bart Silverstrim
  • 31,300
  • I just did some load testing to hit the cache in memcached. Then, it's stuck like this. I cannot restart mysql or hudson service at all. – Chamnap Aug 18 '10 at 12:06
  • I should have looked more closely...usually the canned response is the first thing to check. Your cpu is 100% idle? Swap isn't being hit. You said that it's "stuck"...as in hitting enter does nothing, nothing his happening? Evidently you can switch virtual terminals or you wouldn't have had the top output. does it give a status on what process is zombied (the S[tate] is Z)? Does a regular "ps ax" tell you what's happening? If this consistently happens when hitting your memory hard, maybe you have a bad memory chip in there. – Bart Silverstrim Aug 18 '10 at 12:14
  • i see the cpu is processing very high. – Chamnap Aug 18 '10 at 12:31
  • Then you had a timely snapshot of top output for CPU usage. What process(es) (is|are) taking up the CPU? – Bart Silverstrim Aug 18 '10 at 13:05
  • I wonder that too. I usually run up like this all the time. Do you know which command to run in order to see that? – Chamnap Aug 18 '10 at 13:14
  • "top" tells you the processes using CPU time. %CPU. – Bart Silverstrim Aug 18 '10 at 13:36
1

Wow, from your top output, your load is ridiculously high. Unless you have a 50+ CPU system, that is way too high. The CPUs though are not doing anything:

Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st

This sounds like you have a hardware problem. What does the output of dmesg tell you?

vmfarms
  • 3,157
0

ps aux outputs the list of processes once and quits. It does not periodically update like top does.

A "zombie" process is a process that has completed execution but still has an entry in the process table. This entry is still needed to allow the process that started the (now zombie) process to read its exit status.

Delan Azabani
  • 568