The other day, I was tasked with finding a way to get a list of all running processes on a Windows XP virtual machine. I was also supposed to include information about how much CPU and memory each process used. Fortunately, this didn’t have to be a remote script, but one that could be run on the client. After a fair bit of Googling here and there, I finally found a solution. In this article, we’ll look at some of the rejects as well as the eventual solution, which happens to work cross-platform.
One of the first scripts I found was this one back from March of 2006:
# http://mail.python.org/pipermail/python-win32/2006-March/004340.html
import win32com.client
wmi=win32com.client.GetObject('winmgmts:')
for p in wmi.InstancesOf('win32_process'):
print p.Name, p.Properties_('ProcessId'), \
int(p.Properties_('UserModeTime').Value)+int(p.Properties_('KernelModeTime').Value)
children=wmi.ExecQuery('Select * from win32_process where ParentProcessId=%s' %p.Properties_('ProcessId'))
for child in children:
print '\t',child.Name,child.Properties_('ProcessId'), \
int(child.Properties_('UserModeTime').Value)+int(child.Properties_('KernelModeTime').Value)
This script requires the PyWin32 package to work. However, while it’s a handy little script, it doesn’t show anything that I want except the ProcessId. I don’t really care about the user or kernel mode times (i.e. the total CPU time by user or kernel). Also I don’t really like working with the black magic of COM, so I ended up rejecting this one out of hand.
Next up was an ActiveState recipe. It looked promising:
# http://code.activestate.com/recipes/303339-getting-process-information-on-windows/
import win32pdh, string, win32api
def procids():
#each instance is a process, you can have multiple processes w/same name
junk, instances = win32pdh.EnumObjectItems(None,None,'process', win32pdh.PERF_DETAIL_WIZARD)
proc_ids=[]
proc_dict={}
for instance in instances:
if instance in proc_dict:
proc_dict[instance] = proc_dict[instance] + 1
else:
proc_dict[instance]=0
for instance, max_instances in proc_dict.items():
for inum in xrange(max_instances+1):
hq = win32pdh.OpenQuery() # initializes the query handle
path = win32pdh.MakeCounterPath( (None,'process',instance, None, inum,'ID Process') )
counter_handle=win32pdh.AddCounter(hq, path)
win32pdh.CollectQueryData(hq) #collects data for the counter
type, val = win32pdh.GetFormattedCounterValue(counter_handle, win32pdh.PDH_FMT_LONG)
proc_ids.append((instance,str(val)))
win32pdh.CloseQuery(hq)
proc_ids.sort()
return proc_ids
print procids()
Alas, while this also got me a list of processes from my Windows box (along with the PID), it didn’t give me any information on the CPU and memory utilization. I think this one could work if I used different counter names. I’m guessing if you wanted, you could figure out that information using MSDN. I didn’t want to mess with that, so I continued digging.
That recipe led me to the following one based on ctypes:
# http://code.activestate.com/recipes/305279/
"""
Enumerates active processes as seen under windows Task Manager on Win NT/2k/XP using PSAPI.dll
(new api for processes) and using ctypes.Use it as you please.
Based on information from http://support.microsoft.com/default.aspx?scid=KB;EN-US;Q175030&ID=KB;EN-US;Q175030
By Eric Koome
email ekoome@yahoo.com
license GPL
"""
from ctypes import *
#PSAPI.DLL
psapi = windll.psapi
#Kernel32.DLL
kernel = windll.kernel32
def EnumProcesses():
arr = c_ulong * 256
lpidProcess= arr()
cb = sizeof(lpidProcess)
cbNeeded = c_ulong()
hModule = c_ulong()
count = c_ulong()
modname = c_buffer(30)
PROCESS_QUERY_INFORMATION = 0x0400
PROCESS_VM_READ = 0x0010
#Call Enumprocesses to get hold of process id's
psapi.EnumProcesses(byref(lpidProcess),
cb,
byref(cbNeeded))
#Number of processes returned
nReturned = cbNeeded.value/sizeof(c_ulong())
pidProcess = [i for i in lpidProcess][:nReturned]
for pid in pidProcess:
#Get handle to the process based on PID
hProcess = kernel.OpenProcess(PROCESS_QUERY_INFORMATION | PROCESS_VM_READ,
False, pid)
if hProcess:
psapi.EnumProcessModules(hProcess, byref(hModule), sizeof(hModule), byref(count))
psapi.GetModuleBaseNameA(hProcess, hModule.value, modname, sizeof(modname))
print "".join([ i for i in modname if i != '\x00'])
#-- Clean up
for i in range(modname._length_):
modname[i]='\x00'
kernel.CloseHandle(hProcess)
if __name__ == '__main__':
EnumProcesses()
This is pretty clever looking, but I’m pretty bad at parsing ctypes. It’s something I want to learn, but I had a deadline, doggone it! Plus this one only showed a list of running processes but no information about them. Fortunately, the author included a reference, but I decided to keep looking.
Next I found a thread about using Tim Golden’s handy WMI module to do this sort of thing (below is copied right from the thread):
# http://mail.python.org/pipermail/python-win32/2003-December/001482.html
>>> processes = WMI.InstancesOf('Win32_Process')
>>> len(processes)
41
>>> [process.Properties_('Name').Value for process in processes] # get
the process names
[u'System Idle Process', u'System', u'SMSS.EXE', u'CSRSS.EXE',
u'WINLOGON.EXE', u'SERVICES.EXE', u'LSASS.EXE', u'SVCHOST.EXE',
u'SVCHOST.EXE', u'SVCHOST.EXE', u'SVCHOST.EXE', u'SPOOLSV.EXE',
u'ati2evxx.exe', u'BAsfIpM.exe', u'defwatch.exe', u'inetinfo.exe',
u'mdm.exe', u'rtvscan.exe', u'SCARDSVR.EXE', u'WLTRYSVC.EXE',
u'BCMWLTRY.EXE', u'EXPLORER.EXE', u'Apoint.exe', u'carpserv.exe',
u'atiptaxx.exe', u'quickset.exe', u'DSentry.exe', u'Directcd.exe',
u'vptray.exe', u'ApntEx.exe', u'FaxCtrl.exe', u'digstream.exe',
u'CTFMON.EXE', u'wuauclt.exe', u'IEXPLORE.EXE', u'Pythonwin.exe',
u'MMC.EXE', u'OUTLOOK.EXE', u'LineMgr.exe', u'SAPISVR.EXE',
u'WMIPRVSE.EXE']
Here is how to get a single process and get its PID.
>>> p = WMI.ExecQuery('select * from Win32_Process where
Name="Pythonwin.exe"')
>>> [prop.Name for prop in p[0].Properties_] # let's look at all the
process property names
[u'Caption', u'CommandLine', u'CreationClassName', u'CreationDate',
u'CSCreationClassName', u'CSName', u'Description', u'ExecutablePath',
u'ExecutionState', u'Handle', u'HandleCount', u'InstallDate',
u'KernelModeTime', u'MaximumWorkingSetSize', u'MinimumWorkingSetSize',
u'Name', u'OSCreationClassName', u'OSName', u'OtherOperationCount',
u'OtherTransferCount', u'PageFaults', u'PageFileUsage',
u'ParentProcessId', u'PeakPageFileUsage', u'PeakVirtualSize',
u'PeakWorkingSetSize', u'Priority', u'PrivatePageCount', u'ProcessId',
u'QuotaNonPagedPoolUsage', u'QuotaPagedPoolUsage',
u'QuotaPeakNonPagedPoolUsage', u'QuotaPeakPagedPoolUsage',
u'ReadOperationCount', u'ReadTransferCount', u'SessionId', u'Status',
u'TerminationDate', u'ThreadCount', u'UserModeTime', u'VirtualSize',
u'WindowsVersion', u'WorkingSetSize', u'WriteOperationCount',
u'WriteTransferCount']
>>> p[0].Properties_('ProcessId').Value # get our ProcessId
928
This is some cool stuff and I use Golden’s modules in some of my other code. However, I was still uncertain as to which counters to use to get to my information. I thought most of this stuff would just be coded for me or something! Well, it turned out that there is a package out there that does exactly what I needed AND it works on all the three of the major platforms! Amazing!
The Cross-Platform Solution!
The package’s name is psutil and it was what I decided to use. Here’s what I ended up with:
import os
import psutil
import time
logPath = r'some\path\proclogs'
if not os.path.exists(logPath):
os.mkdir(logPath)
separator = "-" * 80
format = "%7s %7s %12s %12s %30s, %s"
format2 = "%7.4f %7.2f %12s %12s %30s, %s"
while 1:
procs = psutil.get_process_list()
procs = sorted(procs, key=lambda proc: proc.name)
logPath = r'some\path\proclogs\procLog%i.log' % int(time.time())
f = open(logPath, 'w')
f.write(separator + "\n")
f.write(time.ctime() + "\n")
f.write(format % ("%CPU", "%MEM", "VMS", "RSS", "NAME", "PATH"))
f.write("\n")
for proc in procs:
cpu_percent = proc.get_cpu_percent()
mem_percent = proc.get_memory_percent()
rss, vms = proc.get_memory_info()
rss = str(rss)
vms = str(vms)
name = proc.name
path = proc.path
f.write(format2 % (cpu_percent, mem_percent, vms, rss, name, path))
f.write("\n\n")
f.close()
print "Finished log update!"
time.sleep(300)
print "writing new log data!"
Yes, it’s an infinite loop and yes, that’s usually a very bad thing to do (except in GUI programming). However, for my purpose, I needed a way to check the user’s processes every 5 minutes or so to see what was causing the machine to act so weird. Thus, the script needs to run forever and log the results to uniquely named files. That’s all this script does, along with a little formatting magic. Feel free to use it or not as you see fit.
I hope you found this collection of material helpful. Hopefully it will save you all the digging I went through!
Note: While this last script appears to work just fine on Windows XP, on Windows 7 32 and 64-bit, you will get an “Access Denied” traceback, I suspect this is caused by Window 7’s increased security, but I will try to find a workaround.
UPDATE (10/09/2010) – The psutil folks don’t know why it doesn’t work, but one of their developers has confirmed the issue. You can follow along on their Google Groups list.