WinDBG : A rodent killer


Recently, we had to the opportunity to remove the Poison IVY RAT from a customer's system that was compromised by unknown attackers, and remained undetected for a number of weeks. Although the steps involved in the remediation were quite straight forward, we conducted the entire operation from within WinDBG, by attaching it remotely to the target system. In this article we share the key steps that were taken to identify and subsequently disable Poison IVY from executing on the infected system, especially because these key steps showcase the power of WinDBG.

For the uninitiated, Poison IVY is a Remote Administration Tool or a Remote Access Trojan (RAT) which has been around for more than a decade. Recently it has generated renewed interest due to its usage in some high profile Advanced Persistent Threat (APT) campaigns. As there is enough introductory literature on Poison IVY, we jump right into the key steps relevant in this particular scenario, rather than restating what already exists in the background literature (please see reference at the end for additional reading).

For the purpose of this article, we generated our own variant of Poison IVY using default configuration options and deployed it in a controlled environment to loosely replicate our opeating environment. In our setup, a 32-bit Windows XP SP3 system was infected with Poison IVY version 2.3.3 (2007). The Poison IVY client was running on system 192.168.100.71 which also served as the operator control system. The configuration options of the Poison IVY server (payload) deployed on the Windows XP systems were as follows::

- Connection
Connect to: 192.168.100.71:3460:0,
ID: test1
Group: 
Password: admin
Connect through proxy: No

 - Install
ActiveX Startup: Yes
ActiveX Key: {F7B9CADB-DA19-79AB-9437-F098F7484550}
Copy File: Yes
File Name: POISONIVY2.exe
Copy File To: System Folder
Melt File: No

 - Advanced
Process Mutex: PoisonIvy
Inject Server: Yes
Persistence: Yes
Key Logger: No
Format: PE
File Alignment: 512

 - Build
Icon: No
Execute Third-party Applications: No

At our customer site, we deployed DbgSrv.exe (the Process Server tool, part of the Debugging Tools for Windows package) on the infected system and configured a reverse SSH tunnel that let us connect to DbgSrv.exe from WinDBG running outside the customer's firewall. Once the remote debugging setup was up and running, we examined a list of processes running on the system using WinDBG's "Attach to Process" option. We noticed an instance of IEXPLORE.exe (IE Web Browser) which was invoked with the command line argument "-nohome". This option causes IE to start up without displaying the home page, but is seldom used under normal operating conditions, especially when IE is invoked directly from the desktop or Start menu. From past experience we knew that this is exactly how Poison IVY invokes IE when it is configured to inject itself into the default browser. Since this was the only instance of IE running on the system, we attached to this process from within WinDBG. Upon successful attachment we confirmed the Windows version, as shown below:

0:001> vertarget
Windows XP Version 2600 (Service Pack 3) UP Free x86 compatible
Product: WinNt, suite: SingleUserTS
kernel32.dll version: 5.1.2600.5512 (xpsp.080413-2111)

Next we obtained the list of threads running in the IEXPLORE.exe process, along with their call stacks.

0:002> ~*kvn

   0  Id: 7c4.7c8 Suspend: 2 Teb: 7ffdf000 Unfrozen
 # ChildEBP RetAddr  Args to Child              
00 00000000 00000000 00000000 00000000 00000000 kernel32!BaseProcessStartThunk

   1  Id: 7c4.7cc Suspend: 1 Teb: 7ffde000 Unfrozen
 # ChildEBP RetAddr  Args to Child              
00 001af96c 7c90d1fc 7c8023f1 00000000 001af9a0 ntdll!KiFastSystemCallRet (FPO: [0,0,0])
01 001af970 7c8023f1 00000000 001af9a0 00160190 ntdll!NtDelayExecution+0xc (FPO: [2,0,0])
02 001af9c8 7c802455 0000ea60 00000000 001affb4 kernel32!SleepEx+0x61 (FPO: [Non-Fpo])
03 001af9d8 00150541 0000ea60 0000000b 001af944 kernel32!Sleep+0xf (FPO: [Non-Fpo])
WARNING: Frame IP not in any known module. Following frames may be wrong.
04 001affb4 7c80b713 00160000 00000000 00000000 0x150541
05 001affec 00000000 00150000 00160000 00000000 kernel32!BaseThreadStart+0x37 (FPO: [Non-Fpo])

#  2  Id: 7c4.4ec Suspend: 1 Teb: 7ffdd000 Unfrozen
 # ChildEBP RetAddr  Args to Child              
00 00a2ffc8 7c950010 00000005 00000004 00000001 ntdll!DbgBreakPoint (FPO: [0,0,0])
01 00a2fff4 00000000 00000000 00000000 00000000 ntdll!DbgUiRemoteBreakin+0x2d (FPO: [Non-Fpo])

Thread #1 stood out, since it is the only one that has a return address in its call-stack, for which WinDBG was not able to resolve symbols and had no module associated with it. Since the return address did not belong to any module, we assumed that this is code running out of dynamically allocated memory and that the thread was injected into the process from outside. Further it was noted that thread #0, the primary thread of the process, had a suspend count of 2 which indicated that the process was created specifically for the purpose of hosting externally injected code. Upon closer examination of the call-stack of thread #1, we found that the thread was sleeping for 1 minute, as shown below:

0:002> ~1kvn
 # ChildEBP RetAddr  Args to Child              
00 001af96c 7c90d1fc 7c8023f1 00000000 001af9a0 ntdll!KiFastSystemCallRet (FPO: [0,0,0])
01 001af970 7c8023f1 00000000 001af9a0 00160190 ntdll!NtDelayExecution+0xc (FPO: [2,0,0])
02 001af9c8 7c802455 0000ea60 00000000 001affb4 kernel32!SleepEx+0x61 (FPO: [Non-Fpo])
03 001af9d8 00150541 0000ea60 0000000b 001af944 kernel32!Sleep+0xf (FPO: [Non-Fpo])
WARNING: Frame IP not in any known module. Following frames may be wrong.
04 001affb4 7c80b713 00160000 00000000 00000000 0x150541
05 001affec 00000000 00150000 00160000 00000000 kernel32!BaseThreadStart+0x37 (FPO: [Non-Fpo])

The function prototype for kernel32!Sleep() is VOID Sleep ( DWORD dwMilliseconds).

0:002> ?ea60
Evaluate expression: 60000 = 0000ea60

The very first function in the injected thread, responsible for calling the thread's top level function was kernel32!BaseThreadStart(), which has the prototype "VOID BaseThreadSTART( PTHREAD_START_ROUTINE pfnStartAddr, PVOID pvParam)". The pfnStartAddr is the address at which the thread started execution and pvParam was the data context passed to the thread. The values for pfnStartAddr and pvParam were 0x00150000 and 0x00160000 respectively.

Next we located the two the memory regions that encompassed the aforementioned addresses, from the output of the "!address" command.

0:002> !address
.
.
.

*   150000   151000     1000 MEM_PRIVATE MEM_COMMIT  PAGE_EXECUTE_READWRITE <unclassified> 
*   151000   160000     f000             MEM_FREE    PAGE_NOACCESS          Free 
*   160000   161000     1000 MEM_PRIVATE MEM_COMMIT  PAGE_EXECUTE_READWRITE <unclassified> 
*   161000   170000     f000             MEM_FREE    PAGE_NOACCESS          Free 
.
.
.

The fact that both these virtual address ranges were marked as Read-Write-Execute (RWX) raised some suspicion. However, instead of attempting to disassemble the instruction sequences at address 0x00150000, we started looking for interesting strings at address 0x00160000, knowing well that we would find file and registry paths as cataloged in [2]. The following shows excerpts from virtual address range that contained the data used by the injected thread.

0:002> db 160000 161000 
00160000  00 11 42 ab 71 07 4a ab-71 2b 3e ab 71 27 4c ab  ..B.q.J.q+>.q'L.
.
.
.
001603f0  00 00 00 00 00 00 01 02-00 01 01 50 6f 69 73 6f  ...........Poiso
00160400  6e 49 76 79 00 00 00 00-00 00 00 00 00 00 00 53  nIvy...........S
00160410  74 75 62 50 61 74 68 00-53 4f 46 54 57 41 52 45  tubPath.SOFTWARE
00160420  5c 43 6c 61 73 73 65 73-5c 68 74 74 70 5c 73 68  \Classes\http\sh
00160430  65 6c 6c 5c 6f 70 65 6e-5c 63 6f 6d 6d 61 6e 64  ell\open\command
00160440  00 00 6e 6f 74 65 70 61-64 2e 65 78 65 00 00 00  ..notepad.exe...
.
.
.
001604b0  00 00 00 53 6f 66 74 77-61 72 65 5c 4d 69 63 72  ...Software\Micr
001604c0  6f 73 6f 66 74 5c 41 63-74 69 76 65 20 53 65 74  osoft\Active Set
001604d0  75 70 5c 49 6e 73 74 61-6c 6c 65 64 20 43 6f 6d  up\Installed Com
001604e0  70 6f 6e 65 6e 74 73 5c-7b 46 37 42 39 43 41 44  ponents\{F7B9CAD
001604f0  42 2d 44 41 31 39 2d 37-39 41 42 2d 39 34 33 37  B-DA19-79AB-9437
00160500  2d 46 30 39 38 46 37 34-38 34 35 35 30 7d 00 00  -F098F7484550}..

.
.
.
001605b0  00 00 43 3a 5c 57 49 4e-44 4f 57 53 5c 73 79 73  ..C:\WINDOWS\sys
001605c0  74 65 6d 33 32 5c 50 4f-49 53 4f 4e 49 56 59 32  tem32\POISONIVY2
001605d0  2e 65 78 65 00 00 00 00-00 00 00 00 00 00 00 00  .exe............

The two strings in the above output worth noting were - the registry path containing the GUID and the path to the executable which launched PoisonIVY and perfomed the thread injection. We dumped the registry path in a format that would make copy and paste easier using the "da" command.

0:002> da /c100 001604b3
001604b3  "Software\Microsoft\Active Setup\Installed Components\{F7B9CADB-DA19-79AB-9437-F098F7484550}"

WinDBG, when used as a user mode debugger, provides the "!dreg" command for looking up registry keys and their values which we used to enumerate the contents of the above mentioned key.

0:002> !dreg HKLM\Software\Microsoft\Active Setup\Installed Components\{F7B9CADB-DA19-79AB-9437-F098F7484550}!*

Value: "StubPath" - REG_SZ: "C:\WINDOWS\system32\POISONIVY2.exe"
------------------------------------------------------------------------

Poison IVY needs an Auto Start Extension Point (ASEP) to ensure that it gets a chance to execute every time the system is restarted. For this purpose it uses the registry key HKLM\Software\Microsoft\Active Setup\Installed Components\{GUID}. Every time a user logs into the system, explorer checks if the unique {GUID} in HKLM is also present in HKCU (i.e. the current user's profile). If the {GUID} is not present, it creates the same key in HKCU and subsequently executes the executable specified by the StubPath value in the HLKM key. As we had observed from the output of "!dreg" command, in Poison IVY's case the executable was c:\Windows\System32\POISONIVY2.exe i.e. the Poison IVY server. You can read more about how this ASEP works in [5]. Every time Poison IVY executed itself on the system, it deleted the registry key from HKCU to ensure that the executable ran again during the next system restart.

We verified that the HKCU key was indeed deleted by Poison IVY, using "!dreg", as follows:

0:002> !dreg HKCU\Software\Microsoft\Active Setup\Installed Components\{F7B9CADB-DA19-79AB-9437-F098F7484550}!*
Could not open subkey Software\Microsoft\Active Setup\Installed Components\{F7B9CADB-DA19-79AB-9437-F098F7484550}. Error (2).
No values

To thwart the persistence mechanism, it sufficed to create the equivalent registry key in HKCU which would fool explorer.exe into believing that the software identified by the unique GUID i.e. {F7B9CADB-DA19-79AB-9437-F098F7484550} was already present in the current user's profile and hence would not need run the executable specified in StubPath. To achieve this trickery, we leveraged WinDBG's capability to run external executables i.e. ".shell" to execute reg.exe and add the appropriate key, as shown below:

0:002> .shell -x reg add "HKEY_CURRENT_USER\Software\Microsoft\Active Setup\Installed Components\{F7B9CADB-DA19-79AB-9437-F098F7484550}"

We verified again, that the key was created as expected, as follows:

0:002> !dreg HKCU\Software\Microsoft\Active Setup\Installed Components\{F7B9CADB-DA19-79AB-9437-F098F7484550}!*
No values

After the system was rebooted, Poison IVY no longer executed on the infected system at which point the Poison IVY executable and the ASEP registry keys were both deleted using WinDBG to completely remove Poison IVY from the system.

In this article we attached WinDBG, as a user mode debugger, to an instance of IEXPLORE.exe running on a remote system, identified the thread and memory regions created by Poison IVY, identified the ASEP used by Poison IVY's persistence mechanism and then finally took steps to ensure that Poison IVY never executed on the system. And most importantly we performed all these steps without ever leaving the WinDBG environment.

References

[1] Poison IVY: Assessing Damage and Extracting Intelligence

[2] Poison Ivy RAT: Configuration & Communications

[3] Initial Analysis of Poison Ivy

[4] Decrypting Poison Ivy's Communication Using Code Injection and DLL Proxies

[5] Active Setup Registry Key: What it is and how to create in the package using Admin Studio Install Shield.