Category Archives: Security

Rootkit in my lab? (Part III)

First, thanks for all the comments in the previous articles (Part I and Part II).

I decided to analyze one the crash I experienced during registry analysis.
I could reproduce all the time a BSOD with Regshot. I thought it would be nice to see what I could get with WinDBG.

I had my environment set up with the suspicious VM configured to debug activated on the serial port, which is a simple pipe on Mac OS X.
Another VM is configured with a serial port as the other end of this pipe, and WinDBG attached to it.
Another method would be to just configure Windows to create a crashdump file with kernel symbols, that you can later load into WinDBG. Of course, the first method offers more opportunities to check and play with the live system.

Then, I just boot the target and trigger the crash, simply by starting a scan with Regshot:

Windows then crashes, WinDBG catches the exception and stops.

So what do we have ?

First, the error type, PAGE_FAULT_IN_NONPAGED_AREA (50), means that an instruction pointed to an invalid memory address. Let’s check this.

With !analyse -v, you get the full error dump.

Crashing Instruction

It shows the function (nt!CmpGetValueKeyFromCache, offset 0x89) and the memory address where the crash was triggered.

The instruction at this address is:

80637807 f3a5 rep movs dword ptr es:[edi],dword ptr [esi]

This instruction is trying to copy 8 bytes at the address pointed by EDI.
EDI has the value of 0xe1285050 at execution time.

And what do we have at this memory location ?

EDI pointing to invalid memory section

Nothing indeed. Note that this corruption persists at every boot.

So what can we conclude?
We can certainly exclude hardware failure, because it is a virtual machine and because the corruption always occur at the same memory region, even after a reboot.
At least, I can now be sure that something in the kernel is definitely corrupted.

Could it be a rootkit trick? Still the question remains, but to me it now looks very, very suspicious. Some rootkit code, poorly written, could have sat in this non-paged memory area and been paged out, causing the BSOD. I have not much knowledge about it at this time but I am going to search on this. At least, I now have good starting point to look at.

That’s all for today, folks. I wrote it while I am still working on it, so sorry if it looks rough and incomplete. It is sort of live, thoughts are still in process.

Again, I am looking forward to reading your comments and suggestions. (Hopefully) there will be a part IV!

Rootkit in my lab? (part II)

I finished checking the RAM with Volatility and… I found nothing. Nada.

It’s a lot of fustration. There must be something just there, but my findings are certainly limited by my skills.

I attach here some of the main outputs of Volatility. As far as I can tell:

  • no evidence of injection or kernel hooking
  • no suspicious process
  • no suspicious driver
  • no suspicious registry entry
  • etc.

Based on my observations, I first tried to narrow my investigations (drivers and hooks) but as I could not find anything, I ended dumping most of Volatility outputs in hope to see something unusual. I also compared them with a fresh Windows XP SP3 install. I extracted keyboard related drivers (keyboard.sys, kbdclass.sys, i8042prt.sys), hashed them, scanned them: there were native. I am less sure on how to deal with the software certificate system, but I did checked all Microsoft and root certificates in the bank along with their signature with a clean system: nothing wrong.

Dear reader, any help or tip is welcomed! Am I missing something obvious? Could it be possibly not a rootkit but some kind of corruption? If so, how to detect it?

Just drop me an e-mail if you want to have a look on the dump itself.

Volatility outputs:

Rootkit in my lab?

Context

For now, I can’t tell much about the context, mainly because it may – or may not – involve other people. The only thing I am interested in is to spot the issue and understand precisely what is going on.

What makes the case really interesting though, is that it occurred on a fresh install of a Windows XP virtual machine. I aimed it to be a clean malware reversing snapshot. I noticed the weired behavior minutes after finishing the system install and setting up a bunch of reversing and live analysis tools.

So I bet that if I got some malware, it probably comes from one of those. At this time, unfortunately, there are too many and I could not spot the exact time, so I can not start the analysis from this angle.

This article is almost written in live, so pardon my mistakes. I will update it as soon as I find something new. Of course, I am really expecting your feedback, suggestions and corrections. I see it as a great opportunity to learn, even though this one may not be the easiest…

Symptoms

Two things alerted me quickly.

The first one was, at a point, the permanent failure of going through the full windows update process. Believe me, I have tried all ways.

The second one was the weird dialog when trying to access to the keyboard layout settings. It says “Incompatible driver detected“. To me, this looks like there is a keylogger somewhere…

Suspicious activities: the keyboard driver and windows update seem to be messed

Then, as I started to check around, more odd stuff came out.

I fired up Process Explorer, and soon realize that it was “unable to verify” the signatures of all the running Windows processes. I could not find anything else suspicious, though (no odd process, memory content looks normal, etc.).

On the left, Process Explorer fails to validate any Windows process.
On the right, expected behavior on a clean system.

Ok, while I am with the Sysinternal suite, why not scanning with Rootkit Revealer:

Rootkit Revealer cannot access to the SYSTEM hive of the registry

Interesting… and what about GMER:

GMER crashes when accessing the registry…

Oops! Now it crashes when it is accessing the registry…

For the fun, let’s see what happens if we try to set up an antivirus (Security Essentials):

Windows certificate warning when installing… Microsoft Security Essentials!!!

Nice one! Very suspicious! Note that after a full scan, Security Essentials reports me that the system is clean and everything is fine. I am so relieved. :)

Curious to see how my certificates are, I run certmgr.msc. I compared all Microsoft root certificates with a clean machine and could not see anything different. But again something happened:

certmgr.msc crashes

Oh, just one of my last attempts to do live analysis (this the WinPcap setup included with Wireshark):

WinPCAP installation also fails

Ok, so enough played. The thing seems to be nicely done, and live analysis is going to be way too hard and unreliable.

Memory Analysis

This is where I am now. I reverted to a snapshot prior to my live analysis attemps, confirmed the strange behaviors are still observable, and suspended the VM to get the vmem file.

So I have spent the last hours scanning the memory with, of course, Volatility.

So far, I have to confess that I found NOTHING. But analyzing the memory can be a harsh process when it comes to sophisticated threats, and I may have reached the limits of my skills.

But, anyway, I could not dream of a greater and more exciting opportunity to learn!

My discoveries, if there are, will be published in another article.

UPDATE: I forgot to tell that it is a Windows XP SP3 machine, but not fully updated due to the issues.

Soktspy

Soktspy is a small script that may be helpful for some investigation.

Sometimes, you may detect that some suspicious network traffic coming out from a machine. In general, it is easy to spot the process from which the packets originate. You somehow connect to the PC and look for open sockets.

But sometimes, the behavior may be very sneaky, consisting of one or two packets, at rare and random intervals. Unless you spend all the day before the screen, it may be very difficult to trace.

Especially with stock tools or without installing any intrusive hardware, which is also the reason why I did this tool. On a production server, you want to install as little dependencies as possible, right?

So here is the Soktspy, a python script that easily build into a portable and standalone executable to deploy on the target machine.

Once launched, it just loops in the background and log sockets that are created for some given peers (the IP addresses you found involved in the suspicious network activity).

Maybe, some other tool exist, but I could not find anything similar. Let me know if you have any suggestion. Anyway, it was a nice exercise to do :)

Download

soktSpy v1.2

Pre-requisites

  • Install Visual C++ Runtime libraries with vcredist_x86.exe (not necessary if the target machine happens to have Python already installed)
So far, I tested it successfully on Windows XP, Windows 2003, Ubuntu 11.10 and Mac OS Lion. But as it is a simple Python script, it is supposed to work on all platform.

Compiling

You may recompile the program as a Windows binary executable by issuing this command:

> setup.py py2exe

How to use

  • Copy soktSpy.exe and its configuration file config.cfg.
  • Edit config.cfg with the IP you want to monitor
  • Start soktSpy.exe.

Then, as soon as the sneaky process will send out a packet toward the monitored IP, a log record will be triggered:

The log file contains the following info, in that order:

  • Detection time (based on the system local time)
  • Process creation time
  • PID
  • Process Name
  • Protocol Family (2 = IPv4, 23 = IPv6)
  • Process Owner
  • Source socket (IP, port)
  • Destination socket (IP, port)
  • Socket Status

Future Plans

Please tell me if you have any idea on how to improve it.

For now, I plan to add a feature that will dump the memory of the suspicious processes when it is executing.

CVE-2009-3555: Safari not yet patched ???

The other day I was shocked to find this entry in my Apache logs:

[error] SSL Library Error: 336068931 error:14080143:SSL routines:SSL3_ACCEPT:unsafe legacy renegotiation disabled

It occurs appears when I try to use a SSL client certificate with Safari. Of course, authentication is broken as it just fails on an 403 error page.

So it seems that Safari is the last browser which was not patched against CVE-2009-3555 !

2009 !! At least, I quickly checked the other browsers I had around and they were fine: IE, Firefox, Chrome… I am having an issue with Opera also, but although I have not identified the problem yet, it seems unrelated (and does not throw the same error).

Note that I reported the issue to Apple, but I did not receive any answer. Silence on the wire.