Cisco – Phocean.net

Pentest of a Wi-Fi network with Cisco NCS

phocean — Fri, 04 Oct 2013 12:46:56 +0000

I had a chance to audit this device during a Wi-Fi pentest. Cisco Prime Network Control System is a Wi-Fi controller that allows to manage multiple access points and centralize their configuration: Wi-Fi settings, access control, security, etc.

I was surprised how easy it was to compromise this equipment, thanks to default credentials. Of course, Cisco published a patch… however how many network teams would have applied the patch ? Routers, switches, Wi-Fi controllers, especially when they are not part of the core infrastructure, are often dropped and forgotten for years…

Here is the vendor advisory, which is about “unspecified” default credentials. With a little of ninja googling, I managed to glue the pieces.

It was not that easy to find the credentials in questions, but the guys from Tenable Security managed to get the info: wcsdba / wcs123.

Now, you will think, “ok, but we need to find a way to reach the database…’. Piece of cake! By default, the device exposes an Oracle listener on port 1522.

Then, we would need to know the Oracle instance… Don’t think too much, don’t even look up at your wordlists, this is just: WCS.

Of course, as you should guess now, the account has DBA privileges. :-)

So, in summary:

Oracle listener on port TCP 1522
Instance: WCS
Account: wcsdba
Password: wcs123
Privileges: DBA

Now you can do pretty much every thing: control network settings, grab and crack password hashes, etc.

Besides, there is also an XSS on a login page…

Well done Cisco, for not hardening anything. :-/

Have fun!

Network virtualization and the DMZ paradigm

phocean — Sat, 30 Apr 2011 19:15:53 +0000

The virtualization buzz

I have recently worked on network virtualization. Many people, especially the network guys, have been recently excited with the VMware Vswitch or Cisco Nexus stuff. It is something that I understand because virtualization is cool. It brings many convenient features that truly make the life easier.

But what about the security? Convenience and security rarely come together, right? Oh, wait… we are in 2011, so lessons must have been learned. After all, Mr Salesman swear that it is more secure than ever. Convenience and security packed together, he says… it sounds promising. Let’s dig a little to find out what they won’t tell you…

I will focus on what really changes with virtualization : the architecture. One of the main goals of the technology is to reduce the number of physical devices to cut the costs, save space and energy. Of course, it goes with a simplification of the physical architecture. Therefore, some features previously handled by dedicated physical devices are now handled logically by a unique piece of hardware.

This obviously goes against the security best practices about designing network architectures with various degrees of exposure. But has the technology evolved so much that we should reconsider these recommendations?

VMware Vswitches or Nexus 1000V

These technologies are similar in the sense that they are designed to work directly inside the VMware platform. Vswitches are integrated with the solution of VMware, while Nexus benefits from the experience of Cisco and bring more layer 2 control (more settings, more protocols).

As well on the architecture documents of VMware as within the administration interface of Vcenter, it appears so easy to create segregated switches and build this way in a few clicks a DMZ architecture:

But it is slightly different in reality, as Brad Hedlund from Cisco shows in an interesting article: the vswitch illusion and DMZ virtualization. In short, whether you are using VMware Vswitches or Nexus 1000V, a single threaded program runs all the configured virtual switches. In clear, all the virtual switches share the same memory space. So, any vulnerability in the code would compromise all the switches, in other words: the entire network. And, not a surprise here, there have been many vulnerabilities. Just browse a CVE database if you want to check.

So you don’t want to rely on such a design for your datacenter, right?

Nexus 7000

In the case of the Nexus 7000, it is a little bit different because most of the switching work is handled by specific hardware, which have a much smaller attack surface than the vswitches stuff. But is it really safe?

The Nexus family is quite new and from what I could witness, they are quite pushy selling that. Because it is new, there is still neither much info surrounding the technologies used, nor user feedback, nor security research. Anyway, below is a quick sum-up of what I could find.

A few words about the architecture

In a layer 3 Nexus architecture, Nexus 2000, 5000 and 7000 are designed to work together. Nexus 2000 are basically top-of-the-rack port panels, with no intelligence. Nexus 5000 takes care of most of the layer 2 switching, while Nexus 7000 adds layer 2 functionalities and layer 3 support. Nexus 2000 and 5000 can work without the 7000, but in that case there is not so much difference with a classic layer 2 switch in terms of security (but it has the advantage to be more flexible to integrate in a datacenter). This and this may help you to visualize the differences.

So we will focus on the Nexus 7000 architecture, which bring VDC as a way to handle DMZ architectures. VDC are somehow similar to VLANs. But whereas VLANs virtualized LANs on a switch, VDC virtualize switches. So, on the same Nexus 5000 device, VDC will add the capacity to have multiple virtual switches which are in theory properly isolated.

This is a very basic sum-up for what we are interested in, but if you want to learn more, I encourage you to read the Cisco whitepaper about VDCs.

The flaws

Now that the presentations are made, the downside…

George Hedfors is the only researcher that worked notably on this platform, as far as I am aware. He made some really great findings, that you can discover within his slides.
At the time of his work – 2010, it appeared that the NX-OS consisted of a Linux Kernel 2.6.10 (released in 2004!). We can imagine that the OS has been signifiantly customized and hardened by Cisco. They may have include NX-bit support (included since 2.6.8 and later improved). However, there is probably no ALSR support (2.6.12), no MAC system (SELinux or Tomoyo). Of course, I may be wrong but I haven’t found any documentation about that and my Cisco contact did not provide me with any consistent detail.

Anyway, he found a bunch of design flaws:

Poor CLI design: there are 686 hidden commands (system, debugging) that can be launched as root (sudo without password). One of these command is gdb, which can start a network daemon as root. The attacker can then connect to the socket to attach to any process on the system to elevate his privileges. Of course, it requires some shell access, so the exposure is limited. However, it is very instructive of how the system was designed!
Insecure daemon configuration: Daemon are not chrooted and run with the root user.
Embarassing CDP vulnerability : a vulnerability from 2001 was reintroduced in the code handling CDP. So it is possible to crash a daemon running as root. What if another vulnerability on a layer 2 daemon (vtp, hsrp, stp…) was discovered and allowed to rewrite the stack? Game over, the attacker is root.
Strange hidden account : there is a ftpuser hidden account with a dumb password (nbv123). Secret backdoor? I don’t know, but anyway it is not serious at all and should have been revealed by any consistent audit.
Shell design flaw: the VSH shell accepts a parameter (-a) that allow to spawn any command over the security roles normaly in place.
You can also get a root shell by simply spawning ssh `/bin/bash` from the CLI.

To any serious security guy or unix administrator, these should look like amateurism. And what’s the hell are all the security audits for?

So concerning the Nexus 7000, it is obvious that at best it is not specifically designed to be secure, at worst it was simply as poorly designed (or released too quickly) as most stuff.

Conclusion

In conclusion, one thing we can tell for sure is that none of the virtualized networking solutions are designed to be secure. Of course, all these flaws are hopefully already or will be soon fixed. But, despite what Cisco may claim, the facts are here: there is no VDC miracle. The Nexus platform is certainly great, but not more bug-free, flaw-free than any other piece of code.
No virtualized architecture can give the same degree of protection than physical segregation.

In the case of Vswitches or Nexus 1000, the attack surface is just too high to use it for DMZ segregation if you are serious about security. The vulnerabilities are already here and it will be feasible for a skillful and motivated attacker to own your datacenter.

Concerning the Nexus 7000 and its VDC, the attack surface is considerably reduced because there is less code and fewer protocols at layer 2. However, it is undoubtly less secure than physical segregation. Any zero-day vulnerability would potentially expose the datacenter (and we all know that some zero-day sometimes take years before coming to the public, which is a lot of time for the criminals or the government agencies to exploit it). You can’t take it lightly when it comes to the whole datacenter integrity and it doesn’t make sense if you have expensive (in cash or in labor hours) security at upper layers.

But, of course, it may depend on what you have to protect. If your datacenter hosts sensitive data for your company’s buisiness, then you should think twice on how you deploy virtualization or use the cloud.

Don’t get me wrong. These technologies are great and very useful. In many areas, there are an improvement. Simply, they must be used with as much care as always. Concerning the DMZ topic, as far as I am concerned, I will not rely on virtualization and keep physical segregation between zones, supported by different devices from different makers.

One thing I keep an eye on, though, is the development of virtualized firewalls, IPS, etc. In a few years, if these technologies should became really mature (enforcing segregation on all OSI layers) and the hosting OS security should really improved, most of the concerns here would be addressed.

Packetlife.net

phocean — Sat, 20 Feb 2010 08:54:19 +0000

I added a new link to Packetlife.net, which is a really amazing website about networking, with focus on Cisco stuff.

The work that the author made is truly impressive : blog, cheat sheets, gentle forum and even a lab (I haven’t test because I don’t need it, but if you are a student, it is great for students and so far I have never seen any other website offering it), all integrated with a quality level that is rare to find even on professional website.

This is one the best website I have never seen in all categories and certainly the best about Cisco networking. If you are interested in this topic, visit there right now.

Netios 0.73

phocean — Wed, 10 Feb 2010 14:51:59 +0000

Netios 0.73 fixes some bugs and

Complete changelog :

2010-02-10 (0.73) phocean

2010-02-10 (0.73) phocean

* remove useless options
* fix various bugs

Check there (tools page) for more details and a download link.

Netios 0.72

phocean — Fri, 15 Jan 2010 10:20:55 +0000

Netios 0.72 fixes some bugs with the show_run mode and large config files. I also found some issues concerning the prompt detection, so it should be fixed now.

Complete changelog :

2010-01-14 (0.72) phocean

2010-01-14 (0.72) phocean

* ciscoclass.py : forgot to remove a debug print
* ciscoclass.py : finish and fix a bunch of bugs in the show run function, format the config file properly
* ciscoclass.py : fix the prompt regex

Check there (tools page) for more details and a download link.

Netios 0.71

phocean — Sun, 20 Dec 2009 16:34:37 +0000

I release a new version of Netios : 0.71.

There are a lot of changes, starting with cosmetics, but the biggest one is the support of multiprocessing.

It is now able to process several routers at the same time, so using it on a large list of machines results in a big speed up.

A downside is that it now requires at least Python 2.6, as multiprocessing started to be supported with this version only. Most Linux distributions now include Python 2.6, but still not all. Anyway it will be more and more the case. If you can’t uprade your distribution, you can stick with 0.60 which still do most of the work fine.

It is also now able to fetch a configuration file remotly, but it requires more testing before I feel confident in the way it works.

The complete changelog :

2009-12-20 (0.71) phocean ;

2009-12-20 (0.71) phocean ;

* ciscoclass.py : handle correctly the cisco pager — More — so that “show run” mode should work even with large config files
* sshclass.py : allow to override terminal size system settings (make use of the cisco pager to avoid filling the buffer)

2009-11-16 (0.70) phocean (private release)

2009-11-16 (0.70) phocean (private release)

* implement multiprocessing
* improve code documentation
* clean up UI
* reduce useless logging
* netios.py : bug : missing startTime parameter in f_skip_error and f_command functions

I cross my fingers so that there are not too many bugs, but if so, please don’t forget to report it to me.

Check there (tools page) for more details and a download link.

Netios

phocean — Sat, 07 Nov 2009 15:53:30 +0000

I just released an alpha release of a little tool that may help network administrators with a large park of Cisco routers or switches :

Netios is a little tool aimed to help network administrators to administrate a large number of Cisco network devices.
Providing it with a list of equipments, it connects within SSH to remotly apply IOS commands.

It can automatically :

retrieve and export in a CSV file the list of local users

update the local user, the enable password

change NTP settings

execute a file of customed IOS commands

retrieve configuration files

It can read the targets from the command line or from a text file.

Its primary goal is to improve the security by making it easier to renew regularly the local password of these equipments, but it can do more convenient things (and I will continue to work to add more of them).

Check there (tools page) for more details and a download link.

Cisco configuration buffer full

phocean — Sat, 26 Sep 2009 03:30:08 +0000

I got this weired message while trying to save a configuration :


#wr mem
Building configuration...
% Warning: Saving this config to nvram may corrupt any network management or security files stored at the end of nvram.
Continue? [no]: no
% Configuration buffer full, can't add command:
************************************************************************

Looks scary at first, but normally it should be solved with a single command :


# configure terminal
(config-ter)# service compress-config
(config-ter)# end
# write mem

I guess that there are little chances the issue remains after that, but if so, well, you may be in trouble…

Netflow on MLS switches

phocean — Wed, 23 Sep 2009 20:12:50 +0000

I had some trouble today at office with monitoring flows going through a MLS switch.

The Nfsen collector was working well for our routers, but I came to realize there were something definitely wrong with the traffic going through a Cisco 6500 switch.

The traffic reported was way below the real one.

After some research, what I suspected was confirmed by this documentation from Cisco.

Some more steps are required to make Netflow report flow processed by the switching module.

Otherwise, as only the first packet is routed and the following ones are switched by CEF, Netflow will only see the first packet at layer 3.

That explains why the reported traffic was ridicoulously small.

So, where on routers the following commands are enough :

(config)# ip cef
(config)# ip flow-export version 5
(config)# ip flow-export destination 192.168.1.1 1234
(config)# interface fa 1
(config-if)# ip route-cache flow

On MLS switches (like 6500 or 7200), add :

(config)# mls netflow
(config)# mls nde sender
(config)# mls aging long 64
(config)# mls flow ip full

The commands are described there. Unfortunately, as often with Cisco, they may vary depending on the IOS version your are running.

IOS : Configuration buffer full, can’t add command

phocean — Fri, 29 May 2009 15:19:21 +0000

This error message suddenly showed up on one of my router when I tried to save its running configuration.

service compress-config allowed me to workaround this buffer problem :

router#wr mem
Building configuration...

% Warning: Saving this config to nvram may corrupt any network management or security files stored at the end of nvram.
Continue? [no]: no
% Configuration buffer full, can't add command:
************************************************************************

%Aborting Save. Compress the config.[OK]
router#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
router(config)#service compress-config
router(config)#exit
router#wr mem
Building configuration...
Compressed configuration from 21787 bytes to 7991 bytes[OK]