The Forging of the new Network/VMware/Storage Professional

When I first started out in College I needed a work study job. Since I liked to help people with their computer problems I applied and was hired for a position doing phone and in person support for the University. One of the best things about starting out at a school they don’t mind teaching. Our trainer said that in previous years new employees would be slotted into being Windows or Mac or UNIX support. He said we would be Wunder-Cons (our title was consultant instead of help desk dude). We had the privilege of having to support all of it. This thrust me into the world IT no matter what the piece of paper from USC said I was a Bachelor of.

I believe a new kind of Wunder-Consultant/Engineer is being made. With the announcement of the Nexus 1000v last fall the line between Network Engineer and Datacenter/Server Engineer is getting blurred. The SAN and Server Engineers have had this tension for a while now. Virtualization is a fun technology to learn but who gets the responsibility? I have seen where the SAN team owns the ESX’s and the Server team operates the VM’s like they are physical. The Network team not trusting or understanding why they want a bunch of 1GigE trunk ports. Across larger organizations it would look different but the struggle may be just the same. Who is in control of the VM’s? Are they secure? Who gets called at 1am when something dies? This is internal to the IT department and does not consider that Sales doesn’t want to share memory with accounting.

I can see these technologies pushing engineers into being jacks of all trades. To be a truly Architect level in VMware today you must be awesome with storage and servers. You have to be able to SSH into an ESX, choose the right storage for an application, and setup templates of Windows 2003. That is an easy day. You already will have to troubleshoot IO (because all problems get blamed on the virtualization first).

With the Nexus 1000v I picture the Virtualization Admins learning the skills to configure and troubleshoot route/switch inside and outside the Virtual Infrastructure. Add to that Cisco’s push this year with 10GigE and FCoE and their own embedded virtualization products. The lines between job duties are getting blown away.

Who is poised to become the experts in this realm? The network, server or storage admins? In this economy it may be good to know how to do all three jobs. I am sure corporations would love to pay just one salary to perform these tasks.

Randomly I though how would this relate to SOX? Could it pose any problems with compliance? I will save that for next time.

Virtualizing Exchange Mixed Messages from Microsoft, VMware

The release of this article at You had me at EHLO (aka Exchange blog team) and the previous links from VMware’s Virtualized Exchange discussions. Seems to be almost too coincidental that Microsoft’s Exchange blog published an article about virtualizing Exchange on Hyper-V a few days after VMware mades posts and hosted a discussion concerning Exchange. Microsoft’s conclusion is also completely contradictory from VMware’s findings. One says only do this in small environments, the other says we just virtualized 16,000 mailboxes and it is great. First thought is someone is misleading the reading public.

Then my profound (at least in my own mind) thought was no this makes total sense. Microsoft admits that Hyper-V is unable to run Exchange 2007 at an Enterprise level, but it is fine for small offices. VMware (and many storage partners) has proven Exchange can run just fine for large environments in ESX.

So the conclusion for today is this:
1. Hyper-V is not ready. (I am not the first person to say this.) Not just because they can not live migrate a VM. Hyper-V will have to live in the lab for a while longer.

2. VMware ESX is able to live in production environments for the big and small. It would seem VMware is still years ahead of the big guns at MS. So 2009 might not be the year for VMware to watch out for Microsoft. Maybe 2011.

Virtual Twits Powershell Script

I know, about 9 other people already posted Alan Renouf’s list of virtualization people to follow in twitter. The really cool thing is the powershell script to add everyone. I promise I was going to mention it even before I was added to the list. Now that @2vcps made the list it just seems like shameless self promotion. Oh well.

Great work Alan with help from MindofRoot and thanks for making the list.

*Alan, thanks for pointing out my error…

Being a VMware vExpert would be vAwesome

The VMware vExpert program was announced today. I could think of many people in the vCommunity (you know the name is coming) that would easily get this award (at least in my opinion). I will admit I do not know everything and from this website it is easy to discern that very fact.

When I have a problem with anything virtual, it is very rare to not find an answer in the forums or one the many blogs. So much good information out there. So I would like to say thank you and keep being awesome.

Photo from BeAwesomeInstead

Tale of Two Datacenters

I wish I had my camera so I could share with you the difference between the last two server rooms I was in.

I will try to describe. The first was an organizational dream. Color coordinated patch cables, wire management actually used. You could actually see the port numbers on the Cisco 4507. Even more the configs were labeled. So when you were logged into the switch you knew what was going on. Just a brief picture but the rest of the room was the same, the blades, SAN and everthing else was how I wish everyone would be.

In contrast, the next day, I was trying to upgrade some ESX servers. The cables had no rhyme or reason. The switches had no labels and the trunks and access ports were done in the most backward way I could think of. It would be too hard to describe. I vMotioned all the vm’s to another host. Shutdown the host to upgrade to 32GB of Memory. After unplugging everything and trying to slide the Dell 2950 out of the rack I discover the fibre to the server above it is running through the “handle” on the back of the 2950. So the server will only slide about 1 foot. Great, so I hook everything back up and vMotion everything off of the host above. Move the troublesome fibre, and decide to upgrade that server. I start to slide it out for the memory upgrade. The server on top of it is not on rails and starts to slide with the 2950. So I now need to get downtime (that is a physical server) to take that server down so I can upgrade the memory of the ESX servers then upgrade them to ESX 3.5 update 2.

So I would guess one of the quickest ways to eat all of my billable hours is to spend hours fighting poor datacenter work and eventually not get anything done.

Virtualization Security Roundtable

Would like to help spread the word about the Virtualization Security Roundtable it will take place this Thursday January 15 at 230 EST.
Security topics are outlined in the linked article. I would have to say this is a topic that I really want to master.
We consult with many financial institutions and being quicker on this subject would help me answer some of the objections to VMware. Not only to have the right answer but also be able to solve common problems.

Like always I will not be available for the call this week, but I will put in on my calendar so I can go ahead and listen to it every other week.

Education and Virtualization – Oh, the Possibilities

Saw this article the other day. Really good to see the Alma Mater take a plunge into Virtualization. I spent four and half years starting my IT career at USC working while I was studying.
I think amazing things could be done in education with virtualization. That isn’t just Virtual Desktops for computer labs, although that is a good one. Imagine being able to learn programming in a windows environment you could build up and tear down and build up again in an hour?
I remember getting warnings from root for leaving a process running (on accident) on the Unix system for a couple days. Separate virtual machines running linux would be awesome for this.
It has been almost 10 years since I finished at the University. I wonder what ways virtualization is making learning easier. I bet students are coming out now knowing almost all of what it took me the last few years to squire on the job.

Possibilities:
1. Virtual Desktops
2. Dedicated VM’s to learn server OS
3. Research into Malware/Spyware and other Security issues
4. Computer Engineering
5. Application Development and Testing
6. Ease deployment of Apps for teaching.

I am not a huge visionary, I bet there is tons of other ways… Any thoughts?

Fibre or Ethernet Saturation – Which comes First?

I was thinking about how far I can scale a VI3 Enviroment yesterday. I started to think, and that can be dangerous.
What will saturate first? The Fibre network or the Ethernet network?
So in my envisioned setup it would have dual quad core processors so if I can still do math that is 8 cores. If I might fit 4 x 1vCPU virtuals per core, I could theorize 32 VM’s per host? Now lets say I bought 2 of those quad port NICs for each host, so to be simple there is 8 network ports per host. Finally, lets say I have 2 single port HBA’s connecting to the fibre and I am lucking enough for it to be 4gb all the way to the SAN.
We have 2 Cisco 48 port 3560 GigE switches for the ESX hosts to access and 2 24 port Brocade Fibre Switches. So I scale my ESX hosts to fill the Cisco switches and it tops out at 11. I will use 11 ports on each Brocade, the Storage Processors use 4 more ports.
So what fills up first?
– Fibre
– Ethernet
– Disk IO

Practically my bet would be on Memory, but lets say memory can go as high as we need.

Another snag is CPU resources, to generate enough network traffic to kill that many GigE Nics I would think the CPU’s would pin out first?

I really wish I had a good lab with lots of vendor equipment I could test and try to break. That would be fun for me.

ESX Commands – esxcfg-configcheck

The ESX Quick reference has information on this command.

I didn’t find any posts in the VMware Communities or the Knowledge base with any reference to this command. From what I can cypher it checks the settings of the /etc/vmware/esx.conf . I wonder if it does any more or less.

http://screencast.com/t/flViWgGth3

So really, does anyone know the insides of this command?