Saturday, 13 October 2012

VMworld Europe 2012: Wrap up and the next steps...

Having blogged about VMworld Europe 2012 as it happened, I felt it was important to summarise the week and highlight the key points that will shape the next 12 months:

1. Get the basics right

A solid infrastructure foundation is essential. As vSphere deployments grow, both in terms of number of VMs and complexity of applications provisioned, the need to ensure best practices for storage, networks and vSphere configurations is a pre-requisite of any new projects. This can be accomplished through the following of vendor white papers and policy management tools such as Host Profiles.

However, a pragmatic approach needs to be taken to balance any performance gains against operational complexity. For example, while it may be possible to tweak the round robin path selection parameter, or implement jumbo frames on storage switches, the additional complexity can introduce additional management problems later on. So, while tuning the number of IOPS per path may result in a performance improvement for a small number of VMs, the benefit may be negligible when dealing with a large number of hosts, datastores and VMs. Similarly, jumbo frames may provide a minor throughput improvement, but if a new switch is added later, the system administration team must remember to apply the same settings to the new switch or else experience frame fragmentation.

In other words, keep it simple.

2. Learn to automate

VMware vCenter Orchestrator is moving from being a peripheral product to a core part of a complex cloud infrastructure. Therefore, I'd consider it a must-have skill to acquire in the coming year. The knock on requirement is that vSphere admins will need to have a basic understanding of programming languages and development methodologies.

What about other scripting approaches such as PowerCLI? In many ways, Orchestrator and PowerCLI are complimentary. PowerCLI scripts can be used to make repetitive individual tasks easier to perform, while Orchestrator is more about enabling workflow (of larger tasks). It's probable that PowerCLI (or other PowerShell) scripts will be called as part of an Orchestrator workflow.

Bottom line: Learn both.

3. Virtualise more

VMware's "Software Defined Data Center" (SDDC) strategy extends the existing virtual infrastructure and introduces the automation and orchestration of edge devices. If implemented well, this means that provisioning new services should be faster and less error prone. The opportunity to upgrade Enterprise Plus to vCloud Suite Standard enables the rollout of vCNS and vCloud Director, thereby forming the first step in moving to this new "agile" data centre, and also provides the foundations to build a private cloud on top of vSphere.

In order to ensure that a private cloud implementation is "done right", the VMware vCloud Architecture Toolkit will be followed. This will dictate some changes to the existing infrastructure, further extending the use of a dedicated management cluster and separate production clusters.

There is one challenge to consider: The added complexity that comes with the additional functionality makes the learning curve significantly steeper. Today, it is possible for the IT generalist to work with VMware in addition to their regular day job (which may involve supporting everything else in the IT estate). At first glance, the SDDC makes this a lot more difficult. There are more virtual devices to manage and more places to configure more settings.

The responsibility is therefore on the VMware senior administrators/architects to design using clear principles, document and create the orchestration workflows to ensure that complex tasks are well managed so that the IT staff responsible for operations can do their job without needing to fully understand the complexity of the environment.  

IT architects: Be prepared to invest time learning how all this fits together.

4. Make it easier for users

The truth is that today, end users have to jump through too many hoops to get their VMs provisioned and the cost model is still unclear.

Although VMware have demonstrated vCloud Automation Manager, capable of handling the provision request/approval process, by limiting it to customers running vCloud Suite Enterprise, the rest of us are left with no ability to provide a request/approval mechanism for VM creation. It is likely that something will need to be written in-house, perhaps using something like WaveMaker Studio front-ending a series of Orchestrator workflows.

There are other ways to make life easier for our users. As admins, we currently use vSphere templates to make VM deployments simple, but this should be extended to create a full vCloud Director application catalogue for our end users. For example, if our end users want a Red Hat Enterprise Linux server with Apache Tomcat already installed, we should be in a position to make a catalogue item available for this purpose.

Using the vFabric suite to deploy these applications is probably overkill, but there are alternatives. It would be a useful exercise to do some work with Puppet to look into the provisioning of applications.

5. Monitor and plan

Finally, as things get more abstracted and complex, there is a real need to manage this infrastructure and proactively plan for future growth. VCOPS Foundation will be the starting point for this, but it's possible (probable?) that a fuller featured edition may be required over time.

There is going to be a real need to perform capacity planning to ensure that existing infrastructure resources are utilised properly, and that additional hardware is available when required.


The above needs to be done in order to allow IT to keep up with the business demands of our users.  We need to stop being a stumbling block and instead come up with ways to deliver services faster.

The amount of servers and applications we now manage is greater than ever before and there is no sign that this will change in the near future (probably the opposite, and demand will continue to increase!).

We can't keep doing things the same way (it doesn't scale and it's too slow). We need to be smarter and more proactive about managing our infrastructure and providing the applications and environments our users need.

Thursday, 11 October 2012

VMworld Europe 2012: Day Three

The final day of VMworld began and I again opted to spend the majority of my time in the hands on labs.

Back to the Hands on Lab

The first lab of the day was based around the new features and capabilities built into the vCloud Networking and Security (vCNS) product, previously known as vShield Edge and App. This product is a virtual appliance based firewall with some advanced capabilities such as IPsec VPN and load balancing. Previous versions of vShield sat at the border of a vSwitch port group and provided only two interfaces (inside and outside). The new version can support multiple networks with the example given being a traditional three legged design for external, DMZ and internal networks.

Another useful feature of vCNS is the ability to put two appliances in an active/passive configuration. As part of the lab, the active instance was powered off and the failover kicked in, losing only three ping packets before the peer picked up the load. As previous versions of vShield Edge represented a single point of failure, this addition is welcome for mission critical or highly available environments.

The second lab was an epic example of tying a number of products together to illustrate the automation and provisioning of applications. Understatedly titled "Deliver Your IT Services in the Cloud", the lab utilised VMware Service Manager to request a new vApp as a consumer, Zimbra for the administrator to receive the request and then action it in VMware Service Manager, which then kicked off a workflow in Orchestrator that provisioned a VM in vCloud Director and then installed the web application using vFabric Application Manager. This advanced level of automation and integration across the various products provided perhaps the best illustration of where VMware are going with its Software Defined Datacenter strategy. In contrast, it's interesting to see how little focus there is on the "traditional" VMware strengths of virtualising compute, network and storage. A hint to competing hypervisor vendors: the world has moved on and it's now about building and managing automated application stacks built on top of public/private/hybrid clouds.

The third lab was a focus on the new features of the Distributed vSwitch. As new Enterprise Plus users, the Distributed vSwitch will be a new addition to our infrastructure and the lab provided some useful troubleshooting tips.

The final lab covered the new features in the latest release of VCOPS. The analytics engine and presentation of data in VCOPS is amazing (no change there) and it was interesting to see how this is becoming the management platform for VMware.

vCenter Operations Manager

By which point my brain was struggling to assimilate any new information! There was no big bang event to close the conference, just a steady stream of people finishing up and heading back to their hotels.

There is still much to reflect on, and my deliberate strategy of tackling the Hands on Labs at the expense of attending the sessions means there will be significant ongoing catching up online over the coming weeks and months.

The value of attending a conference such as VMworld is that it gives an opportunity to deeply dive into the technology we use everyday, familiarise ourselves with the developments that will be with us soon, and allows us the space to think, plan, learn, discuss and ultimately equips us to do our jobs. For those of us who are passionate about the work we do, it's an amazing experience and I'm personally very grateful to my company for making it possible to attend.

Wednesday, 10 October 2012

VMworld Europe 2012: Day Two

Day two started with another keynote, this time focused on "end user computing". VMware demonstrated a number of new features to View, Horizon and application delivery to iOS and Android devices.

None of this really interests me, so, meh.

My first session of the day was around the vCloud Architecture Toolkit, which I only stayed in for the first ten minutes. The pace of the presentation, along the presenter providing a definition of cloud (really? It's 2012!), led to me bailing and going the Hands On Lab instead.

Hands on Lab

At the lab, I took several sessions throughout the day. Two sessions were based around the vFabric Application Director and vFabric Data Director. These two products are designed to facilitate the provisioning of applications to a cloud environment. In the first lab, you had to provision a database (MySQL), application (Tomcat) and web (Apache HTTP) server, connect them dynamically, and then scale out the middle tier. The resulting blueprint is then built as a vApp which is available through vCloud Director. Clever stuff, but quite a steep learning curve (following the guides were easy, but starting from scratch would be a formidable challenge!).

Of interest was the list of options of components that can be used in building an application blueprint. In addition to the open source stacks were objects for SQL Server and IIS, although the lab didn't touch on these components.

The vFabric Data Director lab was similarly interesting, providing a single web front end into the management and deployment of Oracle and Postgres databases. The database instances could be cloned, resources hot-added and SQL run through the interface. Of interest to me was that the underlying mechanism to manage these instances is to manipulate and clone VMs. In other words, the object that is being managed is a VM. The application doesn't appear to be doing anything to enable multi tenancy within a single database installation (or if it does, this wasn't covered in the lab). It was very impressive to be able to take a Postgres 9.0 installation, rapidly copy the VM using linked clones, and then apply an upgrade to 9.1!

For more information (and to see what the above looks like):

A short video introduction to vFabric Application Director
A short video introduction to vFabric Data Director

The final lab of the day was on the new vCenter Orchestrator, a product most of us VMware admins have had for years, but few of us have used. This lab went through the creation of workflows that are then run against objects in vCenter. The lab then extended this with the use of plugins, showing how Orchestrator can be used to manipulate Active Directory. This is something that deserves a greater look and as a result, I bought the VMware Press book, Automating vSphere with VMware vCenter Orchestrator.

While in the lab, I noticed a plasma with a display that looked as if it was showing a Splunk dashboard. On wandering over to the screen, I discovered that this was labelled "VMware Strata". On enquiring with the lab staff, no one appeared to know what it was, but they thought it may be a new product based on a VMware acquisition that would eventually make it into that all-powerful monitoring and reporting tool, vCenter Operations Manager.

Mysterious VMware Application

Following the labs was the "Hall Crawl", providing another opportunity to meet the vendors. This included a very useful discussion with HP on the use of 3PAR storage and the unique features it provides. The thin provisioning and zero reclaim are very impressive ways to optimise and manage storage usage, although there is currently no deduplication or compression as found in NetApp filers.

The day was finished with the VMworld Party, a huge event with a cool 80s retro computer games theme, many activities and games, live acts and a constant supply of food and drink that ran late into the evening.

The VMworld Party

Tuesday, 9 October 2012

VMworld Europe 2012: Day One

Arrived yesterday afternoon in sunny, hot and humid Barcelona in preparation for VMworld Europe 2012. Having registered, taken a lab session (on the vCloud Suite), settled into the hotel and eaten an evening meal in a small, local cafe (where the owner knew no English and we knew no Spanish, which made the whole affair very interesting), I made sure I had an early night because this conference is going to be packed.

And so it is!

And so it begins

The Keynote

The keynote kicked off at 9am. Not a huge amount of new stuff announced after the US show, but there is a new version of vCenter Operations Manager (aka VCOPS) which now includes application awareness so you can drill into a VM and get the state of the underlying application (such as the RAM utilisation of an individual Oracle instance). While this is very impressive, unfortunately, VCOPS remains reassuringly expensive (but see below for some good news).

A new plugin was shown that allows the vSphere client to manage third party hypervisors. This, along with both DynamicOps and Nicira supporting multiple hypervisors, is a new approach by VMware and probably reflects the reality in many datacenters.

The vCloud Automation Center was revealed, which is based on VMware's acquisition of DynamicOps. This showed the ability for end users to request new VMs from a catalogue and to have management over their VMs (power on, off, restart etc.) through a web portal. This looks like an essential component in the provisioning process, fulfilling the self-service aspect of the cloud while facilitating an approval process, but according to the VMware website, it's priced at $400 per managed VM(!), which makes it too expensive for all but the largest customers.

The new vFabric Application Director was also shown, with the introduction of the idea of "application blueprints" which take the concept of VM templates to the next level, allowing a drag and drop approach to building an application stack. The Cloud Application Marketplace (aka App Store) was shown where third parties can publish their own blueprint components, allowing for even more sophisticated application stacks. As an example, the screenshot showed some Riverbed components, so you could imagine integrating WAN acceleration into your one-click-to-deploy vApp. The list price for this is $6250, which, like the above, is going to be difficult for many of us to justify, no matter how cool.

A couple of screens were shown of the IT Business Management Suite, which would make a CIO very happy. Lots of cost related metrics and pretty dashboards, but not something I'll personally have any need for.

The keynote ended with a demo of a social networking project. In the demo, hosts, clusters and VMs are social network entities that can be "followed" and "liked" in the same way our Facebook friends can be "followed" and "liked". If a host loses a datastore, it posts a message to the news stream. Other hosts that have the same problem "like" the original message which can be used for analytics. If dozens of hosts all report that they have lost access to a datastore, then there's probably a serious issue. All sounds a bit gimmicky, but it did look pretty decent and got the biggest, most spontaneous applause of the keynote(!).

The main things I took away from the keynote are:

VMware offer so much more than just hypervisor-based virtualisation. Where other hypervisor vendors sometimes compare vMotion/Live Migration capabilities, play "My VM is bigger than your VM" (VMware are guilty of this too), and fight to see who offers the most features for free, the VMware strategy is now about the datacentre and everything in it, automating and orchestrating according to policy.

In VMware language, this policy based automation is referred to as the "Software Defined Datacenter" (SDDC) and is a logical continuation of the ability to virtualise resources. We're now comfortable virtualising compute, storage and network connectivity, but VMware are looking to extend this to network edge devices (load balancers, WAN compression, firewalls etc.). While virtual appliances already exist for some of these, the big difference is that the SDDC is built on the idea that these devices can be scripted, orchestrated and therefore automated by policy. This further extends (through the vFabric suite) into the application layer.

Stop and think about this for a bit, because this is actually a big deal. What happens when your entire datacentre is software defined? You get to throw the word "agile" into the mix...

Want to know more about the SDDC? Check this short video:

A negative comment regarding the keynote announcements, it is frustrating that customers who opted for the top end solution in the VI3.5 days with the Enterprise edition were left in the cold with 4.0's introduction of Enterprise Plus and subsequently upgraded, are now migrated to vCloud Suite Standard which still leaves many useful features out, only available in vCloud Suite Advanced and Ednterprise. I understand that each edition adds many new features, but it seems like a ploy by VMware to constantly generate new revenue by making customers pay out for ever bigger software solutions.

Post keynote

Following the keynote, I had a quick wander around the Solutions Exchange and got into a conversation with a Cisco or NetApp employee about the new ExpressPod which appears to be a low cost FlexPod built around the C-series rack mount server and Nexus 3000 switches (with NetApp storage). This may be worth investigating further...

The Solution Exchange

Also in the Solution Exchange, I had a conversation with Veeam about support for vCloud Director and end user, self-service restores on backed up VMs. Nothing yet announced, but Backup and Replication release 7 should be interesting next year. There was also the suggestion that tape support was coming, but on later reflection, this may have limitations for those of us running a purely virtual Veeam deployment.

The Solution Exchange was also the venue for an unexpected encounter with Brent Spiner who was signing photos and chatting to people!

It's Data!

And, to the probable dismay of my wife, I now own a red fedora, courtesy of Red Hat (and she thought last year's bandana was bad...).

The rest of the day was filled with sessions, including an excellent multi-vendor presentation on storage best practices by Chad Sakac (EMC) and Vaughn Stewart (NetApp). Storage is a topic of interest to me and I've read up a fair amount on it, but was pleased to find myself noting down many "to-do" items for the infrastructure I manage. Very useful!

The second session attended was on future storage developments, including vVols, Virtual SAN and Virtual Flash. Interesting technology and it's useful to know this stuff is coming down the road.

While sitting in the Hang Space reading up on the new developments, I came across a reference on VMware's website that a new, entry level version of VCOPS called vCenter Operations Manager Foundation, will be FREE to all vSphere customers. While this removes about 90% of the product's feature set, it will be a good starting point for infrastructure management and alerting.

Part of the Hang Space

The early evening was spent in the Solutions Exchange for the Welcome Reception, basically an opportunity to engage with vendors while eating canap├ęs and drinking free wine/beer/juice/water.
And that, for me, was enough for the day! There are parties running late into the evening, but tomorrow promises to be an equally busy day, so time to get some head down time.

Other thoughts...

On a less IT related note, this venue is very impressive (but not noticeably better or worse than Copenhagen). The conference centre is huge, requiring the use of those automated walkways normally found in airports to get around. There is a constant availability of free food and drink on offer and plenty of space to sit, relax and reflect on all the new developments. Credit to the event organisers who really understand their audience and do everything to keep attendees happy and focused!

As with last year's conference, it's interesting to note the use of smartphones and tablets, alongside laptops. It seems that most people have converged down onto two devices, not one. Of all laptops being used by conference attendees, I'd guess that easily 50% are MacBooks, implying that either VMware admins prefer to spend time doing VMware stuff and not installing Windows patches, or they are easily seduced by shiny hardware...