1. Upgrade and purchase licenses for VSphere, VCenter, etc…
2. Use the VI client data store browse “Move” functionality
3. Try out a third party product like Veeam FastSCP
4. Try standard TAR/GZ combined with cp command
5. Try vmkfstools disk cloning

One important note. While testing some of these options, we became aware of a complicating factor. If you have created your disks as Thin provisioned and are moving to storage that has a different block size, the vmdk will become thick provisioned. This means that the “entire” contents of the fully thick provisioned disk will be moved, not just the used space.

Upgrade and Purchase VSphere Infrastructure Licenses

This option is one we continue to re-evaluate every 6-8 months. Working in a non-profit, we are always evaluating how to be the best stewards of the resources we’ve been provided. One of those resources is obviously financial resources. This seems like the only real option for moving running VMs.

While having VSphere Enterprise in place would make this process of migrating to new storage easier, we just cannot justify the long term expense. Our small environment just does not need what you pay for over the long haul.

As such, we ruled this out. Some downtime is perfectly acceptable in this organization.

Use VI Client Datastore Browse/Move Functionality

When using the VI Client, you can browse the VMFS datastores similar to a Windows Explorer window. This is accessed by viewing the storage configuration of the host in the VI Client. Right-clicking the particular datastore and choosing browse.

Obviously, this function does not work on running VMs. As a matter of fact, in one of our tests, we actually caused a problem doing so. This is a “move” not a copy. If the move fails mid-way through, some files will have been moved while others are still in their original location. Now you’ve got a bigger mess on your hands.

So, this option means the VM has to be powered off. Ok, we can deal with that. The other downside was that, as explained above, the new vmdk is thick provisioned and therefore, much more data is transferring that is really necessary.

Test 3rd Party Product Like Veeam FastSCP

I really do like the folks over at Veeam. I have used the monitoring /reporting product and like there licensing models. We thought we’d check out their free FastSCP product. Why? Well because is says “Fast” in the name.

The tool worked great. It did everything is said it would do. Like the other options, the VMs need to be powered down. We even tested copying between different ESXi hosts. Worked great. The only downside was the lack of speed. Yeah really! We did some reading and it appears that this may be a throttling of the ESXi product, not Veeam FastSCP. Bummer.

Try standard TAR/GZ combined with cp command

“Wow”, I though as I was researching. Why don’t we just tar/gzip everything and then copy to the new storage location and then untar. Well, not so fast our ESXi said to m as it spewed out an error that “tar: cannot store file ‘xxxxxxxxxxx.vmdk’ of size #########, aborting”. Yuck~!

It appears that BusyBox, the underlying unsupported ESXi console has a limitation of 4GB in ESXi3.5 and 8GB in ESXi 4. Therefore, trying to tar a 100GB vmdk file went no where fast. Check this option off the list.

Try vmkfstools disk cloning

This, we thought, is an interesting idea. Yes, the VMs still need to be powered off but perhaps this will shorten the downtime by lowering the amount of data that gets copied. See, as we talk about earlier, going to storage of a different block size will mean yout thin provisioned disks will begin thick. As such, the total size of the thick disk has to be transferred. This could siginificantly increase the copy time if you have lots of unused space on a disk.

The vmkfstools utility has a feature known as cloning. This allows you to make a clone copy of a vmdk disk to anyVMFS datastore. Now that’s all good but where this got us jazzed was the ability to force that newly cloned disk to be created as a thin provisioned disk. Eureka!

Now, first, make sure you remove all snapshots in the vm. Those are just going to cause us a headache.

So a command like this does the deed:
vmkfstools -i /vmfs/volumes/source/vmname/vmname.vmdk -d thin /vmfs/volumes/dest/vmname/vmname.vmdk


Note: you need to do this against the vmname.vmdk NOT the vmname-flat.vmdk. Be forwarned.

Some small scale, non-scientific testing gave us around 1.3GB per minute in transfer speed. Again, this is not true i/o but rather therate used for calculating down time for our migration project by taking the amount of used space on the source drive/time to create cloned drive = transfer rate.

Now, when the vmdks are moved, don’t forget to cp all the other VM files in the VM directories!

This looks to be the path we will be taking and thought we’d share how we arrived here.

12 Standard Screen Patterns – An interesting article from DesigningWebInterfaces.com. These patterns are specifically discussed in relation to the adoption of RIA platforms.

How To Get 1,645 Free Prints – I won’t even pretend to call this an article but it is a quick set of links to photo print online services that offer free prints to new customers. Read each services fine print for the free prints but hey, you can get some free prints without using your ink.

Google’s Next-Gen of Sneakernet – Old is new again. Even with all of the significant progress and exponential growth of computing power, network bandwidth, et al. The best way to deliver such significantly large amounts of data is via SneakerNet, er FedExNet.

Ebooks search engine – Interesting site where your search results are PDF documents only. Did find quite a few duplicates in my test search results, but also found some interesting Drupal PDF docs.

MySQL Cheat Sheets – Thumbnailed list of links to MySQL Cheat sheets.

Until next time, grace and peace to you.

Week of Warehousing Webinar Series – SQL Server Central

Join SQL Server authors and MVPs for a week of warehousing and get the latest on Microsoft Business Intelligence inside your environment. If you are in any stage of a data warehouse design, administration or planning you won’t want to miss out on this free “week of warehousing” webinar series,

Registration is open to the first 1,000 people in each session and seats are going fast.

5 Days 5 Data Warehouse Topic Summary

Designing a Data Warehouse

Monday Feb 23 9:30AM EST
Featuring: Erik Veerman
Improve the speed of your production can be as easy as giving your users a new place to report against. In this session, you’ll learn from SQL Server MVP, Erik Veerman how a data warehouse can benefit your organization and how to design one for SQL Server. You’ll also see at a high level how the data flows into the warehouse in a Microsoft data warehouse ecosystem.

Designing a Data Warehouse

Monday Feb 23 1:30PM EST
Featuring: Erik Veerman
Improve the speed of your production can be as easy as giving your users a new place to report against. In this session, you’ll learn from SQL Server MVP, Erik Veerman how a data warehouse can benefit your organization and how to design one for SQL Server. You’ll also see at a high level how the data flows into the warehouse in a Microsoft data warehouse ecosystem.

Loading a Data Warehouse with SQL Server Integration Services (SSIS)

Tuesday Feb 24 9:30AM EST
Featuring: Brian Knight
Now that you have a data warehouse designed, how would you go about loading the warehouse from your relational database that has been in production for years. Join Microsoft MVP and author Brian Knight in this session on how to use SSIS to load and synchronize a data warehouse.

Loading a Data Warehouse with SQL Server Integration Services (SSIS)

Tuesday Feb 24 1:30PM EST
Featuring: Brian Knight
Now that you have a data warehouse designed, how would you go about loading the warehouse from your relational database that has been in production for years. Join Microsoft MVP and author Brian Knight in this session on how to use SSIS to load and synchronize a data warehouse.

Introduction to Analysis Services (SSAS)

Wednesday Feb 25 9:30AM EST
Featuring: Siva Harinath
Tired of having to write hundreds of one-off reports? SQL Server Analysis Services (SSAS) can help clear your report backlog by giving your users the ability to perform ad-hoc reporting with pivot tables. Learn how to create a SSAS cube, how to process it and finally view quick reports to answers questions you didn’t even know you had about your business.

Introduction to Analysis Services (SSAS)

Wednesday Feb 25 1:30PM EST
Featuring: Siva Harinath
Tired of having to write hundreds of one-off reports? SQL Server Analysis Services (SSAS) can help clear your report backlog by giving your users the ability to perform ad-hoc reporting with pivot tables. Learn how to create a SSAS cube, how to process it and finally view quick reports to answers questions you didn’t even know you had about your business.

Reporting against a Data Warehouse with SSRS

Thursday Feb 26 9:30AM EST
Featuring: Paul Turley
In this session, you’ll learn how to report against your data warehouse and SSAS cubes by using Reporting Services 2008. Learn what’s new and easy ways to visualize data warehouse data in this comprehensive session for beginners.

Performance Tuning and Scaling a Data Warehouse

Thursday Feb 26 1:30PM EST
Featuring: JC Armand
Some of the largest databases in the world are data warehouses, scaling to dozens (or hundreds) of terrabytes in some cases. In this session, you’ll explore how to make your SQL Server 2008 database scale and easy ways to diagnose and troubleshoot problems. You’ll learn how to performance tune and scale the hardware to the OS in this vital session for most organizations.

Register Performance Tuning and Scaling a Data Warehouse

Friday Feb 27 9:30AM EST
Featuring: JC Armand
Some of the largest databases in the world are data warehouses, scaling to dozens (or hundreds) of terrabytes in some cases. In this session, you’ll explore how to make your SQL Server 2008 database scale and easy ways to diagnose and troubleshoot problems. You’ll learn how to performance tune and scale the hardware to the OS in this vital session for most organizations.

Reporting against a Data Warehouse with SSRS

Friday Feb 27 1:30PM EST
Featuring: Paul Turley
In this session, you’ll learn how to report against your data warehouse and SSAS cubes by using Reporting Services 2008. Learn what’s new and easy ways to visualize data warehouse data in this comprehensive session for beginners.

Week of Warehousing Webinar Series – SQL Server Central

So, I was asked by my lovely spouse to move the home office to another room for a particular reason. Unfortunately, the new location did not have an existing outlet for the broadband internet line. Immediately my eyes widened and a grin broke across my face as I knew it was an opportunity to try something I had been wanting to for quite a while.

Rather than run wiring and such, let’s go wireless. Yes, a wireless bridge was going to be the ticket. As it happened, I already had two Linksys WRT54G wireless routers, so I was in business. After spending a bit of time on the DD-WRT website, I had the details I needed and downloaded the bits I needed.

DD-WRT is a free alternative firmware for many major brand wireless routers and access points. Now, yes this may void your warranty but if you’re like me, that limited warranty was out many moons ago. DD-WRT firmware provides for more features and functions to be available to you. You can make your wireless router or AP can then do tricks you never knew it could.

This video gives a good idea of how we did it.

How did we get this to work? Here is the process we used.

DD-WRT Installation Instructions on the wiki.

1. Download the proper firmware for our version of the WRT54G. Use the compatability matrix to determine the correct DD-WRT version for your device and device version.
2. Performed a full 30/30/30 hard reset of the router
3. Open your web browser and connect to the router at 192.168.1.1 and login with the default username/password.
4. You should be in the Web GUI of the router.
5. Click the “Administration” tab
6. Click the “Firmware Upgrade” sub-tab.
7. Click the “Browse” button and select the DD-WRT .bin file you downloaded and confirmed.
8. Click the “Upgrade” button.
9. The router will take a few minutes to upload the file and flash the firmware. During this time, the power light will flash.
10. A new page will open confirming that the upload was successful Now wait about 5 minutes before clicking “Continue”.
11. Lastly, do another hard reset on the unit.
12. If flashed successfully you will now be able to access the DD-WRT web interface at 192.168.1.1

Now, since we wanted to utilized a wireless bridge, we needed to repeat the above steps on the second router. After the firmware updates were complete, we individually tested each router, connected to our broadband service to ensure both were working properly.

Next we followed this post on the setup the wireless bridge.

One thing that was a problem initially was the lack of machines connected to the second router (bridged AP) getting IP addresses from DHCP. It wasn’t until we read completely and did a bit of searching that we found out that this is a known problem with DD-WRT v24. Some folks report that it worked fine in earlier releases, but it appears to be a bug currently.

This was a bit of a bummer, but easily worked around for our small setup. We just setup the desktop machines with static IP addresses outside of the DHCP scope of the primary router but on the same network subnet. The default gateway and DNS were setup to point at the primary router (192.168.1.1) as well.

Immediately we could browse the internet from the office desktop machine which was connected to the second (bridged) router. Success! Now of course, DD-WRT offers many other features that the standard Linksys firmware does not. We’ll leave that as an adventure for you until we report more on those specifics.

Blessings.