1. Upgrade and purchase licenses for VSphere, VCenter, etc…
2. Use the VI client data store browse “Move” functionality
3. Try out a third party product like Veeam FastSCP
4. Try standard TAR/GZ combined with cp command
5. Try vmkfstools disk cloning

One important note. While testing some of these options, we became aware of a complicating factor. If you have created your disks as Thin provisioned and are moving to storage that has a different block size, the vmdk will become thick provisioned. This means that the “entire” contents of the fully thick provisioned disk will be moved, not just the used space.

Upgrade and Purchase VSphere Infrastructure Licenses

This option is one we continue to re-evaluate every 6-8 months. Working in a non-profit, we are always evaluating how to be the best stewards of the resources we’ve been provided. One of those resources is obviously financial resources. This seems like the only real option for moving running VMs.

While having VSphere Enterprise in place would make this process of migrating to new storage easier, we just cannot justify the long term expense. Our small environment just does not need what you pay for over the long haul.

As such, we ruled this out. Some downtime is perfectly acceptable in this organization.

Use VI Client Datastore Browse/Move Functionality

When using the VI Client, you can browse the VMFS datastores similar to a Windows Explorer window. This is accessed by viewing the storage configuration of the host in the VI Client. Right-clicking the particular datastore and choosing browse.

Obviously, this function does not work on running VMs. As a matter of fact, in one of our tests, we actually caused a problem doing so. This is a “move” not a copy. If the move fails mid-way through, some files will have been moved while others are still in their original location. Now you’ve got a bigger mess on your hands.

So, this option means the VM has to be powered off. Ok, we can deal with that. The other downside was that, as explained above, the new vmdk is thick provisioned and therefore, much more data is transferring that is really necessary.

Test 3rd Party Product Like Veeam FastSCP

I really do like the folks over at Veeam. I have used the monitoring /reporting product and like there licensing models. We thought we’d check out their free FastSCP product. Why? Well because is says “Fast” in the name.

The tool worked great. It did everything is said it would do. Like the other options, the VMs need to be powered down. We even tested copying between different ESXi hosts. Worked great. The only downside was the lack of speed. Yeah really! We did some reading and it appears that this may be a throttling of the ESXi product, not Veeam FastSCP. Bummer.

Try standard TAR/GZ combined with cp command

“Wow”, I though as I was researching. Why don’t we just tar/gzip everything and then copy to the new storage location and then untar. Well, not so fast our ESXi said to m as it spewed out an error that “tar: cannot store file ‘xxxxxxxxxxx.vmdk’ of size #########, aborting”. Yuck~!

It appears that BusyBox, the underlying unsupported ESXi console has a limitation of 4GB in ESXi3.5 and 8GB in ESXi 4. Therefore, trying to tar a 100GB vmdk file went no where fast. Check this option off the list.

Try vmkfstools disk cloning

This, we thought, is an interesting idea. Yes, the VMs still need to be powered off but perhaps this will shorten the downtime by lowering the amount of data that gets copied. See, as we talk about earlier, going to storage of a different block size will mean yout thin provisioned disks will begin thick. As such, the total size of the thick disk has to be transferred. This could siginificantly increase the copy time if you have lots of unused space on a disk.

The vmkfstools utility has a feature known as cloning. This allows you to make a clone copy of a vmdk disk to anyVMFS datastore. Now that’s all good but where this got us jazzed was the ability to force that newly cloned disk to be created as a thin provisioned disk. Eureka!

Now, first, make sure you remove all snapshots in the vm. Those are just going to cause us a headache.

So a command like this does the deed:
vmkfstools -i /vmfs/volumes/source/vmname/vmname.vmdk -d thin /vmfs/volumes/dest/vmname/vmname.vmdk


Note: you need to do this against the vmname.vmdk NOT the vmname-flat.vmdk. Be forwarned.

Some small scale, non-scientific testing gave us around 1.3GB per minute in transfer speed. Again, this is not true i/o but rather therate used for calculating down time for our migration project by taking the amount of used space on the source drive/time to create cloned drive = transfer rate.

Now, when the vmdks are moved, don’t forget to cp all the other VM files in the VM directories!

This looks to be the path we will be taking and thought we’d share how we arrived here.

So you can see the MD3000i with the faceplate off directly below the new R710. I’m tempted to leave the faceplate off so that the executives and visitors see more lights when they look in. For some reason, non-technical people tend to have a first reaction to hardware such that more lights is mo’ better.

Along with these two pieces of hardware, we went ahead and got a Dell PowerConnect 5424 iSCSI optimized switch. This will be dedicated to the storage segment for the time being. Configuration of the switch meant actually going back to the rack. Initial configuration must be done via serial cable (included). Once you configure the management IP address and administrator credentials, the rest of the configuration can be done via the web based management tool.

Now the R710 was purchased with the nice iDRAC. This web based, out of band, management tool is wonderful. No client to load. You can configure, manage and review just about anything at the hardware level here.

One of the things I really liked about the new iDRAC is the console redirection or virtual KVM functionality. Of course, this allows you to run a java application the is the server console, however, it also allows you to map client resources as virtual media for the server.

In our case, we mapped the VMWare Vsphere ESXi 4 ISO image to the R710 server. We booted to it and installed the operating system without so much as a single walk back to the serer rack. This is much akin to the VMWare ability to mount client CD/DVD drives or ISO images as media for the guest operating systems. The difference here is that this is for the physical hardware. Lights out operations, indeed!

Finally, the MD3000i is configured with the Dell MD3000i Configuration Utility. Downloading this zip file was easy enough, extracted the contents and ran the SAConfig.bat file. The wizard walks you through the process. We had already wired the management ports on the MD3000i to our LAN while leaving the controllers on the storage network. The Wizard easily identified the unit, asked us for information on how we wanted to configure ip addresses and the like and then applied the configuration to the MD3000i. 5 minutes later we used the Modular Disk Storage Manager to begin the configuration process.

Dell will be helping with the final configuration next week but already setting up iSCSI to the ESXi hosts has been fairly painless following the documented processes outlined by Dell and VMWare.

Now, one of the biggest challenges is that there are multiple instances of the “image_attach” sections on a single content edit form, one for each category. The Drupal 5 version handled this by naming these by appending the category name to the fields. This allows the submitted form value for “iids” to be an array of categories. Each category itself being an array of image iid values.

For the life of me, my Drupal 6 version of this functionality failed to maintain the category arrays. After some frustrated hair pulling (hmmm…. no wonder my hair is so short), I remembered to take a look at the FAPI docs.

http://api.drupal.org/api/drupal/developer–topics–forms_api.html/6

An important thing to note: notice that $form['access'] has a '#tree' => TRUE attribute. this setting retains the full tree structure for all elements under it when it is passed to $form_state['values']. you must explicitly declare this anywhere you wish to retain an array’s full hierarchy when it is passed.

So, Drupal 6 collapses form elements where possible by default. By adding the #tree attribute, the hierarchy of form elements is maintained in the form_state submitted. Eureka!

Here’s an example of what I was getting without the #tree attribute:

And now with the #tree attribute set to true, the form hierarchy is retain and not collapsed:

I’m sure most of you doing Drupal development are saying “Duh!”. Well, let’s just say I really didn’t know. Thought I’d throw it on the blog so I don’t forget.

Blessings.

StackScripts™ provide a flexible way to customize our distribution templates. They’re very easy to use — find a StackScript, answer its questions, and click deploy. When the deployment is first booted, the script is executed and does its thing. You can even watch its progress by viewing the console.

As so many things with Linode.com and its community there is a public library of the Stackscripts available for a variety of purposes. Anything a from a standard LAMP setup to WordPress, Drupal and TRAC/SVN setups.

Already having a Linode up and running, I have also been thinking about re-doing the setup of ourLAMP server. Seeing this as an opportunity to both rebuild our LAMP server and test StackScripts, I set out to give it a try.

We already had a Linode configuration and a disk image actively running. The good news was that when you choose to deploy a new distribution via a stackscript, it will create both a new Linode configuration as well as a new disk image. Doing so keeps you from having to remove the currently running setup.

One of the standard Linode LAMP StackScripts provide a form like the image to the left when executed to deploy a new distribution. MySQL user and password, creation of a default database, as well as root O/S password and such. dec

Once the configuration is created, you now only need to boot into the new configuration. If you are logged into your LISH console, you will see the distribution go through the automated process of the StackScript. In our test case, an Ubuntu 8.04 LTS kernel was booted and installed, system updates installed and applied, postfix installed for local loopback only, MySQL installed and tuned, php installed and tuned and apache2 installed and tuned.

Pretty cool so far. Now, there is plenty left to do to get a fairly decent LAMP going that in our case would run Drupal.

Create a new non-root admin  user account called myadminuser and add to the admin group:

adduser myadminuser
usermod -G admin myadminuser

NOTE: Looks like the admin group is already part of sudoers so your new account has sudo privileges.

Update the timezone:

sudo dpkg-reconfigure tzdata

Secure MySQL services in a quick fashion:

sudo mysql_secure_installation

Enable a few Apache2 modules:

sudo a2enmod deflate
sudo a2enmod expires
sudo a2enmod cache
sudo /etc/init.d/apache2 force-reload

Add a few PHP tools:

sudo apt-get install php5-gd
sudo apt-get install php5-curl
sudo apt-get install php5-cli

Secure SSH daemon by changing the port for SSH and disallow root login:

sudo vim /etc/ssh/sshd_config
-- # Protocol 1
-- Protocol 2
-- ListenAddress 10.54.55.104 #your IP
-- Port 5502 # New port different than 22 and < 65535
-- PermitRootLogin no
-- X11Forwarding no

sudo /etc/init.d/ssh restart

Install the firewall and configure:

sudo apt-get install ufw
sudo ufw default deny
sudo ufw allow http/tcp
sudo ufw allow 5502/tcp
sudo ufw enable
sudo ufw status verbose

Configured PostFix by following pieces of our previous post on PostFix and domain forwarding.

sudo vim main.cf
-- See article referenced
sudo vim virtual
-- See article referenced
sudo postmap virtual
sudo ufw allow 25/tcp
sudo postconf -e "inet_interfaces = all"
sudo /etc/init.d/postfix restart

Lastly upon reboot we noticed that mysqld was failing. The problem was that mysqld was not binding correctly. We editing the my.cnf file and added BindAddress 127.0.0.1 above our existing BindAddress 10.54.55.104. This allows mysqld to bind and listen to both local loopback and the external IP (if needed)

At this point, we had not installed APC or MemCached yet. Performance was quite good with this setup and the minimal memory available to this test Linode. We ran six(6) Drupal , one (1) WordPress and one (1) CMS Made simple site with sub-second response times under fairly good load.

Kudos to Linode.com on their StackScript concept. I can certainly see this being very appealing for providers needing to crank out Linodes with a standard configuration for customers.

Blessings.

http://www.lullabot.com/blog/ie-iframe-insanity

To fix it you have to do this:

Admin menu completely hosed.

Note that we have no “Content” menu in administration. Also, notice that in the screenshot, we are in the Configuration menu but only have a single configuration item. That certainly doesn’t look right.

So after a bit of Googling  we first tried truncating the cache_menu table without success. Finally we decided to try a simple thought. We disabled the Admin menu. Then we re-enabled the Admin menu and Voila, our admin menus have been fixed. Go Figure.

Admin menu all fixed.

Just goes to show that sometimes the basic troubleshooting steps can fix something that looks worse than it really is.

Photo by Photomish Dan

Hire me, please.

Today’s SSWUG.org newsletter contained a great few paragraphs regarding what you look for in a new hire. As a hiring manager at various levels over my career, I have often asked myself, in self-reflection, such questions.

I believe that beyond the specific technical skill sets you are looking, there are some other specific characteristics you want to find. In particular I look for a candidate that shows the following characteristics:

Below is the content on this topic from the SSWUG.org newsletter today.

As a hiring manager, there have been times when I have had two very qualified DBA’s sitting in front of me.  Both of them are looking for a position on a team and both of them know their stuff.  How do you make a decision between the two?

As a hiring manager I have hired Jr. DBA positions as well. How do you make a decision on a role like a Jr. DBA that is really defined as someone who is starting to learn about being a database administrator and not a seasoned professional?

I have been in both of the above situations more than once. It is a difficult decision to make when you step back and realize that, if the DBA stays with the company for at least one year, you are trying to make a decision that will last 2,000 hours just based on a couple interviews and a resume.  The people that I end up with in the long run have given me every reason to hire them.  They have participated in a local user group and they are members of professional organizations. People that go out of their way to become better professionals are worth their weight in gold.

I cannot help but to remember that people can learn a number of technical skills but they have a harder time learning the desire to better themselves or to put in the extra hours to make sure something is just right.  Consider this as an example:

About 5 years ago I received an e-mail that was sent to me when I was the local user group president.  The person who sent me the email wanted to know how to get their first time DBA job.  I had lunch with her and another friend of mine and she talked about how she had paid to go to a conference and how she had been studying at the same time she was going to school.

She became a regular at the user group meetings and today she is starting what I believe to be her 5^th year as a DBA.  She really put forth the effort to make sure she was learning and doing everything she could to become a better DBA.

Unfortunately it seems that all too often, even these less technical criteria are so very difficult to find in candidates. In most of the SMB environments I have been a part of, you need depth and breadth of skills in a single individual. However, so often your IT folks break into the industry in a larger organization and get pigeon holed in a very specific skill, making it difficult to hire for us. Conversely, the more seasoned veterans with depth and breadth of skills can’t work for the salary many SMB’s and non-profits can pay. This dichotomy is were we continue to find ourselves today.

What do you look for? Tell us by posting a comment below.

Stephen Wynkoop

Demo: Capture perfmon counters to binary logs and at the same time capture SQL Profiler. Then you can load both into profiler and see both at the same time. Great for troubleshooting what might be causing a problem or what is happening at a particular point in time.

Historical only (not real-time)

When things go Wrong

• What changed?
• What were you doing?
• What were you seeing?
• Answer: NOTHING.

Add all the counters you need. Don’t be skimpy. Start broad and then narrow down. If you don’t record it, you can’t see it.

Intro to SSAS 2008

Marc Beacom

• www.MarcBeacom.com
• twitter.com/MarcBeacom

You can import an existing cube into BIDS which will essentially create a copy of the project that was used to create the cube. (Probably only to a certain extent)

PerformancePoint Services 101

Craig Utley

• Craig Utley, Mentor with Solid Quality Mentors
  • Consultant specializing in development with Microsoft technologies and data warehousing
  • Published author of books, whitepapers, articles, and courseware
  • Operator of LearnMicrosoftBI.com

Performance Point Services

Monitoring, Analytics and Planning

Planning being removed in the next version

• Monitoring: Scorecards and dashboards.
• Analytics: Drill up, drill down, slicing, dicing, etc…

Performance Point ships only with MOSS. With MOSS 2010, 64-bit only, no 32-bit.

KPI Sources are usually cubes but can be other sources including SQL Server, Excel and other sources.

No Pie Charts until next version.

Financial analytics with SQL Server Analysis Services

Donald Farmer

@donalddotfarmer

Need to listen to the Scottish Starbucks analogy again. It was a great analogy but want to be sure I didn’t miss something.

Financial Calculations

• Currency Conversions
• Time Intelligence
• Account Intelligence

Resources

http://www.microsoft.com/sqlserver/2008/en/us/analysis-services.aspx
http://sqlblog.com/blogs/mosha/default.aspx
http://www.sqlserveranalysisservices.com/default.htm
http://sqlcat.com/Default.aspx

Full SpectrumBusiness Intelligence

Donald Farmer

@donalddotfarmer

These slides are masterful is telling the verbal story Donald is presenting. I am putting up a few of the key slides that will server as good reference to me.

Thanks Donald!

Data quality is “fitness for purpose”. IT cannot tell you is a record is good.

TJ Belt

http://tjaybelt.blogspot.com
http://www.twitter.com/tjaybelt

You are on-call, maybe middle of the night and you get a call and there’s a problem. What do you do?

• What tasks were last performed by what persons?
• Start documenting incidents if you don’t already. Great for helping resolve things later down the road.
• Document, document, document

Items to be sure of and continuously check

• Scan OS error events
• Check on backups
• System specific output files (xml, reports, files)
• Log and data file sizes
• Space available / Free space on drives
• Replication health
• SQL Server logs
• Other notes of interest

Work to get your logs clean over time so that problems become more apparent and you will spend less time chasing red herrings.

“You only need one. Either a good backup or a good resume.”

Only notify on error, not on success. (Of course there may be exceptions)

• What are Microsoft SQL Server operators and how to define them (Part 1)
• Undocumented SQL Server 2005 Database Maintenance Plan Stored Procedures
• DB2 and the Procedural DBA, Part 1

Dimension Table ETL with SSIS

Erik Veerman
Mentor, Solid Quality Mentors
SQL Server MVP

Dimension tables contain the “text descriptors” or attributes. Also organized as hierarchies of categories, levels and members.

Slowly Changing Dimension (Ralph Kimball)

• Type 0:No change (Fixed Attribute)
• Type 1:The value is overwritten (Changing Attribute)
• Type 2:Slow changes. A new row is added (Historical Attribute)

Type 2 – New surrogate key and Start Time/End Time updates so to add the new record.

Inferred Members

• Missing dimension member during Fact load later becomes available in the Dimension source
• Also known as late arriving dimensions
• Fact load, placeholder record added to dimension table with new surrogate key generated
• Dimension process, for inferred members, all dimension attribute columns are updated with new values (type 2 attributes are handled as type 1)

Additional SSWUG Articles

• Troubleshooting SQL Server 2005 Integration Services
• MSDN Webcast Summary: Introducing SQL Server Integration Services for SQL Server 2005
• Tips for using SQL Server 2005 Integration Services (Part 3)

Fact Table ETL with SSIS

Erik Veerman
Mentor, Solid Quality Mentors
SQL Server MVP

Identify Dimension Surrogate Keys

• Locate dimension candidate keys in transaction source
• Validate uniqueness and data integrity
• Considering historical changes
  • Identifying current dimension record, combine business keys with current identifying meta data (data identifier or boolean)
  • Handle missing dimension records
• SSIS Data Flow Options
  • Lookup transformation
  • Merge Join transformation
  • Fuzzy Lookup transformation
• SSIS Control Flow Option
  • Execute SQL Task

• Fact ETL involves surrogate keys, grain changes, and measure calculations
• SSIS handles these through several out-of-the-box transformations

Loading a Data Warehouse with SSIS

Brian Knight
bknight@pragmaticworks.com
CEO, Pragmatic Works

Why Data Warehouse?

• Get users off of transactional systems
• Keeping more data online
• Easier reporting
• Consolidate data sources

Data Separated into fact and dimension tables

• Dimension tables answer the pivot or where clause
  • Make as wide and descriptive as possible
  • Surrogate keys operate as unique ID for each row
  • Keep surrogate keys as small as possible
• Fact tables answer the what or select statement
  • Intersect all dimension tables
  • Surrogate keys from each dimension in this table
  • Measures are the “what” like Price, Quantity, Duration

Additional Articles

• MSDN Webcast Summary: Introducing SQL Server Integration Services for SQL Server 2005
• SQL Server 2008 Katmai Change Data Capture (CDC)
• Troubleshooting SQL Server 2005 Integration Services

• http://Blogs.pragmaticworks.com/brian_knight
• http://www.twitter.com/brianknight

Live – SQL Server Analysis Services – Real World Performance Tuning (4hrs)

Ted Malone

(I only caught a few minutes here and there of this marathon live session, but the parts I caught were good. We’ll have to watch the on demand once it goes up.)

• Recommends looking at a product called Performance Advisor from SQL Century for getting performance insight into SSAS.

DBA Checklist

Buck Woody, Microsoft SQL Server Specialist

http://blogs.msdn.com/buckwoody
http://www.informit.com/guides/guide.aspx?g=sqlserver
http://edge.technet.com/People/BuckWoody/

SSMS

• Object Explorer: Right click to see additional columns including Last Backup.
• Standard Reports available via right click
• Log Viewer

oSQL or SQLCMD

PowerShell – Latest and greatest

Post Installation

Daily, Weekly, Monthly and Yearly

Recommends being alerted of both error AND success in order to know that maintenance task really did run.

Maintenance tasks

• Do not run the auto-shrink option in your tasks.

Take a baseline with PerfMon. Email him for a list of counters.

Keep a SQL Server run book containing all configuration changes.