I recently had to do an UPSERT. (Yes, it’s a real word: UPdate + inSERT.) This would normally be an UPDATE followed by in INSERT wrapped in a transaction so if either fails it all rolls back. This means you will make at least two passes through the data, locking it until the transaction commits. Logically this is a single operation, wouldn’t it be nice to have a single construct that could handle the UPDATE, the INSERT, and maybe even a DELETE if necessary? That’s exactly what we have in the MERGE statement. SQL Books Online gives a good explanation of the syntax and a few examples but it can get very complex very quickly.

USE tempdb
    IF (OBJECT_ID('dbo.#Target')) IS NOT NULL 
        DROP TABLE #Target; 
    IF (OBJECT_ID('dbo.#Source')) IS NOT NULL 
        DROP TABLE #Source;        

    --Dupsert this table...
        CREATE TABLE #Target 
                ,DataCol    CHAR(10)    NOT NULL
        INSERT INTO #Target (DataCol) 
            VALUES ('Static'),('Update Me')
            ,('Delete Me')
    --...with changes from this one.
        CREATE TABLE #Source 
                SourceID    INTEGER    IDENTITY(101,1) PRIMARY KEY NOT NULL
                ,TargetID    INTEGER    NULL 
                ,DataCol    CHAR(10)    NOT NULL
        INSERT INTO #Source (DataCol,TargetID) 
            VALUES ('Static',1),('Change',2),('New',NULL);
    --View tables    
        SELECT * FROM #Target;
        SELECT * FROM #Source;

    --Dupsert the target using MERGE
            INTO #Target 
            USING #Source
                ON    #Target.TargetID = #Source.TargetID
                SET #Target.DataCol = #Source.DataCol
                INSERT (DataCol)
                VALUES (#Source.DataCol)

    --View target    
        SELECT * FROM #Target;

As you can see, the code is a logical unit, straightforward, and flexible.  Enjoy!

Performance Tuning – An Example

Recently we had some trouble with a stored procedure taking between 1.0 and 1.5 seconds with occasional runs of 8 or 9 seconds. This procedure is widely used by applications contacting a widely used database and is called as often as once or twice a second. It returns six recordsets each with a total row count of 50 or less.

This stored procedure is moderately complex and structured as follows: temporary data structures are created and populated by successive calls to the database and then the data is modified slightly and returned. The procedure accepts up to 16 input parameters with one or two “signatures” predominant. For example, looking up records by Customer_ID was the most common use for the procedure but it was also possible to do lookups based on a set of hashed TINS, first and last name, etc. If null values were passed in for the parameters the procedure still proceeded in a top-down fashion without any logic to skip queries that were unnecessary. Each DB call involved joining on a dozen code tables using the NOLOCK query hint.

The Query Plan

We began by examining the query plan. This is a representation of the physical steps SQL Server uses to provide the data you request in a query. Often a DBA can use this plan to discover where SQL Server is spending a lot of time and take action to correct it. This plan can be expensive to create and so once calculated it is stored in SQL Server’s plan cache until manually cleared or SQL Server is restarted. As long as the plan is calculated correctly this is a great efficiency however the input parameters can have a significant effect on the logic of the query and consequently the cost. Suppose the plan was first calculated based on a list of hashed TINS and stored in the plan cache. If the next call is based on Customer_ID the first query plan will be used even though it may not be optimal for the new input parameters. This is known as a Parameter Sniffing problem and is discussed in more detail here.

We believed this was part of the reason for the poor performance. There are at least two ways to deal with a situation like this:

•    Restructure the procedure into modules (sub – procedures) so each module has its own execution plan. This introduces some complexity in maintenance but guarantees that each logical path has an efficient query plan.

•    Recalculate the plan each time you call the procedure with the current parameters. The drawback of course is you have to pay to recalculate the plan each time.

In this case the plan is not overly complex and it seemed worthwhile to recalculate the plan each run by altering the procedure to use the WITH RECOMPILE option. This brought the average run time down to 550 ms with no executions taking longer than two seconds.

NOLOCK Query Hints

SQL Server locks data during inserts and updates. These locks prevent other users from querying that same data until the transaction is complete. This can cause a query to take longer to run. Telling the query optimizer to use the NOLOCK query hint allows the database engine to read the data even if the lock would normally prevent it. One danger in using this hint is it allows a query to return potentially incorrect information. Another more subtle and serious result comes from meddling with the query optimizer. The Optimizer is a very sophisticated piece of code and is far better at making choices for optimizing data retrieval than any single DBA can ever be. Using query hints should always be a last resort and used only after detailed testing. See here for additional details.

This procedure used the NOLOCK hint 57 times even on code tables which would almost never be modified. We removed these NOLOCK hints and saw a modest performance increase of 40 ms bringing our average run time to 515 ms.

Temporary Data Structures

The next area of interest is the temporary data structures. The original developer chose to use table variables rather than temporary tables. One significant difference between these structures is that the table variable is thought to reside only in memory while the temporary table is written to tempdb, SQL Server’s internal working database. I say “thought to reside only in memory” because while that is generally true SQL Server will write table variables to tempdb if it sees fit – as it did in this case. There are cases where tempdb contention is a problem and choosing a table variable over a temporary table makes sense. This is usually the case when the amount of data you are storing is very small. Based on our experience in similar situations we decided to re-factor the procedure to use temporary tables rather than table variables. Using this approach brought our average run time down to about 90 ms.

Other Opportunities

So far the steps are taken are all very low risk and no changes to the logical structure of the query have been made. We believe additional improvements could be realized through altering the flow of the procedure to bring back less data on the intermediate database calls and to eliminate unnecessary calls based on the parameters submitted. However, for now, since performance went from 8 to 9 seconds down to a 10th of a second we decided to leave well enough alone and revisit this later if necessary.

As you can see, SQL Server tuning is as much an art as a science. It involves observation, experimentation, and testing. Sometimes there’s a single silver bullet, other times you take a more iterative approach as we saw in this example.

Run SQLCMD Scripts Using PowerShell

Many of our production DBs are restored nightly onto a development server which is a great way to test your backups and to have production data available to developers. In order to do this “Prod on Dev” though you have to recreate any DB objects you made yesterday because they got stomped in the refresh. You could make one massive script for all the objects and run it first thing every morning but if you are creating many objects maintenance can become complex. A semi-automated solution is to put each object in its own file and use SQLCMD to run each file either manually or using the Windows Task Scheduler. Better but any time you add an object you have to edit the control script. How about putting each object in a file in the same folder and using PowerShell to get the file names and run each one using SQLCMD? The PowerShell script below does just that.  Schedule it to run from Windows Task Scheduler and you are ready to refresh as needed.

# Get a list of sql files in a folder and run
# them in alphabetical order using SQLCMD.

$Folder = “C:\Temp\Source\”
$SQLInstance = “MyServer\MyDB”

#Get file names to run
$FileNames = Get-ChildItem -Path $Folder -Filter *.sql -Name | Sort-Object

#Run SQLCMD for each file
ForEach ($FileName in $FileNames)

 $File = $Folder + $FileName
sqlcmd -S $SQLInstance -i $File


As you can see, the code is quite simple but let’s take a closer look and consider some potential enhancements.

In #Setup we are clearing the screen and setting variables for future use. These values could easily be input parameters or the whole thing could be turned into a function.

In the #Get file names to run section you retrieve a list of files in the source folder and sort them alphabetically. If they need to be run in a certain order (such as tables before procedures) you can add an index to the file name (010_MyTable.sql). If the files are under source control, say TFS, you could use PowerShell to run TF.exe to get the latest version of the files. Running from the command line in PowerShell is very straightforward as you will see in the next section.

Finally #Run SQLCMD for each file is the code that loops through each file and runs it using the SQLCMD utility.  You could easily add logging and notifications if you have the need.

As you can see, you can automate tedious tasks quickly and easily using PowerShell and SQLCMD.

SQL Server & PowerShell – Finding and Changing Job Owners

We’ve been hearing for a long time that PowerShell (PS) is the wave of the future for SQL Server administration. Under the covers SSMS itself is powered by PS so there ought to be benefits but how do you get started. Everybody has seen how you can right-click a node in the SSMS Object Explorer and jump right in to PS.

But then what?

Let’s consider a small administrative task. Fred creates a bunch of jobs without setting the job owner so he owns them all. He decides to seek greener pastures and when his account is dropped the jobs quit working.

I know you all have (or should have) a script for finding and changing the owner of certain SQL Agent jobs on a SQL instance. They all use a cursor (or something like it) to query msdb.dbo.sysjobs and step through the results, executing msdb.dbo.sp_Update_Job where appropriate. This certainly works but it’s kind of a kludge. Wouldn’t it be more intuitive to say something like “Look at all the jobs, where ‘Fred’ is the owner then change the owner to ‘sa’.”?

That’s really all you do in PS and you can find examples from Mr. Google. However, PowerShell’s ability to use shortcuts and abbreviations often makes figuring out what’s really happening like making sense of alphabet soup.

Here are two snippets of code. The first lists all the jobs on an instance with their owners and the second finds all the jobs with a certain owner and changes that owner to another. Let’s unpack each and see what’s going on.

dir | FT Name, OwnerLoginName –auto

dir” is an alias for “Get-ChildItem” (Also abbreviated “GCI”). It functions like the “dir” command at the DOS prompt – it looks at the current location and returns everything “below”. The important difference is that it doesn’t return a string about the objects, it returns the objects themselves. Keep that in mind as we go on.

|” The pipe character represents the PS pipeline which hands the object(s) to the next command. In our case we are passing all the objects returned by Get-ChildItem to the next command. See this link for more information on the pipeline.

FT” is short for “Format-Table” and is used to format output. The comma separated arguments “Name” and “OwnerLoginName” tell FT which columns to return and the “-auto” automatically prettifies the output for human consumption.

Follow the above screenshots and paste this line of code into the PowerShell window and hit “Enter” to see it for yourself.

Now that we can see who owns each job lets change some owners.

dir | ?{$_.OwnerLoginName –eq “Fred”} | %{$_.OwnerLoginName = “sa”; $_.Alter(); $_.Name}

dir|” functions as above but remember the actual objects are being passed, not just a property value (name, owner, etc.). In this case we don’t need all the jobs, just the ones Fred owns.

?” is an alias for the “Where-Objects” commandlet which filters for the objects you want. The curly brackets (“{}“) contain the script block where you put the filter logic. “$_.” refers to each object coming through the pipe and “OwnerLoginName” is the attribute examined as it passes through the filter. “-eq” is the equality operator (=) while ‘“Fred”‘ is the string it is trying to match.

After this the objects that met the filter criteria are piped to the next command. Again we have an alias, “%“. This is short for “ForEach-Object” and allows you to iterate the objects and change properties or execute methods. In this case we will set (“=“) the “OwnerLoginName” value to the string “sa”. The “Alter()” is a method call that updates any changes. Finally the “Name” prints out the name of each job changed.

As you can see, PowerShell uses an intuitive and object oriented approach to working with SQL Server objects though sometimes shortcuts and aliases can make it a little hard for beginners to interpret. It’s well worth the time to take a careful look at any scripts you download so you know exactly what’s going on. PowerShell has a very nice help system that allows you to do this research. Look here if you’d like some help on Help.

Comparing Multiple Files Using PowerShell

I am moving a number of SSIS configuration files from one folder structure to another as part of a SQL Server upgrade. Both folder structures have each of the files but they point to different file drop locations, SQL Instances, etc. My task was to compare the configuration settings in each environment and map all the changes from the old environment to the new.

Rather than opening each file and visually checking for changes I went with a PowerShell approach centered on the Compare-Object cmdlet. This cmdlet compares two objects and produces a comparison report. This will work whether the two objects are folder structures, text files, or other objects.

My approach was to start with two folders containing .dtsconfig files. First compare the two sets of files and return a list of files that are in both folders. Then, file pair by file pair, open each file and compare contents. Write any differences to an output file. See the comments below for more details.

# Setup
$Path1 = “C:\Path1”
$Path2 = “C:\Path2”
$OutFile = “C:\ComparisonOutputFile.txt”

 # Delete outfile if it exists
If (Test-Path($OutFile)) {Remove-Item $OutFile}

# Write the two paths to the outfile so you know what you’re looking at
“Differences in files of the same name between the following folders:” | Out-File $OutFile -append $Path1 + ” AND ” + $Path2 | Out-File $OutFile
“——————————————–“ | Out-File $OutFile

 # Compare two folders and return only files that are in each
$Dir1 = Get-ChildItem -Path $Path1 -filter *.dtsconfig
$Dir2 = Get-ChildItem -Path $Path2 -filter*.dtsconfig
$FileList = Compare-Object $Dir1 $Dir2 -IncludeEqual -ExcludeDifferent

# Loop the file list and compare file contents
ForEach ($File in $FileList)
{$F1 = $Path1 + “\” + $File.InputObject
$F2 = $Path2 + “\” + $File.InputObject
$ | Out-File $OutFile -append
Compare-Object $(Get-Content $F1) $(Get-Content $F2) | Format-Table
-auto | Out-File $OutFile -append

 As you can see, PowerShell provides significant functionality with very little effort. I was particularly impressed with the comparison report. The SideIndicator column (see below) is a simple way to communicate the differences.   This is a typical output file (the names were changed to protect the innocent).

While it isn’t a fancy GUI it is functional, visual, extensible, and built in. The extensible is important because you don’t get that from a GUI. Suppose I wanted to keep a “Gold Copy” of my config files and run a daily comparison against production to pick up changes? The code above is almost custom written for that process – all it needs is an email step. Further, you could use a similar process using SMO to generate database schema and do you own comparisons. The possibilities are truly endless.

Why I Am, And Want To Remain, A Consultant.

I have been in the IT world for over 15 years, as both a full-time employee (FTE) and as a consultant. I have worked for a major IT consulting firm, local contracting companies, and fly-by-night operators. Recently I had the opportunity to go independent so my wife and I formed an S-Corp, hired an accountant, and dove in.

As I discuss consulting opportunities with head hunters and potential clients I continue to hear the siren song of coming back into the fold and being an FTE – either for a client or as a W-2 employee of someone else’s consulting company. As you might imagine this has created a great deal of soul searching and a desire to settle the issue once and for all. I began to ponder the question “Why am I a consultant?” Answers to follow.

“Consultants make more money” – In general I believe this is true but there are two things to consider. First, consultants take a larger risk so it’s only reasonable they should reap a greater reward. Second, companies pay far more for FTEs than simply their salaries – Social Security taxes, healthcare insurance, other insurance, etc. All these expenses are borne by the consultant out of their higher hourly rate. I’m not saying consultants don’t make more, only that the difference is not as great as one might think. Further, if time is money, then the amount of time my sainted wife has spent dealing with the government’s ridiculous administrivia needs to be taken into account. I may make a little more but it comes at a cost – there are easier ways to make a living.

“Consultants are their own boss” – What exactly do you mean “boss”? Do you mean I set my own hours? No more than anyone else in the organization I am working in. If everyone else has flex-time I probably have it as well. If the rules are 8-5 with an hour for lunch, guess what my hours are? If the pressure is on for a major release the boss is every bit as likely to bite my head off as the FTE in the next cubicle – maybe more so. I do have the choice to accept a contract or not but that is no more than the right to starve. Besides, every FTE has the same freedom to say “no” – they just have to find a new gig but they can always say “no”.

“Consultants don’t have to deal with politics” – Some do, some don’t. In my own case this has usually been true but I’m sure there are plenty of horror stories on that subject.

For me, the one major reason I am and want to remain a consultant is…

“Being a consultant is fun” – The constant pressure of making and keeping myself marketable keeps me studying and exploring in my field. I know from experience that as an FTE I get fat, dumb, and lazy and I fight a tendency to coast and complain. I am not saying this is the case for all FTEs, I speak only for myself. Some FTEs are fat, dumb, lazy, and dishonest – just like some consultants. Those are character issues that are independent of whether or not you get a W-2 or a 1099. Because I am concerned with staying marketable I learn new things – which I enjoy. When I enjoy what I’m doing I look forward to the challenges and to new opportunities to learn. It’s a synergistic cycle that makes me happier at work and happier at home. I want to be the best I can be at what I do – to take care of my family and to find pleasure in my work. For me, for now, a consultant is what I am and what I want to remain.

Using A SQL Server Database Snapshot As A Development Tool

I am modifying a number of SSIS packages that extract, transform, and load data into various tables. While unit testing I need a quick way to refresh my development environment. The traditional method is to take a backup and restore but if the database is large it can take a significant amount of time – even a ten minute restore drags on forever if you have to run it a dozen times a day. Another option is to duplicate the data in separate tables then kill and fill to refresh the data. This can run quickly (depending of course on table sizes, indexes, etc.) but it can be complex to set up. My favorite approach is to create a database snapshot as my backup, run the load, evaluate the results, then restore from the snapshot – quick and easy to set up. (If you are not familiar with database snapshots here is a good overview.)


Due to their sparse file architecture DB Snapshots are created instantaneously, all that gets created is an empty file. As changes are made to the original the altered 8KB pages are written to the new file with no logging – in fact the snapshot doesn’t even have a transaction log. When you want to restore from the snapshot the copied pages are just copied back to their original location. If your changes are relatively small in terms of 8KB pages compared to the total DB you will find this lightning fast.


A DB Snapshot is as easy to create as a regular DB. The only complexity is if your DB has multiple files so must your snapshot. Never fear though, I have a code generator that will handle all the gory details, see below. Please be sure you read the comments at the top so you know how to run it. There is also code at the end of the script to restore to the original state as well as to drop the snapshot.



  • Snapshots are only available in Enterprise Edition.
  • The Snapshot is Read-Only which shouldn’t be a problem when used only for restores.
  • Creating a Snapshot requires CREATE DATABASE, CREATE ANY DATABASE, or ALTER ANY DATABASE permission.

Size Issues

Normally DB Snapshots don’t take up much space when used this way but be sure to clean up after yourself! Since the snapshot keeps collecting changes if you forget to drop the Snapshot it will continue to grow to the size of the original if enough changes are made.

Exclusive Use

You are not required to have exclusive use except for the restore but if your buddies are using the original DB and you do they restore the may not be your buddies anymore. Remember, with great power comes great responsibility.



I apologize for the code formatting issues, WordPress and I are having issues. 😦 

This script will write a script to create a snapshot of the source DB, it will
handle multiple files in the DB.  It also supports multiple snapshots of the
same source by changing @SSName.
      Set @SourceDBName to the name of the DB you want a snapshot of.
      Set @SSName to what you want to call your snapshot.
Ctrl+Shift+M will allow you to enter the DB and Snapshot names.
You can enable the Execute statement at the end if you want to build the
snapshot now.
      @SourceDBName As sysname
      ,@SSName As sysname
      ,@SQL as Varchar(4000)
      @SourceDBName = ‘<SourceDBName,sysname,Essence>’
      ,@SSName = ‘<SSName,sysname,EssenceSS>’
      @SQL =
      ‘CREATE DATABASE ‘ + @SSName + ‘ ON ‘
            + LEFT(Script,LEN(Script)-1)
            + ‘AS SNAPSHOT OF ‘ + @SourceDBName
                  SELECT col + ‘, ‘ as [text()]
                              Select Top 100
                                          When Right(mf.physical_name,4) = ‘.ndf’
                                                Then ‘(NAME = ‘ + + ‘ ,FILENAME = ”’
                                                      + Replace(mf.physical_name,’.ndf’,’_’ + @SSName + ‘.ss’) + ”’)’
                                          Else ‘(NAME = ‘ + + ‘ ,FILENAME = ”’
                                                + Replace(mf.physical_name,’.mdf’,’_’ + @SSName + ‘.ss’) + ”’)’
                                    End               As col
                                    sys.master_files  As mf
                                    Inner Join
                                    sys.databases     As db
                                          on mf.database_id = db.database_id
                           = @SourceDBName
                                    mf.type_desc = ‘ROWS’
                              Order By
                        ) As A
                  ORDER BY col ASC
                  FOR XML PATH(”)
            ) AS Script
      ) B
–Execute (@SQL)
Print @SQL
–Other useful code
      /*Restore from snapshot**********************************
            Use Master
            Alter Database <SourceDBName,sysname,Essence>
            Restore Database <SourceDBName,sysname,Essence>
            From Database_Snapshot = ‘<SSName,sysname,EssenceSS>’
      /*Drop Snapshot*****************************************
            Drop Database <SSName,sysname,EssenceSS>

Inserting Binary Objects Into A SQL Server Table Using A Stored Procedure And PowerShell

Our team was tasked with putting scanned expense report receipts (.PDF) in a SQL Server table for review.  As the DBA I thought my part was pretty straight forward.  I knocked out the table and associated objects including the stored procedures to service the table and was enjoying a cup of coffee when I realized I had not tested my insert procedure.  Then it dawned on me – I had no way to test it in SSMS.  How can you use a text based tool to manipulate binary files into the database?  If you’ve figured that one out I’d love to see it.  If you haven’t then perhaps my PowerShell approach will be of some use to you.

First let’s create some basic SQL objects, a table and a stored procedure:




      INSERT INTO dbo.TAPTEST (Col1) VALUES (@Col1)


Here is the PowerShell code if you are in a hurry but it’s worth going through line by line as we will do below.

#Get the file

      [Byte[]]$file = get-content -Encoding Byte c:\Temp\TAPTEST.pdf 

#Connect to DB   

      $DB = “server=MyInstanceName;integrated security=sspi;database=Bob”

      $conn = new-object System.Data.SqlClient.SqlConnection($DB)

#Build the command and parameters

      $cmd = new-object System.Data.SqlClient.SqlCommand(usp_TAPTEST”,$conn)

      $cmd.CommandType = [System.Data.CommandType]StoredProcedure

      $cmd.Parameters.Add(“@Col1”, [System.Data.SqlDbType]VarBinary)

      $cmd.Parameters[“@Col1”].Size = 1

      $cmd.Parameters[“@Col1”].Value = $file

#Execute the command



 Taking a closer look.

#Get the file

      [Byte[]]$file = get-content -Encoding Byte c:\Temp\TAPTEST.pdf

 The square brackets force the data type for the $file variable.  Additionally, since Get-Content normally reads the file as text the –Encoding parameter is used to tell it to read it as binary.

Connecting to the database is just as you would do for any other SMO connection, nothing special here.

#Connect to DB   

      $DB = “server=MyInstanceName;integrated security=sspi;database=Bob”

      $conn = new-object System.Data.SqlClient.SqlConnection($DB)

Building the command and parameter objects was a little more interesting.


#Build the command and parameters

      $cmd = new-object System.Data.SqlClient.SqlCommand(usp_TAPTEST”,$conn)


Note the command text above:  usp_TAPTEST which will be executed at the server.  Any valid DML should execute, including the following. usp_TAPTEST @Col1 = 1″ which works great as long as you have a printable data type.  If it weren’t for reading the file into a Byte data type in the first line we’d be stuck again.   So how DO you tell it this is a binary object?  Read on.

In the first line below we tell PowerShell to treat the CommandText as a stored procedure.   Then we set the first parameter’s data type to varbinary.  Setting the size parameter to -1 tells it that the size is MAX  Finally, set the value of the parameter to the file you read in initially.


      $cmd.CommandType = [System.Data.CommandType]StoredProcedure

      $cmd.Parameters.Add(“@Col1”, [System.Data.SqlDbType]VarBinary)

      $cmd.Parameters[“@Col1”].Size = 1

      $cmd.Parameters[“@Col1”].Value = $file

 At this point you are ready to go, just open the connection and execute.

#Execute the command




And voila!

Notice the French lingo – this is a high class blog!






As you can see, the code is pretty simple with the only tricky part having to do with fitting the binary nature of the document with the parameter used by the SQL Command object.

Hopefully you will find this useful both as stand-alone code and as another step in the drive for PowerShell mastery.


T-SQL Tuesday #016 – Getting Totals And Subtotals With ROLLUP And CUBE


olap_1Welcome to TSQL Tuesday!  Click the picture on the right for more information.

TSQL is a great tool for aggregating lots of data but what do you do with it once it’s aggregated?  Often the answer is to aggregate it again through subtotals and totals.  Normally this will require a second query or if you want to see the totals “in-line”, a trip to Excel.  But, when using aggregation, you can use the WITH ROLLUP or WITH CUBE operator in the GROUP BY clause to generate totals.

MSDN explains the difference between ROLLUP and CUBE as follows:

  • CUBE generates a result set that shows aggregates for all combinations of values in the selected columns. 
  •  ROLLUP generates a result set that shows aggregates for a hierarchy of values in the selected columns.

 If that seems a little fuzzy don’t worry about it – just run the query below with each operator and note the difference in output. 

This query will run on SQL 2005 or 2008 DBs and shows the number of reads since the last restart.  Normally these operators (ROLLUP & CUBE) are used to present an application with pre-aggregated data for performance reasons.  However in this example it is quite useful for ad hoc I/O performance queries. 


      OBJECT_SCHEMA_NAME(S.object_id)     AS SchemaName

      ,OBJECT_NAME(S.object_id)           AS ObjectName

      ,SUM(S.user_seeks + S.user_scans + S.user_lookups)                           AS TotalReads


      sys.dm_db_index_usage_stats         AS S


      sys.indexes                         AS I

            ON S.object_id = I.object_id


      OBJECTPROPERTY(S.object_id,‘IsUserTable’) = 1

      AND I.index_id = S.index_id

      AND S.database_id = DB_ID()







  In Figure1 we see the results from the basic aggregation.





Figure 1

Adding WITH ROLLUP produces the following.  Row 1 (NULL NULL) is a grand total.  Note that row 2 gives the sub-total for the Person group and so on.






Figure 2

Finally WITH CUBE produces all possible subtotals. 









Figure 3

 These operators aren’t Rocket Science but if they save you acouple minutes every time you’d have to make a trip to Excel then I think they are worth adding to the toolbox. 

Using PowerShell and SMO to Script SQL Server Database Objects

I’ve been telling myself to learn PowerShell for quite awhile but I’ve never been able to get over the learning curve.  What I needed was a piece of work that was bigger than trivial and smaller than huge that would force me into PowerShell.  It came along this week in the form of a request to script all the views in a test database before the refresh from production and then “putting them back” into the new test database.  Doing this from SSMS is pretty easy but I sure don’t want to do it every time we refresh.  I needed something repeatable, simple, and parameter driven. 

 I knew PS would be relatively simple to manage so I looked into using PS with SQL Server Management Objects (SMO).  First I searched the Web for something like “PowerShell SMO Script Objects” and found this, which pointed me in the right direction. 

The complete script is at the bottom of the article but I’ve broken it up into sections which I’ll cover in detail.  My focus is those of us learning PS so if you are a guru please be patient with the rest of us.


Figure 1

All the user configurable parameters can be set in Figure 1 above.  In line 13 note that $ObjType will be inserted as a literal in the code so PowerShell will read MyObject.$ObjType but execute MyObject.views.  This allows you to script tables, views, stored procedures, and other Db objects by simply changing the parameters.  For even more flexibility you could read in parameters from a text file.

Also note in line 15 that the “@” indicates a list of string objects that will be excluded from the script.  You could also add other schemas if you wish.

Finally, the If block (lines 18-21) tests for the output file and deletes it if it exists.  Without this check successive runs of the script would append the same scripts to the file.


Figure 2

Next, let’s look at the code to create the SMO we’ll use to script our objects.  Line 25 tells PowerShell which assembly we want to use so it can create the objects we need.  Lines 28 and 29 create the Server and Databases collection objects respectively.  Note that the Server object takes the instance name on creation.

Once you have the DB collection available you are almost ready to start scripting.  The Script method is available for each object (table, view, etc.) but there are a number of options you can use to control the scripts generated.  For example, if you want to test for existence before creating an object you would set the IncludeIfNotExists = 1.  To use the option you will create a ScriptingOptions object (line 32) and set various properties as needed.  A complete list of options is here.

If you want to perform multiple scripting tasks, say Drop and then Create you can perform the scripting multiple times, appending the results as we see in the following two code blocks.


Figure 3

First you set the ScriptingOptions as in Figure 3 for the operation you want to perform (lines 38 & 50).  Then you iterate through every object of the specified type in the database using a foreach loop (lines 41 & 53).  Each time the foreach starts it filters out the object if the schema is any one of the values passed in from the list discussed above.

Without the whitespace, the whole PowerShell script takes up only about 20 lines.  As you can see, PowerShell and SMO together make a very powerful tool to put in your toolbox.    I’d be interested to hear about how you modified this code to meet your own needs.

Here is the whole script, ready to use with your parameters.

# PS Script DB Objects
# Uses PS and SMO to script the drop and create a type of
# SQL Server object (views, tables, etc.)
# Tom Powell                                       2/24/2011
    $SQLInstance = “Server\Instance”
    $DB = “DBName ”
    $ObjType = “views”
    $OutFile = “C:\temp\Scripts.sql”
    $ExcludeSchemas = @(“sys”,”Information_Schema”) #List of schemas to exclude
#Create smo objects##########################################
    [System.Reflection.Assembly]::LoadWithPartialName(‘Microsoft.SqlServer.SMO’) | out-null
    #SQL Instance & DB object
    $s = new-object (‘Microsoft.SqlServer.Management.Smo.Server’) $SQLInstance
    #ScriptingOption object to pass parms for Script method
    $so = new-object (‘Microsoft.SqlServer.Management.Smo.ScriptingOptions’)
    $so.IncludeIfNotExists = 1
    $so.SchemaQualify = 1
    $so.AllowSystemObjects = 0

#Script Drop Objects###########################################
    $so.ScriptDrops = 1         #Script Drop Objects

    #Generate script for all objects##################################
    foreach ($objs in $dbs[$DB].$ObjType)
        If ($ExcludeSchemas -notcontains $objs.Schema )  
            $objs.Script($so) + “`r GO `r ” | out-File $OutFile #-Append

#Script Create Objects#########################################
    $so.ScriptDrops = 0         #Script Create Objects

    #Generate script for all objects#################################
    foreach ($objs in $dbs[$DB].$ObjType)
        If ($ExcludeSchemas -notcontains $objs.Schema )
            $objs.Script($so) + “`r GO `r ” | out-File $OutFile -Append