Caffeinated Coder

One of my goals for the next six months was to contribute to an open source project and I think I’ve chosen one. I downloaded and installed the PowerShell Community Extensions (a.k.a. pscx) from codeplex a while ago, but I just now got around to really tinkering with the cmdLets and reviewing the source code and scripts. I have to say I’m impressed. Keith Hill and his team of volunteer developers did a first rate job of supplementing version one of Powershell with some really useful cmdlets. I can’t wait to dig into the source code some more and figure out how the snap-in model works.

Here are a few of my favorite features:

Provider:

• GAC Provider - It’s a little slow when it first loads up, but it is very helpful to be able to easily navigate around in the GAC and do wildcard searches on version numbers.

CmdLets

• Get/Stop-TerminalSession - It seems like every couple of weeks I have to kill a remote terminal services connection on one of our development servers because someone forgot to log off and pushed us over the two simultaneous users limit. This cmdlet is a lot quicker than loading up the administrative GUI and will help make the experience much less aggravating.
• Write-Zip - This will be very useful addition to backup/archiving scripts. At home, I just did the following: dir “G:\Music\Radiohead\Pablo Honey” | write-zip
• Get-FileVersionInfo - We just started auto-versioning all of our dll’s during our build process and set it up so we can match the version of a dll in production with a build directory on our build server. I’m sure we’ll be asked to create some automated way to help auditors figure out the version numbers and this will definitely help. 
• Set-Writeable - This just removes the Read-Only flag from files, but it would have come in handy last year when we had to upgrade several projects to the 2.0 framework and move them from VSS to TFS. Removing a project from VSS is painful and one of the more tedious steps is clearing the read-only flag on all the source files. I remember wasting an hour of my life writing and debugging a VBScript because the right-click/recursive method never seemed to fully work.
• Out-Clipboard - One of the things that still sucks about working in the shell environment is the editing capabilities, especially when it comes to copy and pasting. When I want to transfer the results of a command to an email or word doc, I simply pipe it to this cmdlet.
• Send-Smtp - This will be great for the error-handling notification part of scripts. It would be even better if I could figure out how to populate the body parameter from the pipeline.

You can get a list of the 59 new cmdLets added when installing this snap-in by calling the function gcmpscx.

Anything that can be done with ADO.NET is possible in PowerShell through a call to Assembly::Load and the New-Object cmdLet, but that doesn’t mean that the resulting code will seem very shell-like or practical. Two approaches to database access that definitely do seem compelling to me, however, are the sample SQL Provider recently demoed by the PowerShell team and Dan Sullivan’s PowerSMO script, which is really just a thin wrapper around the SQL Management Objects found in the Microsoft.SqlServer.Management.Smo namespace. The SQL Provider allows traditional command line navigation through the database using dir and cd and the SMO objects provide access to nearly any administrative function in SQL Server.

In order to use the PowerShell team’s SQL Provider, you’ll need to download SQLProvider.dll, install it in the GAC, and call the Add-PSSnapin cmdLet. Then you can treat the database just like the filesystem and cd or dir your way around with statements such as the following:

PS>cd SQL:

SQL:>cd MySQLServerName

SQL:>dir

master
tempdb
model
msdb
pubs
Northwind

SQL:>cd Northwind

SQL:>dir

name
—-
Categories
CustomerCustomerDemo
CustomerDemographics
.. etc…

SQL:>cd Employees

SQL:>Dir -filter {where EmployeeId=8}

EmployeeID      : 8
LastName        : Callahan
…etc…

Unfortunately, this provider is only a partially implemented sample and I quickly lost interest when I figured out that I couldn’t easily or naturally pipe the results to the where, select, and sort cmdLets. I did some quick searching to see if someone else had taken the next step and fully implemented a sql provider but I didn’t come across anything. However, I’m sure it won’t be long before somebody creates a production-ready version. The PS team did include the code for their sample provider, so perhaps I’ll take a look at it myself to see how hard it is.

To run the PowerSMO examples found on Dan Sullivan’s blog, you simply need to run a powershell script or place it in your profile so that it is run automatically when you start the shell. The script preloads some required dll’s and provides a factory function for retrieving some of the key objects, so retrieving a server object feels more like a cmdLet experience.

$server = get-SMO_Server mySQLServerName

However, since he warns that his script is for instructional purposes only, you might also just want to load the server object directly: 

[System.Reflection.Assembly]::Load(”Microsoft.SqlServer.Smo,Culture=Neutral,Version=9.0.242.0,PublicKeyToken=89845dcd8080cc91″) | out-null

$server = new-object “Microsoft.SqlServer.Management.Smo.Server” myServer

Then you can grab a database object and do interesting things like finding which tables and indexes in your database are taking up the most space.

$pubs = $s.databases | ?{$_.name -eq “pubs”}

$pubs.tables | ?{$_.name -notlike “sys*”} | select name, dataspaceused,indexspaceused,@{name=”Total Space”;expression={$_.dataspaceused + $_.indexspaceused}} | sort dataspaceused -desc

…or finding out the firve widest tables in the database as potential candidates for normalization and save it to an html file.

 $pubs.tables | ?{$_.name -notlike “sys*”} |select name, @{name=”Columns”;expression={$_.columns.count}} -first 5 | sort columns -desc | convertto-html > C:\wideTables.html

Again, you can do almost any of these functions through some other way (system sproc, sql function, utility), but what is significant is the capabilities that powerShell gives you to be able to manipulate the resulting data once you have it (sort, group, filter, format, etc.).

This weekend I completed my 10th marathon/ultra marathon in Lake Okoboji in Nebraska . That makes nine states in my quest to join the 50 state club, which requires you to run a marathon in each state. I picked this goal mostly because it gives me a good excuse to travel around to some places I probably wouldn’t have otherwise seen and it also helps keep me motivated with running. Eventually I’d also like to run a marathon on each of the continents, although this would mean running in the North Pole Marathon which seems a little crazy even to me. Here are some highlights from this Iowa road trip and marathon:

• We ain’t in Chicago anymore… - When I ran the Chicago marathon a few years back there were 40,000 runners. It was a little claustrophobic in the beginning (it took almost 10 minutes to cross the starting line after the gun went off), but it was thrilling to have so many spectators cheering along the way. By contrast, there were less than one hundred people running the in this marathon. The mile markers were done in chalk on the street, there was no post race food, and the majority of spectators along the way were simply in the process of getting their morning newspaper or trying not to kill us as they drove by (no streets were closed). Nevertheless, they had plenty of water stops and someone actually announced your name and where you were from as you crossed the finish line. I wouldn’t recommend it for a first time marathoner, but I still enjoyed it.
• An Oasis in the Middle of Cornfield Land - We saw nothing but corn fields for the last two hours of driving, so I was pleasantly surprised when we entered the town that hosted the marathon and saw a beautiful blue lake with lots of boats that was surrounded by expensive looking houses and quaint shops. I’ll definitely consider this as a destination for my next mini vacation.
• Perfect Weather - It was 60 degrees with a cool breeze at 6:00 AM when we started, which is quite unexpected for a July marathon in the Midwest. Running a marathon in summer heat can be brutal, which is why most marathons are in the spring or fall (even then it is a crap shoot…I ran one in Kansas City a few years ago in late September where it was over ninety degrees). This is why the weather seemed to be a favorite topic of conversation among the runners.
• Negative Split - I finished in just under four hours, which overall wasn’t a great time for me, but I was nevertheless very excited because this was the first time I ever ran a negative split. This means that I ran the second 13 miles faster than the first 13 miles (an average of 2 minutes faster per mile which is a big difference in pace). I’ve had a history of bonking at around mile 20, so it was very invigorating to finish with a bounce in my step and be zipping past other people during the last six miles. This was partly possible because I ran with a group of friends who weren’t in a hurry until about the halfway point when I got antsy. I also attribute my strong finish to a little bag of salt that I carried with me and dug into a few times after mile 16. I think that helped counter balance the electrolyte imbalance I usually fall victim to late in the race.
• Peanut Butter & Jelly and Pizza Bagels - Since running 26 miles can often anger your digestive system, it is usually inadvisable to deviate from your normal eating routine before the race. However, this time the only thing we could scrounge up for breakfast were bagels with peanut butter and jelly, which turned out to be a surprisingly tastey and energizing combination. The real food star, however, was the pepperoni pizza bagel after the race. My normal ravenous post-race hunger was exacerbated by the fact that there was no food at the finish line like there has been at every other race I’ve run. Luckily, someone from our group had some cash and we were near a bagel shop. Nobody could think of a single super model/lingerie combination that we would have traded for that pepperoni pizza bagel at that moment. I think I may have actually growled while eating it. 
• The Helpful German Dominatrix - For the first hour of the road trip, we were all totally enamored with the Garmin GPS device that we programmed to speak to us in a rather firm female German voice and that we soon dubbed ‘the Dominatrix’. In all seriousness, I totally want one of these devices. I have the worst sense of direction of anyone I’ve ever met and I’m surprised that I haven’t been mandated by the state yet to get one permanently implanted on my body somewhere. 
• Funky smells - Although much of the run was within view of a beautiful lake, we did put quite a few miles in on some country roads that were surrounded by farms and reeked of cow manure. Believe me, the last thing you want to smell when you are gasping for air is cow manure. Even worse was the smell on the bus that shuttled all the sweat-encrusted, odiferous finishers back back to the starting line where everybody’s cars were parked. I fear my nose may never fully recover.
• A Nearly Fatal Ill-Timed Joke - Three of us were waiting just around the corner from the finish line for the last two to finish. They arrived later than we expected and I noticed right away that one of them was not a happy camper. Our running companion next to me apparently did not notice this, because she jokingly cheered them on by saying “only two more miles”. I almost dived for cover as I anticipated the violence that surely would have ensued if the poor beleaguered runners weren’t so exhausted. Whew…close one.

Mostly, I knew it was a good race because JR and I were already looking at maps and trying to strategically pick our next marathon within a few hours of finishing the race. Usually it takes us several days to block out the grueling experience enough to openly admit to wanting to do another one.

The more I work with powershell, the more I start seeing everything that lives on the computer as part of a giant database that I can query. When used in conjunction with the pipeline and get-member -type property to discover the available properties on an object, the where-object, sort-object, select-object, and group-object cmdlets can be strung together to answer some very sophisticated questions. We often seem to run out of space on our development sql servers, so one tedious question that I am often forced to answer manually or with a utility such as windirstat is which sql backups are taking the most space so we can delete unused ones and free up space. This question can be answered easily and quickly in one line from powershell and then exported to text, csv, or html so that it can be sent to anyone else that needs to assist in the decision making process. In fact, with an extra line of code you could easily load up the .NET SMTPClient class in system.net.mail and email the output without leaving the command line. Here is the powershell one-liner that I used to find the largest 10 backups on a development server and save the output to a text file:

Dir \\devsql\sqlbackups *.bak -rec | sort length -desc | select name,directory, @{name=”Size in GB”;Expression={[System.Math]::Round($_.length/1073741824,1)}},CreationTime -first 10 | ft -auto > C:\LargeBackups.txt

This produces output that looks something like this:

Name                                      Directory                   Size-GB   CreationTime 
—-                                           ———                          ——–       ———— 
LendingHistory.bak              \\devsql\sqlbackups\Dev03       7.5     0/27/2006
Lending0702.bak                 \\devsql\sqlbackups\Dev01       6.2     06/29/2007
LendingPostEOD0628.bak   \\devsql\sqlbackups\Dev01       6.2     06/29/2007
LendingSOD0628.bak          \\devsql\sqlbackups\Dev01      6.1      06/12/2007
LendingSOD0627.bak          \\devsql\sqlbackups\Dev01      6.1      06/29/2007
Lending.bak                        \\devsql\sqlbackups\Dev02      6.1      11/27/2006
Investments.bak                  \\devsql\sqlbackups\Dev02      6.1      05/23/2007
InvestmentsHistory.bak        \\devsql\sqlbackups\Dev02     6 1       1/27/2006
Warehouse.bak                   \\devsql\sqlbackups\Dev01     5.9        04/25/2007
Lending.bak                        \\devsql\sqlbackups\Dev01     5.9        10/27/2006

I’m doing some work now on an internal web application where I had to logon with an external test account before I could do anything. That got old fast, so I created a powershell script file to automatically launch IE, input the username and password, and click the submit button. I originally tried to do this with WatiN, which provides a much easier interface for controlling the browser than the COM object InternetExplorer.Application, but I couldn’t get by the security exceptions explained in this Scott Hanselman post. I then added a keystroke shortcut in WinKey to execute this ps1 file via the hstart utility so that no command window would flash on the screen before the browser window could load. It’s probably not the cleanest solution, but it has been a huge time and annoyance saver.

Here’s the winkey info

command: <>\hstart.exe
parameters: /nowindow “C:\WINDOWS\system32\windowspowershell\v1.0\powershell.exe -command “”&”  ‘C:\myScriptsDirectory\launchWebApp.ps1′”"

…And the powershell script

$psie = new-object -com InternetExplorer.Application
$psie.Navigate2(”http://www.someWebApp/”)
$psie.visible=1
($psie.document.getElementsByTagName(”input”) | where { $_.Name -eq “UserID”}).value = “userID”
($psie.document.getElementsByTagName(”input”) | where { $_.Name -eq “Password”}).value = “password”
($psie.Document.GetElementsByTagname(”input”) | where { $_.Value -eq “Login”}).click()

 

I just heard about Justice Gray’s blog challenge from the latest episode of Hanselminutes while running today. Although I think his plan to read one developer book a week for the next six months is rather insane, I agree with the premise that setting specific goals publicly is probably the best way to hold yourself accountable. Here are my goals for the next 6 months:

Add Tools to my Developer Toolbox - My productivity and troubleshooting skills would plummet if I were to suddenly stop using utilities like FileMon, RegMon, Fiddler, ProcessExplorer, TestDriven.NET, MSBuild, SQLPrompt, SQL Profiler, WinKey, and Lutz’s Reflector. In the next six months I am going to learn to use Resharper, NDepend, Roy Osherove’s Regulator, and a good packet sniffer (not sure which one yet).

Become an Avid Code Reader - Writers are commonly given the advice to read good books in order to become a better writer and I think the same holds true for developers and code. Unfortunately, it is easy to get caught up in only writing code or only reading a small subset of internal code that you are stuck supporting. I have my eye on several open source projects that I’d like to spelunk in the near future, including the Rotor implementation of the .NET framework, SubText, DasBlog, RSS Bandit, NHibernate, Rhino.Mocks, and WatiN. I also plan to use Reflector more often to decompile .NET assemblies and figure out how .NET framework and how other third party, non-open source .NET utilities work.

Step outside of my MS-OOP comfort zone - My big goal is to get through the Ruby For Rails book that has been sitting on my shelf for the last six months so I can see why there are so many Ruby fanatics and why dynamic languages in general are becoming so popular that even Microsoft has jumped on the bandwagon (Lamda Expressions, the DLR, IronPython, IronRuby, etc.). I am also interested in learning more about the various Castle projects, especially the Windsor Container which is supposed to help enforce the IoC (Inversion of Control) design principle that helps remove complexity and increase testability by parameterizing embedded dependencies. I would also like to install Ubuntu on a VM so I can see what the Linux OS looks like these days as well as make my way through an F# tutorial so I can break away from my OOP bias and understand the functional language perspective. Finally, I want to experiment with Boo, a CLI language that is a hybrid of Python, Ruby, and C#.

Become a Better Developer Community Citizen-  I’ve already taken steps in that direction this year by becoming more tuned in to the blogger community, starting this blog, attending local .NET user group meetings, and even agreeing to speak on Powershell at the next Topeka .NET User Group meeting. What I would really like to do in the next six months is to start contributing code to an open source project. I’m open to suggestions at this point…

Push Myself Lower and Higher on the Abstraction Stack - From a productivity standpoint, I think it is crucial at this evolutionary stage of the software industry to figure out ways to avoid writing tedious boilerplate and plumbing code by utilizing higher level abstraction tools as much as possible such as code generators (CodeSmith) and ORM tools (NHibernate). Conversely, I also plan on filling in some gaps in my own understanding by dropping lower on the abstraction stack and finishing a MASM Assembler tutorial I downloaded a while ago.

I am not going to create a firm schedule for these goals like Justice Gray did because I am much more productive when I let natural curiosity determine the sequence of learning. However, I am going to commit to writing at least one post on each of the following topics over the next 6 months:

• Resharper
• NDepend
• Regulator
• Packet Sniffer
• Impressions on the source code from at least one open source project
• Ruby On Rails
• F#
• Ubuntu
• Anouncement of which open source project I joined
• NHibernate
• Windsor Container
• CodeSmith
• Assembly

Whew, maybe I should have just agreed to read one book week too…

Here is a good joke to play on one of your developer buddies if they have powershell and SQL Server installed on their local machines and you have sa rights on the SQL Server for whatever reason (former debugging help, open environment between developers, blank sa password, or mad hacker skills on your part). Wait until they are busy typing and execute the following query in Query Analyzer (or using sqlcmd.exe if you want to be a purist) against their SQL Server. You may want to lock down your own machine before trying this in order to avoid certain retribution…:-)

master..xp_cmdshell “powershell.exe (new-object -com ‘SAPI.spvoice’).speak(’Ouch. You are typing too hard’)”

This is a little more difficult with SQL 2005 because remote connections and xp_cmdshell are both disabled by default for obvious security reasons. You can enable remote connections by adding exceptions to the windows firewall for SQLServer and SQL Browser as described in the following KB article. You can enable xp_cmdshell in SQL 2005 with the following script:

EXECUTE sp_configure ’show advanced options’, 1
RECONFIGURE WITH OVERRIDE
GO
EXECUTE sp_configure ‘xp_cmdshell’, ‘1′
RECONFIGURE WITH OVERRIDE
GO
EXECUTE sp_configure ’show advanced options’, 0
RECONFIGURE WITH OVERRIDE
GO

If you’re really itching to do this prank but they don’t have Powershell or SQL Server installed, there are still lots of options. All you really need is a way to shell out a command against someone else’s computer since the underlying voice capabilities come from a COM component, which you can call from vbs script. Have fun…

It appears as though we will have a new version of SQL Server 2008 next February 27th. This is well ahead of what my sarcastically titled post on SQL Server 2008 predicted. So much for my career as a technical psychic… It will also be the launch for Visual Studio 2008 and Windows Server 2008. That should be one heck of a party. Hopefully I can find some launch event that will be serving free beer (preferably a nice stout or porter).

I’ve been impressed by the attention paid to discoverabiltiy in powershell, which has allowed me to quite a bit of learning without ever leaving the command line. Here are a few of my favorite features:

Guessing - This may seem strange to list as the number one discoverability feature, but I think the Powershell design team did an excellent job of exposing functionality through well scoped cmdLets that are logically and consistently named. With cmd.exe, the majority of the heavy lifting is done by executables with terse names that have become bloated with too many responsibilities so that you have to constantly rely on references to figure things out. In powershell, the verb-noun pattern is strictly enforced and the names are always consistent so that I can write Get-Command *service and be assured that I will get all the cmdLets related to services and then figure out what they do through the verb part of their name.

Built-In Help - If you’re not sure what a cmdLet does, then use the Get-Help cmdLet to see the documentation. DOS exes usually have a ? switch for help, but what really makes this powershell feature shine are the -full and -example switches that bring back detailed examples and additional notes about usage and each of the parameters. The powershell team also embedded plenty of conceptual help outside of cmdlets that follow the about_ pattern. You can see them through Get-Help about_*.

Trusting Operations - Are you afraid to execute a script because you’re not sure what it will do? Try the -whatif parameter that is ubiquitous to all commands. For example, del C:\f*.txt -whatif will just tell you which files would have been deleted rather than actually deleting them.

Get-Member - I find this cmdLet most useful when I’m not sure what object I am getting back from the pipeline. I simply add | Get-member to the end of the command and I can see what additional methods I have to work with or what properties I can perform filtering and sorting on.

Get-Alias- I love all the transitional and convenience aliases provided by powershell to maximize productivity for throw-away, administrative tasks, but sometimes it makes it hard to decipher examples that you see on the web. Calling Get-Alias gps will return the full, logical name of the cmdLet and will usually allow you to figure out its function. Better yet, use this command with no parameters to return all the aliases so you can learn them and increase your own command line speed.

Other resources - Despite all these great features, it would be silly to not also seek outside resources. Here are a few that I’ve found helpful

WebCasts - I’ve made my way through 2 of the 5 Powershell Week web casts. They are a little long (usually over an hour), but are full of good information and perfect for the visual learners.

Blogs - The best is probably the one put out by the Powershell Team. I also recommend Keith Hill and The Powershell Guy.

PodCasts - I highly recommend the all 3 shows that Scott Hanselman did on the subject. The Powerscripting Podcast is a little too slow paced for my taste, but it could be good if you’re brand new and want something to listen to while mowing the lawn.

Tutorials - The Powershell Owner’s Manual on the technet script center isn’t bad, although the authors went a little overboard at times to make the writing more interesting than the average tech spec.

Books - I agree with Chris Sells and Don Box that books are still the best way to grok the zen of a technology. I’m finishing up with an excellent book by the Powershell Developer Lead, Bruce Payette, called Powershell in Action that I would highly recommend.

A good friend of mine just sent me an email razzing me about my blog. Besides being a current running and drinking buddy, he also happens to be former project manager of mine who put some time in on the early stages of a large waterfall project that I reminisced about in a previous post. He gave me a tongue-in-cheek thanks for not blaming the project manager for the “whooshing sound“. I’ll probably have to do shots for this, but I dedicate this most excellent project manager joke to him. Cheers!

A young Programmer and his Project Manager board a train headed through the mountains on its way to Wichita. They can find no place to sit except for two seats right across the aisle from a young woman and her grandmother. After a while, it is obvious that the young woman and the young programmer are interested in each other, because they are giving each other looks. Soon the train passes into a tunnel and it is pitch
black. There is a sound of a kiss followed by the sound of a slap.

When the train emerges from the tunnel, the four sit there without saying a word. The grandmother is thinking to herself, “It was very brash for that young man to kiss my granddaughter, but I’m glad she slapped him.”

The Project manager is sitting there thinking, “I didn’t know the young tech was brave enough to kiss the girl, but I sure wish she hadn’t missed him when she slapped me!”

The young woman was sitting and thinking, “I’m glad the guy kissed me, but I wish my grandmother had not slapped him!”

The young programmer sat there with a satisfied smile on his face. He thought to himself, “Life is good. How often does a guy have the chance to kiss a beautiful girl and slap his Project manager all at the same time!”