Archive for the 'Tools' Category

Essential Tool List Reloaded

Posted August 31, 2009 * Comments(1)

Here’s an update to my Essential Tool list that I published last year. I highlighted all of the additions and changes in red.

I’m in JP’s Nothin’ But NET course this week, so I’m guessing that I’ll have many new ones to add shortly. However, I’ll probably also be so worn out from the class that I won’t be able to publish it for a while. The prep work alone almost did me in…:-)

As always, any new suggestions are welcome.

Development

Code Analysis

Development Lifecycle

Debugging

Utility

MISC

Essential WebSites

Popularity: 2% [?]

Frictionless Code Reviews

Posted August 28, 2009 * Comments(8)

Having a second set of eyes on your code has multiple benefits.

Besides increasing the odds of catching bugs earlier in the process when they are less expensive to fix, code reviews tend to yield higher quality code that is more maintainable because you get feedback from someone who has not been immersed in the code and thus does not suffer from the blind spots that can be caused by working on something for too long.

It also leads to greater cross training in the code base (the hit by a bus factor) as well as knowledge sharing about coding practices in general. I’ve learned a lot of new techniques by simply reading code written by my co-workers.

Unfortunately, regular code reviews seem to be a rare practice in the industry because they are expensive in terms of time and difficult to manage due to egos. Most developers are very sensitive about the code they write and therefore it’s difficult to offer criticism without it being interpreted as a personal attack.

Even if reviews are built into a software shop’s SDLC, code reviews are often viewed as an ancillary activity and thus are usually first to be cut once a deadline looms (which is pretty much all the time in my experience).

Pair programming is one popular way to tackle this problem. It has all the benefits of code review, but bypasses the emotional issues by turning the review into a collaborative exercise. It also makes reviews more likely to happen because it becomes intertwined with the development process.

Although I clearly see the conceptual benefits of pair programming, I must admit that I am never very excited to participate in pair programming sessions on a regular basis. It’s probably just due to my introverted nature, but I feel most productive when I am in “the zone” and I can never seem to attain that state unless I am alone and in an environment with few distractions.

An alternative approach to pair programming that still makes the code review process inexpensive, stress free, and an integral part of the development process is to make it a required step of the development workflow and then adopt tools that make the process quick and painless to perform.

If code review happens all the time, then it ceases to be a stressful activity. If review is a required step before testing can occur (e.g. a step in the deployment script queries the issue tracking system and stops the deployment process if any unreviewed issues are returned), then it is no longer ancillary and avoidable. If tools help make it a frictionless process then it won’t be an undue burden that kills productivity.

I think we’ve achieved a relatively frictionless review process at StoreFinancial with the help of JIRA, Fisheye, TortoiseSVN, and a SVN hook.

First we ensure that a valid Jira key is entered into the comment field of the TortoiseSVN commit window with the help of a custom pre-commit hook that queries Jira to verify that the issue exists and is in an opened or resolved state before allowing the commit. I’ll describe how I did the hook in my next post and provide the source code.

If a developer forgets to enter the key or fat fingers the entry, then the code commit fails.

We originally purchased Fisheye to allow fast and sophisticated source control searching, but by far the biggest benefit we gained was the automatic association that happens between source code files and JIRA issues and the ability to view code directly in the browser. Within seconds of doing a commit, links to source control files will show up on a tab of the issue.

This allows the reviewer (in our case whoever finishes their developer tasks first) to grab an issue and quickly review it by simply clicking on the diff links of the Fisheye tab. The color coding scheme makes changes easy to see and the fact that it is within the browser minimizes the number of clicks required to do the reviews.

After reviewing all the files associated with the issue, the review can either mark the issue as reviewed or reopen it with comments and reassign it to the original developer to make the changes.

There are still parts of the code review process that I would like to streamline more.

In particular, I don’t like being a code formatting nazi so I’d like to move to shared ReSharper templates that automatically enforce formatting and syntax standards.

Otherwise, I’m quite pleased with the current state of our process and tools. With the addition of some simple tools it went from being expensive drudgery to being an relatively painless and valuable part of the development process.

Popularity: 2% [?]

The 10 Second Visual Studio Makeover

Posted January 27, 2009 * Comments(7)

I blame it on being forced to sit through one too many episodes of my wife’s favorite TV show, What Not To Wear, but I finally became inspired to retire my old, tired-looking default Visual Studio color scheme in favor of one of the hipper ones with a black background.

It turned out to be much easier than I thought since several bloggers already went to the trouble of exporting and publishing their favorite Visual Studio settings files, which you can import by simply selecting the ‘Import and Export Settings’ menu option under the Tools menu.

My favorites are the ones that try to mimic the Vibrant Ink theme from TextMate, the popular Rails IDE for the Mac. Rob Connery posted a nice version, but I finally settled on the one published by John Lam here.

Here’s a screen shot of me perusing CodeCampServer, the most excellent sample app for the upcoming ASP.NET MVC in Action book from Manning and that’s being written by Jeffrey Palermo, Ben Scheirman, and Jimmy Bogard.

vs-theme-screenshot

Mostly I’m happy I made the switch because I think the new color schema is easier on my eye, but I also can’t help but notice that I feel slightly more positive towards my otherwise homely and rather bloated IDE.

I guess the moral of the story is that you should never underestimate the power of lipstick when confronted with a pig.

Popularity: 24% [?]

Tips on Upgrading from VSS to Subversion

Posted January 22, 2009 * Comments(2)

We’ve almost completed our first week on Subversion and all is well. Here are a few pieces of advice to those of you about to embark on a VSS to SVN migration of your own.

  1. Spend Time Getting Buy-In from all Developers BEFORE the Migration – Force feeding a new tool to your co-workers is a recipe for disaster. Developers who aren’t given any input or choice in the matter may be compliant at first but will likely turn passive-aggressive on you at the first sign of trouble. We did some lunch-and-learns and focused specifically on current source control related problems that would be solved by migrating to a new tool well before we did the actual migration. This made life much easier during the inevitable bumps that came up during the first few days after the migration.
  2. Be Aware of Tool Limitations When Migrating History – Unfortunately there don’t seem to be very many good options out there when it comes to migrating VSS history. After some failed attempts at using VSS2SVN and SVNImporter, I settled on using Eric Lane’s version of VSSMigrate. It accurately converted all the files in a 5GB VSS database, preserved history from the perspective of a single file, and stripped the VSS bindings from the project and solution files. The utility is also written in C# and the source code is provided so you can make tweaks as needed (I changed it to put the associated version label in the comment of the revision). Unfortunately, it doesn’t convert labels and more importantly it doesn’t preserve history from a holistic perspective since it loops through and commits the version history of one file at a time. If you really only care about doing diffs and not about reconstructing your apps as of a certain point in time, then I would recommend using this tool. Otherwise, keep looking.
  3. Do Several Practice Migrations – I set up a Virtual Machine that mirrored our source control server and not only went through the migration several times on a copy of the full VSS database, but also documented each step to make life easier during the actual migration, which took place over the weekend when I was less…focused.
  4. Don’t Forget About Dependencies – We had quite a few Nant, CruiseControl, and VisualBuild scripts that needed to be modified, which added quite a bit of time and effort to the migration process. A VM definitely came in handy when testing these changes as well.
  5. Run VSSAnalyze Before Migration – Corrupted VSS chains are common and will likely cause havoc with a migration tool unless you try to fix them prior to the migration. Even after running Analyze, the VSS equivalent of duct tape, there were still 2 files with corrupted version history that I had to remove from the VSS database prior to the migration and manually add to SVN after the migration.
  6. Use BeyondCompare to Verify Migration – Make sure that you do a full comparison of working directories from the new SVN repository and old VSS repository. Before performing the comparison, you’ll want to configure exclusions for the svn and vss specific files as well as the project and solution files if they have been stripped of bindings either by the migration tool or manually.
  7. Open Up and Compile All Solutions After Migration – I discovered a resource file that somehow got dropped during the migration because one of the solutions would no longer compile.
  8. Use SharpSVN to Extend and Script For SVN – I was surprised at how easy it was to create a pre-commit hook for SVN that would analyze commits and prevent the ones that didn’t contain a valid issue for our issue tracker (JIRA). This was partially because I used SharpSVN, an open source API, and was thus able to interact with the repository without having to parse command line return values from SvnLook.
  9. Practice Backups and Restores Before Migration – Resist the temptation to put this off until after the migration and make sure you fully test any scripts that you use, especially if you don’t use the recommended Python scripts that can be found on the official SVN site . I couldn’t convince the Powers-That-Be to let me install a Python interpreter on our server, so I went surfing for PowerShell scripts and the first one I tried for incremental backups only actually saved the latest revision instead of the latest range of revisions. I didn’t discover until doing some testing on a practice restore that I did.
  10. Plan the Migration over a Weekend – Most of the processes are very long running. The one that migrated our VSS history took almost 10 hours for a 5 GB database and we ran into some problems with our build scripts that took much of the day Sunday to debug.

Happy Migration!

Popularity: 27% [?]

Don’t Be a Cotton-Headed-Ninny-Muggins When Copying Files

Posted January 20, 2009 * Comments(4)

I just submitted a feature request to Microsoft.

I thought that the perhaps the Windows Explorer file copy dialog could be a bit more truthful and instead of displaying something like this whenever a user is trying to copy something over 100 MB in size through Windows Explorer…

windowscopydialog

…it should instead show something like this…

windowscopydialog_2

Just to be safe,  I sent the above mockup directly to Bill Gates. I know he doesn’t run things in Redmond anymore but I figured he probably still eats a half dozen alpha-geeks raw for breakfast every morning and could surely make this happen.

In the meantime, I highly recommend that you find some other utility to hold you over whenever you have to perform a large copy job.

I used RoboCopy this weekend when I was forced to do an enormous one over a slow and unreliable VPN and was pleasantly surprised by how fast and robust it was.

Not only did it magically pick up where it left off every time I reconnected after a network failure, but it was even smart enough to only copy over missing files when I reran it this morning after discovering a few files were missing when I did a BeyondCompare sanity check (I didn’t know about the /R switch). I was prepared for several more hours of copying but I was greeted with this after only about 30 seconds.

robocopy


Not too shabby.

Moreover, there are a ton of other switches that allow fine grained control over how file attributes and ACL’s are copied over.

Despite it being an unassuming command line utility that ships with the Windows Server 2003 Resource Kit Tools, RoboCopy would clearly make Windows Explorer its bitch if they ever had it out in a File Copy Death Match.

Of course this is but one of a million good file copy options. The ToolMeister himself wrote about his favorites here and his HanselFollowers provided dozens of other good options in the comment section.

Whatever you decide to use for your future large copy jobs, just don’t be a cotton-headed-ninny-muggins and rely on Windows Explorer.

Popularity: 18% [?]

Using YSlow and HammerHead to Enhance Web App Performance

Posted January 12, 2009 * Comments(3)

I recently installed YSlow, a Firefox add-on integrated with the popular Firebug web development tool that does performance analysis based on the rules for high performance web sites developed by Yahoo.

I tried it on DosPecesCreations.com, the e-commerce site I recently built for my wife using Ruby on Rails, and was surprised to learn that the site received a failing grade according to the YSlow Report Card.

yslow_grade_before

It took me just over an hour of clicking on the explanations for failed failed categories and researching how to make some of the configuration changes in Apache to bring that grade up to a high B.

Bringing it up to an A would require me to use a Content Delivery Network like Akamai Technologies, which deploys content across multiple, geographically dispersed servers and dynamically chooses the fastest option in a given scenario based on proximity and available connections. However, since the customer base for the site is not international and we don’t have thousands of dollars a month to spend at the moment, I’ll just have to content myself with being a B student.

So what was the payoff for making these changes?

I wanted to have a somewhat objective way to measure the performance improvements, so I also downloaded HammerHead, another firebug add-on that provides average load times for pages with and without the cache being used.

Here are my baseline metrics.

yslow_2

Here are the results after I made the changes.

yslow_after_2

As you can see, I more than doubled the speed of the site with little effort and almost no coding changes. Here’s a quick summary of what I did:

  1. Reduced the size of the HTTP response
    • Enabled Gzip Compression -  This is done in Apache with an output filter, which I enabled by adding one line to the .htaccess in the root directory of the site.
    • Minified Javascript Files – This involved removing all the comments as well as unneeded white space characters (space, newline, and tab) from javascript files. Luckily there are tools such as JSMin that will do this for you. I ran the Ruby version of JSMin as a Rake task and was able to reduce my Prototype Javascript Library files by about 25%.
    • Removed Unused Javascript Reference – This sounds obvious, but by using the default option with the javascript_include_tag Rails helper tag I was referencing 6 Prototype files even though I was only using 2 of them. By removing the unused references, I was able to trim 65k from my pages.
  2. Reduced # of trips
    • Merged javascript files – The Rake task I used above also merged the remaining prototype files into one file so the client would have to make one less round trip to retrieve the files.
    • Enabled Caching – I configured Apache to use Far Future Expire tags for my static content (images, js, & css) and Cache-Control tags for my dynamic content (2 hour expiration). I followed the recommendation of disabling ETags as well even though this only appears to only interfere with caching on web farms and I’m currently on a single server.
  3. Maximized parallel download speed – I did this by moving all my scripts from the html header to the bottom of the page. Apparently javascript files can block parallel downloads so simply moving these from the header to the bottom of the page will increase the overall load time.

I used this resource while making all of the Apache configuration changes. If you’re using IIS, then you might want to try this post.

As someone who spends most of his time doing middle tier and database development, I was surprised to learn how much of a web application’s performance is affected by these front end concerns.

Luckily, all of these front-end recommendations were much easier to fix than almost any performance tuning efforts applied to the back end.

Popularity: 20% [?]

Finding a Good Cheap Home for Side Projects with Unfuddle

Posted December 29, 2008 * Comments(6)

If you’re looking for an external, non-open source home for your code or a light-weight, web-based project management tool, I highly recommend using Unfuddle.

Actually, up until recently I wasn’t all that crazy about this site. We had been using it at work for the last year as a project management tool for external clients who wanted to integrate with our web services and it seemed rather limited and clunky when compared to JIRA, the most excellent issue tracking tool we use internally (there were various internal constraints that prevented us from using JIRA externally).

It wasn’t really until I started searching for an external location for my DosPecesCreation repository, the project I’ve been working on in my spare time recently, that I gained a new appreciation for Unfuddle.

Most of the free options for externally hosted repositories out there seem to be limited to open source projects, but Unfuddle offers a private plan for free that provides an unlimited number of Subversion or Git repositories along with 1 associated project for issue tracking and access for 2 people.

By moving my code from a local repository to unfuddle, I was not only able to work from multiple computers without the headaches involved in using a tool like Live Sync (formerly FolderShare) in conjunction with SVN, but I was also able to offload much of the CSS and HTML work to my wife. Before she had been sending me email approximations of the changes she wanted, but with a quick 5 minute tutorial on TortoiseSVN along with some automated scripts and shortcuts to set up her Rails environment, she was able to make the changes directly herself.

Of course from my lovely stakeholder’s perspective, that that mostly just meant that I now had more time to work on the now rather large list of programming-related feature requests that she had thought up for version 2.0 of the site.

Luckily, Unfuddle has helped with this too. It dramatically increased my ability to organize and group tasks while adding only a minimal amount of overhead. The last thing I want when I am working on a project in my free time is anything remotely heavy weight in terms of process.

While this minimalist approach might not work as well in heavily regulated corporate environments, it fits beautifully for small, agile teams and especially for small 1-2 developer projects.

Here’s a screenshot of the current tickets broken down by milestones (which we decided to base upon 2 week release cycles). As you’ll notice, I’ll be spending some nights in the near future working on caching and implementing all of YSlow’s suggestions in order to speed up the site a bit.

DPC_Unfuddle

I had a sneaking suspicion that my nights would be busy for quite some time, but now thanks to Unfuddle I have a much clearer picture of the extent of my servitude.

Looks like I’ll have free time again sometime in the fall of 2014…Thanks unfuddle!

Popularity: 7% [?]

Curing Cancer, Preventing Global Warming, and Achieving World Peace by Migrating from VSS to SVN

Posted December 15, 2008 * Comments(15)

I admit that the title is slightly misleading. There will obviously never be meaningful peace between the Microtards and Macophiles.

Nevertheless, if you are currently languishing under an oppressive Visual Source Safe regime, then I hope to inspire you to rally your coworkers and end the source control tyranny once and for all.

By switching from VSS to Subversion (VisualSVN Server makes it extra easy), you will be able to …

  1. Rename files within Visual Studio – I can’t believe that after all these years something as simple and basic as renaming a file within the IDE (even in VS2008) still doesn’t work correctly with VSS. Even if you manage to save the history of your file by remembering to change it first with the VSS client and then change it again in the Visual Studio, you are still likely to break either the continuous integration build or screw up one of your co-workers who happens to get the latest version while you are in the middle of doing this. Even AnkhSVN, the free open source Visual Studio plugin that provides integration for SVN, can seemlessly handle this scenario. All I can say is WTF Microsoft?
  2. TortoiseSVN Interact with the repository through Windows Explorer – I definitely recommend using TortoiseSVN when interacting with your SVN repository outside of Visual Studio. It is a Windows Explorer shell extension, so you can execute all of the source control commands by simply right clicking within a normal Windows Explorer window. Some of my co-workers were reluctant to give SVN a chance at first because they assumed that it required a substantial amount of CommandLine-Fu. While SVN does have great command line support, TortoiseSVN makes it extremely intuitive and easy to use.
  3. Use your windows password – How is it that SVN can use integrated security but VSS cannot (WTF # 2)? Not only are you forced to deal with another set of passwords for VSS, but it won’t even enforce any type of password complexity rules. As a result, most of the developers at my old job used blank passwords for 5-10 years thus making security a joke even though it was a heavily audited environment. Even if you’re not too concerned with securing your source code, you will at least appreciate having one less password to manage.
  4. Cease to worry about the repository size – Microsoft recommends that VSS database sizes not exceed 3 to 5 GB (no, it hasn’t improved at all in VSS2005) This is a rather serious limitation since the two places I’ve worked at before that have used VSS both easily surpassed that mark despite having a relatively small number of developers. Besides requiring a lot less space because revisions are stored as differentials, SVN has no inherent size limitations after which corruption starts becoming likely.
  5. No longer have to regularly duct-tape your repository – I always found it troubling that an official Microsoft best practice was to frequently run Analzye.exe, a utility whose sole purpose is to repair corrupted files. If that wasn’t bad enough, this utility takes forever to run (4 hours where I work) and requires everyone to be out of the database while it attempts to fix the corruptions that appear to happen whenever someone hits the enter key too forcefully.
  6. Retrieve files faster – The solution I work in most often has 16 projects (yes I know we need to refactor it…) and doing a ‘Get Latest Version’ in VSS was painful to say the least (often taking as much as 40-60 seconds to complete). SVN updates only take that long when I am retrieving the files for the first time. After that it only transmits files from the server that have changed so it is much faster.
  7. Branch without fear - The last time I tried creating a branch in our VSS repository, I gave up after an hour. Subversion does the same operation in about 30 seconds and it doesn’t make the disk size swell because it just creates links to existing tree structures rather than duplicating data like VSS does.
  8. Work remotely without disconnecting - When VPN‘ing into work, I always had to disconnect my solutions from VSS because any interaction slowed my work down to a crawl. SVN not only consumes less bandwidth because it only sends differentials instead of whole files across the wire, but it also stores base copies locally thus allowing you to analyze changes you’ve made without even being connected.
  9. Maintain all of your VSS revision history – I settled on Eric Lane’s enhanced version of VSSMigrate and was able to successfully convert all of our VSS version history to SVN. Eric’s enhanced version of this utility even stripped the VSS binding information from the project and solution files during the migration, thus saving quite a bit of tedious manual work. My only complaint is that it wasn’t able to convert labels over as well, so if anyone knows of another utility that does this please let me know.
  10. Group and commit changes atomically – VSS doesn’t have any way to group changes together and doesn’t allow you to commit a group atomically, which means that you will be in for a world of hurt if your transmission fails halfway through a commit. SVN will not only rollback changes to the repository if any failures occur, but the revision number schema that groups changes together is also really handy for auditing and code review purposes.
  11. BlameMore easily blame co-workers for screwups – The SVN Blame report sure does live up to its name. You aren’t sure who added that scary looking condition to a core function 3 years ago? Run the blame report and you will see a name next to each line of code so you know exactly who to go “talk” to.
  12. Use some pretty slick tools – Since SVN is the defacto standard when it comes to source control, migrating to it will open the door to a number of very slick tools. We use JIRA and Confluence, so I’m particularly excited to try Crucible, another pretty slick tool by Altassian aimed at streamlining the code review process.

There are definitely more cutting edge options out there for source control (like Git), but if you’re coming from a VSS shop and are at all worried about encountering resistance from your coworkers or boss to a migration, then Subversion is probably your best bet. It is an extremely popular, stable, usable, and cost effective option.

Most importantly, it beats the hell out of VSS.

Now, go forth and do your part to rid the world of this horrible VSS scourge.

May God have mercy on your soul…

P.S. If you want to read a rhyming ranting by me about VSS, then check out this post

Popularity: 10% [?]

My Essential Tool List

Posted December 12, 2008 * Comments(5)

My heart skipped a few beats the other day due to that awful hard drive clicking sound that usually precedes a hard disk failure.

So, in honor of the impending machine rebuild that I will likely face soon, I thought I would create my own essential tool list to document my current thoughts on the matter.

I am not so ambitious as to try to follow in the footsteps of Scott Hanselman with his famous ultimate tool list (which I still consult on a regular basis) and create an exhaustive survey of almost every good tool out there. Instead, I opted to only list my current tool of choice for a particular category and task.

I really like experimenting with new tools and am usually content to use any number of tools for a given task. However, I also try to simplify my life as much as possible by only keeping what I think is the best tool for a particular category of tasks installed on my computer at one time.

For example, I used to be a big fan of SlickRun and WinKey to launch my programs. But as soon as I discovered Launchy and AutoHotKey and came to the conclusion that they were better, I uninstalled SlickRun and WinKey lickety split. I still think they are both good tools, but who needs all the clutter…

I definitely welcome any suggestions for improvements to my current lineup.

Development

Code Analysis

Development Lifecycle

Debugging

Utility

MISC

Essential WebSites

Popularity: 16% [?]

KC Day of Dot Net ReSharper Presentation and Resources

Posted December 8, 2008 * Comments(1)

I presented a session on ReSharper at the Kansas City Day of Dot Net conference this last Saturday. Kudos to Lee Brandt for organizing the event, which seemed to be a big success based on the attendance and the number of interesting topics being presented.

A few attendees requested that I post the slide deck, so here it is. It’s in 7-zip format, which only takes a minute to install and beats winzip if you’re not already using it. Most of the session involved me demonstrating the various features of ReSharper through a Test-Driven Development style coding session, so the slide deck won’t be very helpful if you weren’t there or are looking for anything other than a laundry list of features.

If you’ve never seen ReSharper before, you might want to take a few minutes and watch one of these short videos to see the tool in action:

For those of you looking for a way to get started or learn a few new features, I recommend first checking out the features page on JetBrains. It does a pretty good job of providing high level descriptions of all the features along with helpful screenshots that will help you grok the basics.

After that you might want to check out Joe White’s 31 Days of ReSharper. He wrote it back during version 2.5, so some of the features have been enhanced since then but I really like how he dug into the nuances of the various features.

I also wrote a few posts on the topic, including my favorites from about a year ago and my recent efforts to break out of my R# rut. I learned a ton of new things as I was preparing for the presentation, so look forward to a few posts in the near future on this tool as well.

Popularity: 7% [?]

Next Page »