Caffeinated Coder

I’ve been working a lot with Castle’s Active Record and Ruby on Rails in the last month and as a result have written significantly fewer basic CRUD operations and database access code. It’s been an addictive experience and has caused me to rethink the proper role of hand-written database code (sprocs) within an application.

Although I feel perfectly comfortable in a set-based world writing SQL, it has traditionally been one of my least favorite areas of coding. Besides being relatively repetitive and tedious, at least when it comes to basic CRUD operations, sprocs are much more difficult to handle when it comes to source control, versioning, debugging, and unit testing.

For example, at my last job we were tasked by auditors to come up with a build and deployment process that included version traceability and rollback capabilities. It was pretty easy to put together an acceptable solution for assemblies, since automatically versioning dll’s in a trusted way is simple via the AssemblyInfo and rolling them back is trivial since everything is contained in a manageable number of dlls that simply need to be copied from one folder to the next. When it came to sprocs, however, the best we could offer auditors was some hackery around adding comments about the version at the top of each sproc definition file along with a big disclaimer that there was no guarantee that files were not modified by DBA’s along the way.

At my current job, database code causes us even more trouble because the database is larger and contains more sensitive data (thus making restores more difficult), there is a heavier reliance on shared code, and only the deltas of database code are currently under source control. It feels like I am constantly tracking down some ghost bug that is caused by my local database schema somehow being out of sync with the codebase.

Despite these misgivings, I’ve dutifully followed the traditional Microsoft recommended best practice of funneling all data access through sprocs up until recently because sprocs were supposedly faster, more secure, and provided a beneficial layer of abstraction.

Although I have heard several arguments against sprocs in the last several years, I recently embarked on a more thorough investigation while trying to convince my team to switch over from using an in-house code generation\sproc-based solution for data access to using NHibernate\ActiveRecord. Here are a few resources I found that present good counter arguments against the conventional sproc wisdom.

1. Stored procedures are bad, m’kay?: This classic post that was written by Frans Bouma of LLBLGen fame in 2003 spawned quite the flame war on the topic. Most notably, Frans counters the “Sprocs are faster because they are compiled” argument by quoting passages from the SQL Server BOL documentation that clearly suggest otherwise. He also counters security arguments by pointing out that parameterized queries prevent SQL injection just as much as sprocs and that assigning permissions to views and roles provide just as much protection as assigning execute rights on sprocs.
2. Who Needs Stored Procedures, Anyways?: This is also an older post from Jeff Atwood where he nicely summarizes the negative aspects of SQL when compared with a traditional coding languages and came up with the quotable phrase “Stored Procedures should be considered database assembly language: for use in only the most performance critical situations”. I definitely think that some developers are reluctant to embrace ORM’s for the same reasons that many old C++ programmers scoffed at the idea of letting the CLR garbage collector manage memory for them instead of manually doing it themselves with raw pointers.
3. Why I do not use Stored Procedures: Jeremy Miller dismisses performance arguments by declaring them instances of premature optimization and elaborates on all the problems caused by sprocs when it comes to maintainability, testability, and architecture. He also points out that the touted benefit of allowing DBA’s to make changes is actually a dangerous practice since it represents a breaking API change from the application’s point of view and thus should go through thorough regression testing before any DBA should be allowed to make changes.
4. Foundations of Programming - Part 6 - NHibernate: Besides offering a nice introduction to NHibernate, Karl Sequine provides a nice summary of the historical sproc debate, including a counter point for the increased network traffic argument, which he says is a moot point since most traffic occurs between a app and database servers sitting on the same internal GigE networks where bandwidth is fast, plentiful, and free.
5. DotNetRocks ORM Smackdown: For a more balanced debate, listen to this podcast episode (or download the transcripts) where Oren Eini and Ted Neward face off over the value of ORM’s in the software industry. Be sure to check out the commentary on the episode in the comment section of this Ayende post as well as the rebuttal in this post by Ted Neward.

In my opinion, one of the strongest denunciations of traditional sproc dogma comes Redmond itself, which seems to be straying from its original sproc recommendations in favor of a more more dynamic SQL generation world-view with its recent release of LINQ to SQL and the Entity Framework.

If you follow the open source world or program in some language other than .NET, then you’re bound to feel a little smug right now because ORM’s have been around for a long time. In fact, I have a vivid memory from 6 years ago of a co-worker who was fresh from the Java world being dumb-founded that Microsoft didn’t have any ORM solution. He was used to using HIbernate and the thought of manually mapping database tables to domain objects was hard for him to grasp. Even in the .NET open source world, I’ve been reading blog posts that sing the praises of NHibernate and IBatis.NET, two popular .NET ORM ports, for several years.

On one hand, Microsoft’s entrance into the fray is good news for ORM enthusiasts since it means that a larger audience of developers will begin to see the technology as legitimate. On the other hand, Microsoft clearly has some catching up to do in this space, so you might want to think twice about starting off with Microsoft’s offering rather than one of the more proven open source or third party alternatives.

If you are a .NET developer and new to ORM’s, then I recommend starting out with Castle’s Active Record, which you can learn in less than an hour by reading this Getting Started with Active Record tutorial. My co-worker’s were reluctant to try NHibernate because of the perceived learning curve and the plethora of mapping files required, but they quickly agreed to use Active Record after only a short demo.

If you are a POCO purists, which means that you want to keep your domain objects free of any non-business related concerns (such as persistance), then you’ll want to follow the repository pattern using the ActiveRecordMediator class rather than inheriting from ActiveRecordBase like the tutorial shows. Some of the more experienced ORM users seem to see ActiveRecord as more of a gateway drug to NHibernate and ultimately prefer to forgo the conveniences offered by the ActiveRecord layer in favor of the increased flexibility and loosely coupled design offered by dealing directly with NHibernate instead.

Regardless of the approach taken, I definitely no longer believe that sprocs should play any significant role in any application. The current mandate in the software industry is to strive to lower costs by increasing developer productivity and ORM’s clearly help to do this by eliminating the need to write and maintain countless simple CRUD sprocs.

It’s definitely time for all of us .NET developers to abandon our convention sproc wisdom and start playing catch-up with the rest of the industry when it comes to using ORM’s.

In my last post I mentioned that I had recently decided that it was time to overhaul my approach to managing tasks. The changes I implemented were mostly inspired by a book by David Allen called Getting Things Done and I have been extremely pleased with the results so far. As promised, here are some of the concrete ideas that I’ve adopted and incorporated over the last month.

1. Following a Workflow: I found that my procrastination was mostly fueled by not knowing where to start on something or else feeling that it was too big for the small time slots that I usually had available during the day. Following this workflow whenever I a faced with a new item in my inbox gives me an automatic starting place and a way to boil down a massive, disorganized, tangle of items into a relatively small set of very specific tasks that I need to do.
2. Treating Collecting, Processing, and Doing as Different Activities - The workflow diagram really only covers the processing part of the equation, which is where you make decisions about things. Two other distinctly different activities are collecting and doing. Collecting simply means recording items that are floating around in your head and placing them into some kind of trusted inbox so that you can process them later (I use rememberTheMilk.com, a Samsung smartPhone which syncs with rememberTheMilk.com, my email inbox, a physical inbox on my desk, and delicious). The idea is that each process requires a separate type of thinking, thus context switches detrimental in terms of productivity and effectiveness. I find that I also gain the most benefit in terms of peace of mind by completing the collecting and processing stages before I attempt to actually do anything (with the exception of the 2 minute rule below).
3. Transitioning from To Do’s to Next Actions - My TO DO’s used to be very vague, like ‘Get Supplemental Life Insurance’. Now I take the effort to formulate the exact action I need to take and then attach any relevant information that I will need, such as ‘Call Mark (xxx-xxxx) and ask for quote on 20 year, $xx term life policy’. Since all the preliminary decisions and information gathering have been done, I am finding that I am much more likely to actually do it when I have a free minute during the day.
4. The 2 Minute Rule - If I grab a task from my inbox that will take less than two minutes, I just do it. This keeps my list short. If it will take longer than two minutes, I force myself to put it in one of the ‘deferred’ piles (Next Action Lists) until I am finished processing so I don’t get distracted and run out of time before the processing is done.
5. Grouping Tasks according to Contexts - Another helpful hint that I picked up from the ‘Getting Things Done’ book is to organize my Next Actions according to the context required to do them (call, errand, home, work, computer, etc.). Now, whenever I am out running errands, I scan my ‘Errand’ list to see if there is any other errand I can tackle that is close to where I’ll be. I’ve been surprised at how much more I get done now that I’ve picked up the habit of scanning the appropriate context list based on where I am and what I am doing whenever.
6. WaitFor List - This is a very useful list where I now track everything that I am waiting for (the phone company to send me a rebate, the insurance guy to send me a quote, my wife to make an appointment, etc.). I set due dates on them so that I know when it is time to start bugging somebody if things aren’t getting done.
7. Someday List - By explicitly separating all the things that I am explicitly agreeing to do (my Next Action and Project lists) from the things that are interesting to me but not appropriate to tackle at the current time, I dramatically decrease my tasks list and don’t feel nearly as guilty for those things that I’m not choosing to do at the moment.
8. Reference Materials - I realized that a lot of items that I thought were TO DO items were simply things that I needed to keep as reference in a system that would allow me to quickly retrieve it when and if I should ever need it (that meant overhauling my existing black-hole-of-a-filing system). To have a good filing system, you really need to take an agile approach and constantly tweak it as new items come in and old ones become obsolete. The best way to do this is to buy an electronic label maker ($30) so that you can quickly relabel folders in a way that is neat and easy to read.
9. Delegating and Deleting - These were two activities that I was not nearly aggressive enough with over the years. By having these questions as part of the workflow, I find that I am more likely to weed items out by throwing them away or letting someone else do them instead who is better suited to the task in the first place.
10. Using Projects to Organize Multi-Step Actions - I found that the items I was most likely to procrastinate on were the things that had multiple steps involved but were too small to traditionally be considered a project. David Allen recommends considering these to be a project, which he defines simply as a place holder that is useful for periodically generating Next Action items and organizing supporting material. This is great for smaller projects because it naturally lends itself to breaking things down into more manageable chunks. He also has some great suggestions for larger projects in terms of defining goals, envisioning outcomes, and brainstorming possible tasks.

If you’re interested in learning more about these concepts, I definitely recommend buying the book (which is a quick read) or visiting some of the resources here.

I’ve always been somewhat prone to procrastination, especially in my personal life, but I’ve watched with growing trepidation as my task list has slowly grown out of control over the last several months.

Part of this trend is due to being a new parent. I expected that having a newborn would consume the majority of my free time, although I must admit that it didn’t completely sink in until it happened. What I didn’t expect was that my “honey-do” and “male provider” lists would also grow dramatically as my both mine and my wife’s biological instincts kicked into high gear. Meanwhile, my list of new tools and technologies to explore continue to mount as suggestions pour in from twitter, my blog, and my RSS reader.

The result is that I have many times more tasks to do than before but now I only have a faction of the time to get them done. This new reality has left me feeling overwhelmed, paralyzed, and generally less productive with the time I do have.

A few weeks ago I finally came to the conclusion that I needed to severely overhaul my approach to managing my tasks if I ever had any hope of feeling calm and productive again. Someone on Twitter suggested I pick up a copy of David Allen’s Getting Things Done book, so I started there. I’ve always had an aversion to business self-help books, but desperate times called for desperate measures so I picked up a copy in the bookstore.

I was surprised by how quickly my skepticism disappeared. I was especially sold on the author’s clear explanation of the negative consequences of relying primarily on our brain (mental RAM) to juggle all of our TO DO’s. Not only is this common approach tremendously inefficient since our brain randomly reminds us of things we need to do repeatedly at times when we can’t usually do anything about it, but it also perpetually consumes a limited resource (especially in my case) thus slowing down our overall mental functioning. In other words, it is like trying to get something done on your computer while running a dozen instances of Visual Studio 2008. You can do it, but it just isn’t what I would call an optimal experience.

The premise of the book is that it is possible is overcome this dynamic and achieve a calmer and more productive state of mind by simply transferring all of your tasks from your head to a trusted external system. In GeekSpeak, it would be the equivalent of flushing all those nagging thoughts from RAM to disk, which makes sense since you really only need them in memory when you are actually in a position to do something about them.

This analogy, along with the remarkably clear and helpful diagram below, convinced me to make a several week investment in optimizing my approach to getting things done.

The results have been dramatic. Not only do I feel like a huge psychic weight has been lifted from my shoulders, but I’ve completed a huge list of tasks that I had been procrastinating on for months and in some cases years. I also find myself fitting tasks into much smaller chunks of time during the day because I have organized them in such a way that they take much less mental energy to tackle, thus I no longer feel that I need to set aside a huge block of time to do them.

Even more surprising is that I find that doing things on my “Next Action” list is slightly addictive. Crossing off an item feels like lightening the load by removing something from a heavy backpack I have to carry around with me everywhere I go. There is an immediate relief in it.

If any of this sounds familiar to you, then I would recommend picking up a copy of David Allen’s book. It’s a very quick read.

For those of you that are intrigued but not sold enough to actually buy the book, stay tuned for my next post where I’m going to share all of the concrete tips that I’ve edited out of this post so as to keep it a reasonable length for a change.