Increasing Processes, Sessions and Transactions in Oracle XE

RSS Feed First time here? you may want to subscribe to my RSS feed. Thanks for visiting. Andrew :o)
Disclaimer: following my technical advice has been known to scratch cars, void lottery tickets and confuse guide dogs – proceed at your own risk..

Oracle Express Edition (XE)

Out of the box, Oracle Datatbase 10g Express Edition RDBMS is fast and powerful. The stated limitations of 2GB of maximum RAM usage and 2GB of total datafile management are plentiful for it to easily run as the back-end for a small to medium-sized office application. However, we soon hit a connection limit as characterised by the following Oracle Errors:

ORA-12516: TNS:listener could not find available handler with matching protocol stack

ORA-00020: maximum number of processes (%s) exceeded

ORA-00020: maximum number of processes (%s) exceeded

We can get this second message because Oracle creates Operating System processes to handle Connections (or Sessions) – which means Processes, Sessions (and as we’ll soon see..) Transactions are all related.

The default values in Oracle XE for these parameters are:

  • Processes =40
  • Sessions = 49
  • Transactions = 53

I was able to generate the above error message (ORA-00020) from about ~30 connections on a vanilla Oracle XE installation (on Windows 7).

So, let’s increase these limits to allow more connections to our Oracle Server..

1. Log in as SYSDBA

From the menu ‘Oracle Database 10g Express Edition’, find and select ‘Run SQL Command Line’, then type:

connect sys as sysdba

and enter your SYS, or SYSTEM password at the prompt

oraclexe-default-processes-sessions-transactions

2. ALTER SYSTEM commands

The Oracle 10g Documentation states that TRANSACTIONS is derived from SESSIONS, which in turn is derived from PROCESSES, thus:

PROCESSES = 40 to Operating System Dependant

SESSIONS = (1.1 * PROCESSES) + 5

TRANSACTIONS = 1.1 * SESSIONS

So, what to start with for PROCESSES?  Trebbling it is as good a start as any, then I’d add a few more for good measure..  Here are the values I recommend:

  • PROCESSES = 150
  • SESSIONS = 300
  • TRANSACTIONS = 330

type the following commands:

alter system set processes = 150 scope = spfile;

alter system set sessions = 300 scope = spfile;

alter system set transactions = 330 scope = spfile;

then to make the settings take effect, we need to bounce the database..

shutdown immediate;

startup;

oraclexe-alter-system-commands

3. Verify the new parameters

with this simple select statement..

select name, value
from v$parameter
where name in ('processes', 'sessions', 'transactions');

oraclexe-updated-processes-sessions-transactions

And we’re done – a free, light and powerful Oracle RDBMS that’s able to serve more connections

On StackOverflow.com.. the founders dilemma

StackOverflow.com

StackOverflow.com is a the programming Question and Answer resource.

It combines elements of a forum where anyone can ask a Question or Answer one, elements of Digg and Reddit where anyone can vote on anything (thereby creating credibility), elements of a blog where anything can be commented on, and elements of a wiki because the Questions, Answers and Comments can be edited to keep it up to date.

Joel Spolsky and Jeff Atwood

Founded by Joel Spolsky (of JoelOnSoftware.com) and Jeff Atwood (of CodingHorror.com), the quality and sheer speed of answers on StackOverflow was due initially to their existing readerships getting involved as early adopters, helping them and each other share their knowledge on day-to-day programming problems.

But therein lies a dilemma..

The readerships of JoelOnSoftware and CodingHorror are also interested in the wider Software Development topics for which Joel and Jeff write about.

The kind of questions that concern code quality, or best practice, or what makes a good developer.

The kind of questions that draw on experience to debate or answer.

The kind of questions that Jeff has asked the community not to ask on StackOverflow.

And I understand his reasons – StackOverflow isn’t really built for discussion or debatable questions because it orders responses by votes instead of time (by default), which makes conversational threads difficult to follow, and also that the ‘correct’ answer will be different for different situations. As Jeff points out, ultimately all of these questions can be answered with the same, simple reply – ‘it depends’ – the synonym for ‘it depends’ on StackOverflow has become the ’subjective’ tag.

The ‘answer’ to the subjective style of question is a combination of the answers – and that combination will be different for each person reading, and that combination could well be different each time it is read with a different perspective. The discussion around the various elements of ‘it depends’ answers the question. And the consideration of the depending elements of any subjective programming question is a huge part of improving as a software developer.

Unfortunately for Jeff, StackOverflow is in many ways a honey-pot for subjective questions. Like mini blog-posts, the poster gets to start the debate with an established, skilled, experienced and knowledgable community. It’s like Joel and Jeff handing them the attention and contributions of their combined blogging traffic – all the better to get everyone considering topics they hadn’t yet thought about.

Subjective questions have a much broader readership appeal than narrow focussed niche questions – more readers means more answers, comments and votes – collectively called ‘activity’. Activity keeps the question on the home-page and visible to more readership potential, which in turn means more readers, and more activity. Broader appeal also means greater rewards for the original poster in terms of Badges – from people simply reading the question, and of Reputation – from people liking the question and voting it up.

But, fundamentally, the JoelOnSoftware and CodingHorror readerships are already predisposed to the public deliberation of wider Software Development topics, and those discussions will take place wherever those readerships congregate. The Badges and Reputation are icing on the real rewards for this group – that consideration and discussion with their programming peers helps everyone improve.

Is StackOverflow the home for those subjective questions and their discussion?  Well, I guess ‘it depends’.

My favourite gadget of 2008 – the Synology DS107

Yep, I know it’s March already :) but back in December I picked up one of these because I was after some Network Attached Storage, and it has impressed me ever since.  This isn’t a review as such, more a highlight of my favourite features and possibly some tips and tricks I’ve found..

Synology DS107e

My main reason for picking up the Synology DS-107+ was to have a large, always on, networked disk for multiple computer backups. It was easy to drop in a Samsung Terrabyte disk (as the box itself comes without a disk), download the latest firmware, and follow the simple setup wizard which formatted the disk and installed itself.

Disk Station Manager 2.0

Having read a few reviews before I decided on it, I wasn’t able to find one that talked about the new ‘Disk Station Manager 2.0‘ which Synology released in March 2008 (according to the release notes) – this is the typical web-browser-based approach to device management we’re used to seeing in ADSL routers, VoIP devices and NAS boxes.

Synology have included everything and the kitchen sink in thiers – it’s all easily findable and configurable, including full ‘root’ access via telnet and ssh (the ‘root’ password is the same as the one you’re asked for when initially setting up the ‘admin’ account – more on this a little later).

First, here’s some stuff I’ve found really useful out of the box:

Apple Time Machine backups

I mainly use a Mac Mini at home, which I had been backing up with Time Machine to a Firewire External Hard Disk. I was pleased to find a pretty straightforward guide for using the Synology instead.

Download Station 2.0 – HTTP, FTP, BitTorrent, NZB and eMule downloads

Being able to set away downloads of fairly large files is pretty handy – I like to check out the latest enhancements to various Linux distributions so I’ve only had need for the HTTP options so far

Web Station

This basically starts an Apache2 webserver, with the option of MySQL too – a LAMP stack in a small, good looking box!  PHP is enabled by default, and Synology host a wiki of user confirmed compatible web applications, such as phpBB, Joomla, Drupal, Piwigo, SugarCRM, oh, and DIY blogging tool called Wordpress ;o)

I’ve got DokuWiki running at the moment as the user comments said MediaWiki (which runs Wikipedia) is a bit slow on the DS-107. Backups are easier with DokuWiki too as it’s file based.

Terminal – Telnet and SSH, getting under the hood

Reading a few blog posts on the earlier DS-106 model, it seems there were patches that granted telnet access. Since then it seems Synology have quite rightly decided to provide simple Telnet and SSH access:

This means I’ve got an (albeit lightweight) Linux server, always on, sitting on the network..

With some trivial instructions, I installed the Itsy Package Management System (which is called ‘bootstrapping’), and gives access to 1,172 applications, tools and libraries!

And after securing SSH and poking a hole to it in my router, I can log into my Synology DS-107 from pretty much anywhere and get access not only to the files, but to the web interface via the magic of SSH Tunneling.

Subversion

Among those applications is Subversion – the open source version control system which was a snap to install, configure and then (using the web Disk Station Manager 2.0 options) configure backups of both the Subversion repository and DokuWiki every hour to an old USB stick I plugged in the back.

The only thing that took me a bit of time to figure out was how to get ’svnserve’ to start automatically. Do check out my comment on Rob’s Synology Subversion post if you’d like to know what I came up with  :o )

In summary

It’s a fantastic peice of kit – full of features, well made, small, looks good, has a great Ajax-y web interface, and with a little tweaking there’s enough of a Linux server in there to run pretty much anything.

Well done Synology  :o )

Why are software developers so bad at estimating time?

In order to answer this question, we need to understand the term “flow”. From Peopleware:

During single-minded work time, people are ideally in a state that psychologists call flow. Flow is a condition of deep, nearly meditative involvement. In this state, there is a gentle sense of euphoria, and one is largely unaware of the passage of time: “I began to work. I looked up, and three hours had passed.”  There is no consciousness of effort; the work just seems to, well, flow. You’ve been in this state often, so we don’t have to describe it to you.

Not all work roles require that you attain a state of flow in order to be productive, but for anyone involved in engineering, design, development, writing, or like tasks, flow is a must. These are high-momentum tasks. It’s only when you’re in flow that the work goes well.

To paraphrase, “It’s only when you’re in flow that you’re at your most productive”.

The quotation is taken from chapter 10, page 65. I know that because after reading those short paragraphs, I realised something that made me stop and put the book down..

When a software developer thinks up an estimated time to complete a development task – they’re thinking solely in “flow” time.

That realisation is profound.

Sure, more experienced software developers will then add time to this based on their experience before giving their answer, but their first thought is based on flow-time, which is their best-case, uninterrupted development time.

Why is this?

I think it’s a combination of 2 things;

  1. The question itself.. “how long would it take you to do / fix / add / change x?”.
    A developers starting point is their experience of development to date, and that development is at it’s most productive in flow-time. So they’re going to answer you based on them being in productive flow-time. However, this means they’re starting their estimate based on an unaware assumption that they’re quoting flow-time.
    For example, if a developer responds with “it’ll take me half a day”, what they’re really saying is “with a half-day of flow-time, I can do / fix /add / change x”.
  2. If you didn’t already know, developers are optimistic. They’re optimistic about how much they can get done, and they’re optimistic about how much flow-time they’re going to have.

So what is the answer?!

To quote more from Peopleware:

Chances are, your company’s present time accounting system is based on a conventional model. It assumes that work accomplished is proportional to the number of hours put in. When workers fill out their time sheets in this scheme, they make no distinction between doing meaningful work and hours of pure frustration. So they’re reporting body time rather than brain time.

The phenomena of flow and immersion give us a more realistic way to model how time is applied to a development task. What matters is not the amount of time you’re present, but the amount of time that you’re working at full potential. An hour in flow really accomplishes something, but ten six-minute work periods sandwiched between eleven interruptions won’t accomplish anything.

The answer, then, is to take the developers estimate with a pinch of salt (the more ’seasoned’ the developer, the less salt required), remember that their estimate is an estimate, and use it with evidence based scheduling (if you aren’t already).

The next question you may be asking of course, is how do I reduce the difference between the estimate and the actual – the ’slip’.

That’s a topic for another day. I’ll let you know when I’ve finished Peopleware, or you can grab a copy for yourself  :o )

Automatic SVN Revision Numbering in ASP.Net MVC

While working on an internal ASP.Net MVC project at work, I wondered if it was possible to get the SubVersion (SVN) repository number to be automatically updated on the webpages I was creating every time I did a build or check-in.

The footer at the bottom of the StackOverflow.com site is what we’re trying to do here (though I suspect after listening to their podcasts that they’ve got theirs updated using custom build tasks in CruiseControl.Net)..

StackOverflow.com - SVN revision

After a couple of hours of investigation, trial and error, here’s one way to do it.

Before we begin, we’ll need TortoiseSVN installed, and a local working copy of your ASP.Net MVC project checked out.

There are essentially 3 steps..

1. Create a Html Helper

This greatly simplifies the return of the SVN number as a string which is then trivial to place in our View Page or footer.

Create the following file:

'/Helpers/HtmlHelpers.vb
Imports System.Runtime.CompilerServices

Public Module HtmlHelpers

     _
    Public Function SVNRevision(ByVal helper As HtmlHelper) As String
        Dim svnFile As IO.StreamReader
        Dim svnRev As String = String.Empty
        Try
            svnFile = New IO.StreamReader("svn_rev.txt")
            svnRev = svnFile.ReadLine()
            svnRev = svnRev.Replace("""", "")
            svnFile.Close()
        Catch FnFex As IO.FileNotFoundException
            'swallow, but write out the file and location we tried to read..
            Trace.WriteLine("HtmlHelper.SVNRevision: Ex: [" & FnFex.Message & "]")
        Finally
            If svnRev.Length = 0 Then svnRev = "-"
        End Try

        Return svnRev
    End Function
End Module
'

2. Create a Batch File and add it as a custom build task to the project

Next we create a Batch File (.bat) in the root of our project, which will call the TortoiseSVN command to retrieve the SVN revision number from our working copy.
We’ll call it “CreateSvnRevFile.bat”. Here’s the contents:

@echo off
rem # CreateSvnRevFile.bat
rem # Check the Working Copy folder is passed in..
if /I [%1%]==[] goto usage else goto start
:start
rem # (Re)Create the base file so SubWCRev has something to replace
echo "$WCREV$" > svn_rev.txt
rem # Ask SubWCRev to replace with the latest revision
rem #  from the current (working copy) directory
C:\Progra~1\TortoiseSVN\bin\SubWCRev.exe %1 svn_rev.txt svn_rev.txt
rem # Copy the file for local hosting and testing purposes..
copy svn_rev.txt "C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE" /Y
del svn_rev.txt /Q
goto end

:usage
echo Usage: %0 SvnWorkingCopyDir

:end
rem exit 0

Then we need to add a custom Build Event action to call the Batch File. (The Build Events dialog is in “My Project > Compile > Build Events”)

My Project - Compile - Build Events

As shown above, we need to add the line to the “Post-build event command line:” box..

call "$(ProjectDir)CreateSvnRevFile.bat" "$(SolutionDir)\"

3. Add the SVN revision number to our Page

With all of that background plumbing in place, all we need to do now is add a couple of lines of code in the appropriate .aspx file wherever we want our SVN revision number to appear:

...
<%@ import Namespace="MyMvcProject" %>
...
<%=Html.SVNRevision()%>
...

the first line imports our Project Namespace (which will obviously depend on your project – you may need to rebuild at this point before the =Html. autocomplete picks up the ‘SVNRevision()’ HtmlHelper).

And we’re done:

So, how does it work?

The Post-build task CreateSvnRevFile.bat does most of the work by creating a temporary text file called svn_rev.txt containing the string “$WCREV$”, and then calls the TortoiseSVN command SubWCRev.exe on the text file to replace the string with the working copy revision number.  It’s worth noting that the revision number is held in the working copy .svn folder so this method doesn’t require a call to your SVN server to work.

The Html Helper code then simply tries to read the updated text file and returns the contents as a string, or it returns a dash (“-”) if there was a problem.

Obviously, if we’re using this on a high-traffic website or frequently referenced page then we’d want to cache the string rather than reading it from the file every time. We could do this by modifying the Html Helper to read the file once into a variable, which we could do by wrapping the actual file reading up in a singleton object, for example.

I hope you find it useful :o )




Andrew’s Toblerone Cheesecake

Toblerone Cheesecake

I don’t have a sweet tooth as such, but I have discovered a talent for making desserts – that or my friends are very complimentary  ;o)

I think I’d heard of The Cheesecake Factory before my first trip to the States, so when our hotel taxi took us past the one on Camelback Road in Phoenix we had to double back after checking in. I can testify to that first comment on Yelp.com – the “Portions are ginormous” !

A couple of months later, I decided to make a cheesecake to take into work for my Birthday, and although The Cheesecake Factory had pretty much every flavour, they didn’t have a Toblerone one, so, inspired by their “Chocolate Chip Cookie-Dough Cheesecake” – I’d not seen ‘floating’ pieces of yumminess in the cheesecake bit before – I found a basic cheesecake recipe and experimented.

This is the latest version that was a huge success at my most recent Birthday celebrations.  Those mathematically minded of you will note the recipe uses only 2/3rds of the Toblerone..

For the Base
1 Packet of Hob Nobs or Digestives
1 Packet of Oreos (optional)
1/3 Block of Butter

For the Cheesecake
400g Bar of Toblerone
450g of Cream Cheese (Philadelphia)
250g Caster Sugar
250g Crème Fraîche
475ml Double Cream

For the Topping
200ml of Double Cream

Serves: 8-10

Method
Create the base first

  1. Break-up the Hobs Nobs or Digestives and optional Oreos into crumbs, I do this by emptying them all into a Sandwich Bag (or Ziploc), and hitting them repeatedly with a large Branston Pickle jar which I can also employ as a makeshift rolling pin.
  2. Make sure there aren’t any large crumbs left, then melt the butter in a medium sized pan over a low heat and stir all of the mashed-up biscuit crumbs into the butter.  The crumbs will easily soak up the butter, become a little darker in colour and just start to stick together. Save the sandwich bag..
  3. Take a 23cm (8″) springform tin, and cut a piece of grease-proof paper for the bottom (I lightly dab the tray with butter in a couple of places to hold the paper in place)
  4. Turn out the buttery-crumb mixture into the tin, even out, then using something with a flat surface (such as a church candle wrapped in clingfilm) pat the mixture down evenly to compress it
  5. Place in the fridge to set while you continue with the main cheesecake part..

Create the cheesecake

  1. Break off about a 3rd of the Toblerone peaks and place them in the same Sandwich bag (give it a good shake out first!), and with the same Branston Pickle jar mash the Toblerone into smallish chunks, roughly 1-1.5cm in size
  2. Place all of the other ingredients into a large bowl, and electric whisk together until smooth
  3. Stir in the battered Toblerone pieces to the mixture, then empty the lot onto the base in your springform tin, pushing the mixture into the edges and getting the top as smooth and level as you can
  4. Put it back in the fridge to set, ideally overnight

Create the topping

  1. Break off about a 3rd of the Toblerone, and melt it in a small bowl over a pan of simmering water.
  2. Add the double cream a bit at a time, stirring it all in before adding more, until it’s all in there.  It will get quite runny but that’s fine
  3. Retrieve your cheesecake from the fridge and slowly pour the melted Toblerone topping onto it. Guide the topping so it all gets covered then place it back in the fridge to turn sticky
  4. It should be ready in about an hour or so.

Best served chilled with squirty cream  ;o)

Variations
To make the base a bit more interesting you can use broken up bits of Cinder Toffee instead of Oreos (which you can make, though I bought!). You can also add extra topping layers – I made a Toblerone Cheesecake with a layer of melted marshmallow under the melted Toblerone topping once which worked very well too

Reply to Iain Arthurs – Software Development Course Content

Recently, our little software company was contacted by a local college, to provide feedback on the content of a couple of new software engineering and advanced programming courses they are looking to run.

I saw this is my first opportunity to offer my assistance to those asking how should we teach software engineering?

Hi Iain, thank you for your reminder – please accept my apologies for not getting back to you sooner!

I’ve read the 2 course outlines you sent and I think you’ve got a good focus for students wishing to go on to develop software development roles. I like the outline of the courses very much – especially your choice of .Net as we’re a .Net-based development company ourselves.

If I may, I would like to suggest a few areas that I found lacking in my own formal education that I think, should a graduate make note of on their C.V. as having had experience of on your courses, would put them on top of the pile based on my experience of hiring developers at OnePoint Systems. These are, namely:

An introduction to source control
this could be as little as a single session describing what it is, why it’s essential in software development, release and maintenance lifecycles – without a doubt in team-based development, but it can also offer huge benefits for individuation development projects. I think it would be ideal to get the students to have hands-one experience with it – say as part of a team project, or as a requirement for handing in software for marking that it is checked in to some source control that the teacher would then check-out and build so as to mark

An overview of Agile Methodologies
I see you have R.A.D. and Prototyping listed which is excellent (as is understanding the Waterfall Lifecycle), but I’ve found in my professional development that Agile – specifically Test-Driven Development (TDD) is fast becoming commonplace in software development companies – to quote Steve McConnell (author of Code Complete): “within the next few years we won’t talk about Agile much anymore; we’ll just talk about programming”. I would not wish to cover this topic in any more detail as I assume you are already familiar with it

A lesson on 3rd party tools and controls
Something that I found completely missing in my formal education was (even the notion of) the practice of using 3rd party tools and controls within my own software projects. While tools could be covered by introducing source-control and add-ins for development IDE’s (such as Microsoft’s SourceSafe or Team System), or debugging tools such as Lutz Roeder’s .Net Reflector, the integration of 3rd party controls for their re-use value alone is, in my opinion, worth a session on its own – even if the students don’t get to actually use 3rd part controls. I’m thinking of listing a few resources such as ComponentSource.com, telerik.com, infragistics.com or janussys.com – to open the students mind to the idea that developing everything in-house is rarely the best way to proceed.

Reporting
for most application development a little hands-on experience about how software reports are put together is, again, in my opinion essential. The reporting engine, be it Microsoft Reporting Services, Crystal Reports or a 3rd party reporting add-in such as Active Reports for .Net is of little differentiating factor for the simple kind of reports that allow the students to get the data out of their software creations and onto a physical piece of paper. In many ways, I think report creation is the final, completing step in creating a software application – it’s the tangible output of an intangible product.

I thank you very much for this opportunity to offer my feedback, and I wish you and your students the very best. If I may be of any further assistance, please get in touch and I shall endeavour to be a little more responsive with my reply.

Kindest regards,

Andrew

Andrew Freemantle

Valuing criticism, encouragement and feedback

Make a distinction between criticism, feedback, and encouragement, and ask for the one you really want.

  • Criticism is filled with judgements based on right and wrong, good and evil.
  • Encouragement is the intentional act of supporting someone in pursuing their objectives, regardless of how it’s going at the moment.
  • Feedback is open, honest communication about how we or our actions are being perceived by other people

- quoted from Michael Neill’s book, “You can have what you want”

Any form of communication about our software or services from our development co-workers, testing team members, quality assurance, management and especially our end users is an opportunity for us to learn and improve – to learn and improve our software, the domain or business area it works in, and our relationship with the person we’re communicating with.

Of the 3 options Michael Neill presents, I think everyone prefers receiving Encouragement and Feedback rather than Criticism, but that’s more to do with favouring a pleasant delivery – especially considering that the message itself comprises only 7% of the communication. Wouldn’t you rather receive the same message in a nicer way?

And the chances are that if you would, then it’s highly likely that everyone else would too. Therefore, wouldn’t it make sense to phrase your criticism as encouragement or feedback? and encourage feedback from those you communicate with?

How far should your unit tests go?

I don’t think it would be appropriate to talk about Unit Testing without first starting with the bigger picture of Agile Software Engineering Methodologies.

I started working to the Agile methodologies, as described by Robert C. Martin about 18 months ago. It was hard not to see it as a ‘magic bullet’ that enables the production of quality software because it just made so much sense to me. Here’s Steve McConnell’s view of Agile development:

Q4: What is the future of Agile?
Agile has largely become a synonym for “modern software practices that work,” so I think the future of Agile with a capital “A” is the same as the past of Object Oriented or Structured. We rarely talk about Object Oriented programming anymore; it’s just programming. Similarly, I think Agile has worked its way into the mainstream such that within the next few years we won’t talk about Agile much anymore; we’ll just talk about programming, and it will be assumed that everyone means Agile whenever that’s appropriate.

I have emphasised Steve’s sentence that beautifully sums up Agile for me – ‘Agile’ is my definition of programming.

Changing my way of thinking from initially ‘where do I start coding the solution?’ to ‘how am I going to TEST the solution?’ was definitely challenging, but I still remember sitting down to start actually writing test code and wondering ‘how far should my unit tests go?’ – meaning, if I’m testing that an Object can persist, or save its state to a database, how do I test that it’s working?

My current project incorporates the excellent Rockford Lhotka’s CSLA.Net framework, and my first attempts at testing for persistence did so by reading the database tables to verify the data written – in the same unit test..

Unit Testing - first attempts

In practice, these unit test cases were well over a screen full of code which, when testing different scenario’s around the edit and save logic, meant I’d created an overhead that added a quite considerable amount of time to their maintenance when features were added and changes were inevitably made to the application code being tested.

I was already thinking about separating out the unit test code into it’s component parts to make handling change less painful, when I saw the title of Carl and Richard’s audio talk show “Dot Net Rocks!” #312 – “Andy Leonard on Unit Testing your Database”. Being a Database guy, Andy’s main reason for unit testing the Database is that we developers sometimes forget that we write code there, and that our Database code is as much a part of the application – but I think his rationale also solves the maintenance issue with my single, large unit tests..

Unit Testing - separating out the database layer

By separating out the unit tests to check the Database layer, it massively simplifies the readability and maintenance of the application layer unit tests. Moreover, if you locate the Database layer unit tests in their own library, then they can be executed before the application layer unit tests so they can halt further testing as it’s highly likely that failure at the lower database level will manifest itself higher up. This will ultimately save us time locating and fixing failing code, as well as saving time modifying existing unit tests when changes occur.

Writing useful comments

It was drummed into me throughout my software engineering training years – the essential internal documentation of software – the comment.

After a few years actually engineering software, I’ve found that I can categorise most comments into 4 basic types:

  1. The obsolete code block, often with an accompanying note about why it was removed or replaced
  2. The overly-verbose or overly-descriptive, highlighting the obvious
  3. The reference; a Bug tracking reference number or URL link to where the block of code originated
  4. The one or two-liner descriptive, concise explanation

And having done so, I have found that the first 3 are actually detrimental to the process of maintaining or otherwise changing software. Let’s look at them in closer detail to find out why..

1. If we’re using any decent form of source code version control, then there is no benefit whatsoever to leaving commented out, obsolete code blocks in the software – in fact, I’d argue that when following a thread of logic through function to function, having to skip over comments that look like code is actually detrimental to understanding why the code does what it does. Consider that source control makes it a trivial task to compare or replace the current code with any previous checked-in version of the code, and there simply isn’t a sensible reason to leave old code in place.

2. I take this to be an indication that there is no accompanying documentation, or it\’s going to be sparse at best. If it\’s a block comment about why a particular feature exists then a better place for it is in some form of requirements documentation; if it’s a block comment about a design decision then it would be better placed in the design documentation. Any other documentation format easily wins over plain-text descriptions, and designs are better illustrated, well, with illustrations.

3. I’ve worked out that both of the above are bad, yet I still catch myself thinking that it’s somehow beneficial to the maintenance of the code if I include a Bug tracking reference number next to the code I’ve changed. If I’m coming back to the code because of a problem with the fix I’ve made, then it’s much easier to search through the source code version control check-in comments for the Bug reference number than through the source code itself – if only because it’ll be a hell of a lot faster. If I’m in the code for another reason – say I’m adding something new – finding a Bug tracking number isn’t going to make me head off to the Bug System and look up it’s history.. I know what the code is doing because I’m in there reading it and stepping through it. URLs serve as a similar distraction from the actual workings of the software. That said, I think URLs are useful in referencing the origin of an entire Class, algorithm or Library.

4. The most helpful comments I’ve found are the concise, single-line description of what’s going on that help keep the reader on track – such as what we’re looping through or why we’re looping when we’re 2 or 3 Arrays deep. Like breadcrumbs, they assist you and your maintainer in un-ravelling the mystery of your software.

I’ve found that putting as much thought into the content of my comments, as I do into the content of my code, makes it much easier to maintain my software.