Thinking Of “Refactoring”? Not Got Any Unit Tests In Place? Bad Coder!

Refactoring is the process of changing a software system in such a way that it does not alter the external behaviour of the code yet improves its internal structure.

Martin Fowler

I believe that sometimes code needs to be refactored. For example, an improvement in the internal structure of a software system is sometimes necessary in order to improve its extensibility. All too often however refactoring is taken to mean ripping out the guts of the system and starting again. It is a common problem that programmers will try to make the move to cleaner code by throwing out the existing body of work and beginning from scratch.

To be effective refactoring needs to be understood for what it really is. There are a couple of keys to effective use of refactoring in a project:

  • Refactor in small iterations. After each iteration the external behaviour should be the same.
  • Check that the external behaviour remains the same! This is where you run the test suite. You do have one don’t you?
  • Modifying or adding functionality is not refactoring! They are separate steps, keep them separate (using small refactoring steps reduces the temptation to merge these different processes)

Without rigorous testing of refactored code can you be sure that the code is functionally equivalent? If you are “refactoring” code and not testing it then you are potentially introducing bugs for no external benefit! I am reminded of a quote from Lou Montulli:

I laughed heartily as I got questions from one of my former employees about FTP code the he was rewriting. It had taken 3 years of tuning to get code that could read the 60 different types of FTP servers, those 5000 lines of code may have looked ugly, but at least they worked.

You cannot refactor code if you do not understand what its external behaviour is. When you perform refactoring in small steps with frequent testing then you will begin to get the benefits from it. Two key insights to refactoring are:

  1. It’s easier to rearrange the code correctly if you don’t simultaneously try to change its functionality.
  2. It’s easier to change functionality when you have clean (refactored) code.

I cover this subject today because of a discussion I was involved in concerning the CSS property visibility and its value collapse. In investigating the situation with respect to the rendering of these properties I discovered that it was rendered incorrectly in a variety of browsers. Investigating whether this had been brought up in bug tracking system of the mozilla project I came across bug number 77019. The problem had been identified, reading the comments on the bug I came across this beauty of a comment:

Oh Noooooo, we had the collapse code working in the tree for years and it was simply optimized away. Hmm may be optimized is not the correct word for it.

And here lies the moral of this tale, refactoring without understanding the existing code or without rigorous testing equals the introduction of both regressions and new errors into the software system.

Recommended Reading

Independents Day To Relaunch

This is just a quick pointer to Independents Day. It appears there is going to be a relaunch of independents day.

WHY ID?

WE HAVE NO GRIPE with for-profit sites, and many of us contribute to such businesses. But we also believe that the web is a remarkable medium for new forms of art, personal storytelling, and all manner of information and services whose rewards are not necessarily financial.

The independent content scene is alive and well, but is largely unknown by the general web-using public.

We seek to support each other as a community, and to increase, if possible, the general public’s awareness not only of existing independent sites, but of the fact that they can create their own.

I joined Independents Day “back in the day”. Independent content has become far more well known among the web community due to the adoption of blogging and similar activities. I am interested to see how independents day proposes to support independent content creation and promote participation among the general public.

Mozilla Foundation Setup as AOL Dumps Netscape

AOL has unceremoniously dumped Netscape and its accompanying open source browser suite, Mozilla. Netscape (the commercial version of the Mozilla suite) is dead, Mozilla still has a future; an independent non-profit organisation called the Mozilla Foundation has been setup with support from Sun, IBM and AOL (so AOL haven’t completely cut off Mozilla then).

The announcement concerning the formation of the Mozilla Foundation lists some of the contributions that have been made to help keep Mozilla running, AOL for instance has pledged $2 million to the Mozilla Foundation, spaced over the next two years, as well as equipment and domain names to help launch the new venture.

Death of Netscape, Rebirth of Mozilla?

Mozilla is now under the auspices of the Mozilla Foundation, a non-profit organisation. What does this mean? Interested companies can donate money essentially for free; the sums donated can be offset against their taxes. In this way the cost of supporting Mozilla will not be a significant burden to the corporate supporters listed previously. Mozilla is no longer tied to doing things the “Netscape” way, the monolithic applications that have blighted Netscape and Mozilla can now be done away with as Firebird and Thunderbird begin to take hold.

My first impressions are that this could be a good move for Mozilla in the long term. The crucial element is that the transition goes smoothly. There are encouraging signs already such as the very nicely redesigned front page of mozilla.org which gives much more emphasis to the products, is cleaner looking and really showcases the mozilla software products well. I hope that the momentum people put into the project during this transitional period will continue and take the browser from strength to strength in the future.

Site Accessibility Action Taken By RNIB

The RNIB has begun the first UK action on site accessibility. Action along these lines has been predicted by others. This is an important step by the RNIB in the promotion of accessible web design. By raising the possibility of legal action against web site, web designers now have a stick to match the carrot of creating an accessible web site.

The accessible web will not become the default for web sites without a few changes:

  1. Education of web professionals
  2. Financial motivation for site owners

From an educational perspective there are many sites out there that help to teach the “how” of accessible web design. Here are a couple to get started:

Learning new techniques is something that all web designers should be familiar with, after all how many “quality” professional web developers started out in web design? Education is something that should be undertaken by anyone who wants to make a professional career for themselves in any field, so I’m not going to talk further on that matter here.

The Big Issue (as seen from the financial trenches)

What’s the financial motivation? We’ve had the carrot of web accessibility for sometime now. If you can get a larger number of people to use your web site (such as people with visual impairments) then you can make a larger number of sales.

What’s the cost? The benefits haven’t moved a substantial number of web sites from being inaccessible to accessible! Let’s bring on the stick, the term in potentia. In other words inaccessible web sites may be seen as a future expense. An accessible web site may now be seen as a business benefit because it removes the threat of future lawsuits which cause these potential future expenses. Tesco.com has understood this point and when approached a few years ago by the RNIB created an accessible interface to their online store. Some companies will not have the foresight and will get stuck with the legal costs and the cost of redoing their web site.

By beginning to bring action against inaccessible web sites the RNIB has indicated that the risk factor inherent in having an inaccessible web site is real and growing. With this risk factor now clearly demonstrated the business case for creating a standards based and accessible web site is easier to make.

RNIB open letter

Below is an open letter to Julie Howell of the RNIB concerning the recent redesign of rnib.org.uk.

Dear Julie,
I have written to you specifically as I know you were responsible for the campaign for good web design, I have also cc’d the webteam in on this email as it contains feedback on the new site design.

I feel that I have to write to you to express the disappointment several web designers have with the current RNIB website and the harm it causes to those of us trying to evangelise accessible web sites based around published web standards. Given the costs involved in redesigning a site it is often a difficult task to persuade companies to spend money on a redesign of their sites to make them more accessible to a wider range of people. This task is made even more difficult when business owners can point to a site such as rnib.org.uk and say “The RNIB doesn’t follow the ‘standards’, why should I?”

On my own behalf, and I am sure on behalf of many in the web design community, I would ask you to raise the issue of non compliance with the w3c specifications for both HTML and the web content accessibility guideline 1.0. Failure to comply with these guidelines, or even to attempt to comply with them, is a disappointing state of affairs.

Link to the w3c validator demonstrating the failure to pass validation, even when compensating for a missing doctype and character encoding.

The document also contains a litany of other errors that cause it to fail in many browsers, for example many of the relative links use a ” instead of a ‘/’, this is an error and breaks the links in many browsers. For example the page http://www.rnib.org.uk/xpedio/groups/public/documents/publicwebsite/public_relaunch.hcsp demonstrates these errors, for example links to RNIB advertising are provided as both
“/xpedio/groups/public/documents/publicwebsite/public_rnibadvertising.hcsp”
and
“xpediogroupspublicdocumentspublicwebsitepublic_rnibadvertising.hcsp”
the first of which is correctly written, however the second is incorrect and relies on Internet Explorers error correction mechanisms, and fails in other browsers such as Netscape and Mozilla. When this error affects the users ability to get to content such as the “Tips for screenreader users of this site” you see how the usability of the site is severely damaged.

I think that the new design is poorly thought out with respect to accessibility and usability and sends a poor message in the campaign for accessible websites. The RNIB puts shows itself in a poor light when its own website is inaccessible and broken to a large number of users. As a professional web developer I have to state that I am disappointed in your redesign, especially as I cannot even reach many parts of the site due to the errors in coding on the site.

A couple of other points for the web team to implement, there is no default language specified using the lang attribute, this means that aural browsers cannot automatically pick the best voice to read a page in.

There are no summary attributes on the tables.

You are using large amounts of “presentational” HTML that has been depreciated, this is a tactic that focuses on visual users excessively and should be replaced by style sheets which can be targeted to different peoples browsers, visual and aural.

yours sincerely,
Benjamin Meadowcroft
http://www.benmeadowcroft.com/

After sending the letter I discovered the following response by Julie to another web designers post concerning the matter.

I read through the things that have been said so far. Some of the technical points are beyond my understanding (I’m a policy officer rather than a technical expert) but I have many colleagues who do understand that stuff and who will be able to help me appreciate the issues that have been raised.

Julie is mainly concerned with policy matters and is not directly responsible for the redesign, however she is responsible for the RNIBs accessible web site campaign which is why I directed my comments to her. It is understandable that she would not know the minutiae of web development issues that affect accessible design, however one would image that the people who are employed or contracted to develop the website for the RNIB would have a good understanding the web content accessibility guidelines and standards based web authoring. The fact is that the current state of affairs is extremely poor and a blow against the evangelism of accessible web design.

Liberal HTML Parsing, Not Big, Not Clever

Revisiting the topic of liberal parsing, which has gathered some publicity recently, it is good to reflect on what the problems invalid HTML causes actually are. Suffice to say that toleration of errors and the associated handling of them that is required leads to the incompatibilities and inconsistencies in error handling between clients, coupled with proprietary extensions. Interspersed amongst my own commentary below are a few quotes gathered from a recent e-mail exchange with Dagfinn R. Parnas, the author of a paper on HTML error correction.

After doing some research on the history of browsers and HTML, I really didn’t find it very surprising that the standards compliance was really awful. A wise man once said: “It takes two […] to lie, one to lie and one to listen”. If the early browsers hadn’t started with error-correcting behind the backs of the authors, a lot of the incorrect code would have disappeared shortly. But then again the Internet might not have been as diverse as it is now (writing valid html code takes much more skill than writing tag soup). I think the solution would have been a simple smiley face incorporated in the browser or another means of telling the user how standard compliant the site was. Most web designers (both pro and novice) are very dedicated in making the best site they can, and clients would not be to happy about an angry smiley as they want a professional site. By the way, the wise man was Homer J. Simpson.

What are the most common causes of invalid HTML then?

Type of Error % of documents
with error
Example
missing end tag 41.9% <p>some text, <a href = "http://example.org> link text </p>
invalid end-tag 45.8% <p>Some text </strong></p>
invalid element content 38.3% <p>Some text <p>Some more text </p></p>

In addition to these basic errors the spectre of browser specific extensions raises its head.

70.8 and 23.9 percent [of HTML documents] have defined non-standard attributes and non-standard elements respectively.

This heavy use of proprietary markup significantly raises the entry level for new browsers into the marketplace, a new browser entrant cannot rely on published standards but must examine a multitude of vendor sources to understand how to deal with these proprietary elements. By raising the cost of entry in this manner the current browsers help to maintain their market leading position.

As we move towards XML and the well formed paradigm let’s leave behind the legacy of invalid HTML and adopt the strictness of well formed and valid XHTML. If you can produce valid XHTML do it, if you can’t then stick with HTML and don’t pollute XHTML like we did with HTML.

A Web site in Transition to a Web log (or the Opposite)

While checking my referrer logs the other day I came across a web log listing other blogs which discussed web standards or accessibility. What made me smile was the comment alongside the link to my site…

BenMedowcroft.com – a web site in transition to a web log (or the opposite); info on CSS, WebDAV, HTML. source

As well as making me smile it helped me to think about the direction of this site, hopefully a few of my new ideas prompted by this will lead to a better site experience when I implement them.

Anyway while browsing that list I came across a french site on web standards, very nice looking, if only I could remember more of the french I learnt at school, anyway if you speak french then check this site out, openweb.eu.org

Attribution and the spreading of a meme

This is nothing more than a few observations concerning the significant increase in traffic I have been getting these last few days regarding my post that 99.29% of web sites are obsolete, from the fact that only 0.71% of sites testes were valid HTML.

  1. Dagfinn R. Parnas wrote a message about his masters thesis.
  2. I read the thesis, found it interesting, tabulated the results and blogged it.
  3. Got some normal linkage for a while, no major increases in traffic.
  4. Read an interesting W3c article about improving invalid sites.
  5. The article states that running a survey to determine how many sites are valid, I comment that this has been done, in the thesis.
  6. The time between my posting of that comment and Bill Masons tipping of Zeldman to my blog entry summarising the data leads to the conclusion that Bill read my comment.
  7. Number of recorded Hits to the site quadruples (even more than my last Zeldman mention)

So why mention this? Without permalinks and archives this information may not have been spread as easily over the web, the number of referrers is growing rapidly as different sites pick up on this and comment on it. As I have noted when we create a mechanism by which knowledge can be perpetuated we increase the likelihood that this knowledge will be built upon and more widely disseminated. Especially when authors are liberal with their citations. The practice of citing references allows the web of interconnections to grow and knowledge to be more easily located.

Anyway I’ll leave you with something a little lighter to look over.