Windward Wrocks

Useful XML .net classes

XmlZipResolver

An XmlUrlResolver for files in a zip file

In the java world a file you can pass a filename of "zip:MyStuff.zip!MyData.xml" and it then handles the MyData.xml file as the xml file to read. The attached file here does the same thing on the .NET side by providing a XmlUrlResolver derived class to handle this case.

This code was about 95% the code from news.oreilly.com/XmlZipResolver.cs where I cleaned it up a little and fixed a couple of subtle bugs.

Just use this class instead of XmlUrlResolver. You can use it for all cases because it calls XmlUrlResolver if the url does not start with zip: or jar:. Hopefully Microsoft will just add this in to XmlUrlResolver in the future.

Download: XmlZipResolver

XmlFileOpen

Opening an XML file that requires a username & password

So you need to open an XML file in .NET and it has a username & password. So piece of cake, you just use XmlUrlResolver.Credentials except... that only works if it's an http or ftp url that requires a username & password.

But what if it's a share on a network drive that requires a username & password to access the file? That gets complicated. Is your app running on a workgroup login or a domain login? Is the file one a workgroup or domain? Is it the same domain? Are the required credentials a user on that machine or a domain user? Or are the permissions a domain group which requires a domain user set of credentials?

Each of those combinations requires a different way of accessing the file. And a couple of combinations are impossible to access, even though "net use" can do it. This is definitely a place where I think Microsoft decided it was too hard to do and so gave up.

And if you find ways to implement some of the remaining (very rare) combinations, please post a comment.

Download: XmlFileOpen.cs

Free book: No Bugs!

I wrote this book in what now looks like the dawn of antiquity. I was on the Windows 95 development team at the time – C++ was still new, we still wrote the time critical parts in assembler, and Java was a word for coffee. It was a challenging world back then. Not better than today, not worse – but different challenges.

I went back and looked at what I wrote. A lot of it is specific to C, C++, and assembler. Some of it is specific to the state of the industry at that time. And those parts will only be of interest to a few with a large degree of curiosity about that time. A bit more will be of interest to people still writing C++ in an unmanaged environment.

But the first two chapters, those are still every bit as relevant today as they were when the book was written. I think most developers can be well served by reading this as it takes a philosophical look at how to write code with fewer bugs.

You can download the entire book for free at No Bugs! You may forward the free copy to friends, post it on your own website, etc. Below is a short excerpt from chapters 1 & 2 if you want a peek before downloading.

One interesting note, in chapter 2 it declares bugs should be called Massive Fuck Ups. The use of the F word had to be approved by a Vice President at the publisher and I think to this day No Bugs! Remains the one programming book from a major publisher where the word appears.

The following is © Copyright 1992 by David Thielen - All Rights Reserved

Chapter 1

INTRODUCTION

Attempt 1

We've all been there. We've finished up our new application. It's been copied and shipped out the door. It's arriving at our first customers - and the phone rings.

It starts as a technical support question. But the problem gets escalated and pretty soon it becomes obvious - there is a bug in the program.

And not just some little insignificant bug that user's can work around - this is a big hairy bug that gets right in the face of each and every user and essentially stops them from using the program.

It doesn't matter how many customers you have, having to replace 100% of the copies of your program out there is a horrendous expense. And the cost is not limited to actually replacing the disks. You now have a reputation for delivering buggy software. This reputation will be costing you sales for the next several years - perhaps enough that a product that would have succeeded now fails.

If you cringe every time the phone rings after shipping a new product - then this book is for you.

These bugs can have more than just a financial impact. The most standard program in the world may be in use in a cardiac care unit in a hospital. A small failure in a copy program could change a number in a loan application. A vertical market application may have a businessman reporting incorrect information to the government (which happens to be a felony). Somebody's word processor was used to type up the specifications to the airplane you are flying in.

People come to depend on programs in ways that the author never foresaw or intended. But they do depend on these programs. And just as you expect the airplane you are flying in to not come apart in mid-air, people expect their programs to operate as expected.

Users now expect, and in many cases get, software which, while not bug free, they will never see a bug in. Over the next several years, they will demand this robustness in all of their programs.

Attempt 2

I recently had this bug in some code reading from a data port. What was driving me crazy was that while I was reading the data a byte at a time, every once in a while my buffer pointer would be off by two.

I placed assertion checks throughout my code to make consistency checks. And the same thing kept happening - it would pass the tests until suddenly it was off by two.

Finally I asked a co-worker to take a look at the code. After studying it for awhile, he came back and said he saw nothing wrong assuming the function calling me hadn't set the direction flag (we're into x86 assembler here).

To make a long story short, the function calling me did not guarantee to clear the direction flag. It's just that it was usually clear.

Not only could this bug have slipped through - I wasn't even testing properly for it. In an environment we make assumptions (such as n = 1 + 2; will give us 3). Unfortunately, we almost always include in this list of assumptions things that are almost always true.

To add insult to injury, a summer hire porting this same code came in and pointed out that I didn't clear the direction flag. I showed him where I did it earlier but told him about spending a day determining that I had to do it. He told me that I just should have shown it to him first - which is unequivocal proof that summer hires should be tortured.

Another case many years ago I wrote a program that included a weird kind of word processor. Because of the requirements, the file format of the word processor was quite complex and we had some bugs in the code.

This was in the early days of the PC and I didn't have all the ideas laid out in this book to use writing the program. There were some bugs in how we were handling the file format. I knew that much but I couldn't seem to find that bugs - at least not all.

Finally, in desperation, I put code in to check the consistency of the file and to correct it when it found problems. As time went on, I found additional bugs in my regular code, which I fixed and I added to the consistency checks and error correction.

While I kept reducing the bug occurrences, I couldn't eliminate them. Finally, to make testing easier, I ripped out the consistency checks and error correction to track down the bugs - and the bugs went away. I had apparently fixed the original bugs but bugs in my bug-fix code had introduced new bugs in the meantime.

And then there are the horror stories you hear of elsewhere (thank god there is always someone doing something much dumber or worse than you).

There is one company where they preferred to not test the product because then bugs were reported and they would have to fix them. Of course, when the users reported the bugs they had to be fixed anyway.

At another company, management would schedule when all bugs were going to be fixed and on that day, by definition there were no remaining bugs. Of course, there were lots of immediate feature enhancements that users requested that were implemented shortly after shipping.

…Chapter continued in No Bugs!

Chapter 2

WHAT IS A BUG?

This is the most important chapter in the entire book. The very simple question, What is a bug, actually has a very complicated answer. This chapter is devoted to answering that question. Because, until we know what a bug is, we don't have a chance of minimizing them.

A developer I know once said, "Bugs should not be called bugs, they should be called Massive Fuck-Ups [MFUs]." "Bug" connotes that some outside agency decided to infest your program with bugs and that if you live a clean life, and sacrifice small furry animals at the foot of your computer, they will go away.

MFUs exist because programs are written by people, and people make mistakes. The only way to eliminate MFUs is to go into your code, find the bugs, and fix them.

This is the most critical concept to understand (and the one most often blown off). You will write MFUs. You will sit down and with complete malice of forethought put MFUs in your code. Think about it - you know that you are the one putting the bugs in there. So if you sit down to code, you will be inserting some bugs.

And somebody needs to find them so you can remove them. No amount of designing, prototyping, CASE, reviewing, management, and so on, will avoid this.

This concept is important because it colors your approach to debugging your code. If you view mistakes as "bugs," you hope none are found. (You hope the good fairy came by, sprinkled pixie dust, and the bugs left.) If you view the mistakes as MFUs, you know they unquestionably exist, and you hope to find all of them so that they can be fixed.

It is critical that developers and managers feel good, not bad, if MFUs are discovered. They should worry if they don't find any. Rather than assuming your program works and waiting for someone to prove otherwise, a developer should assume it doesn't work and search desperately for the bug that will make it fail.

So, for the moment, you have to accept that you will create MFUs in your code (I sure manage to put a lot in my code). I do not know of any way to avoid that. The trick then becomes to find and correct them as painlessly as possible.

As a side note, the term "bug" is so firmly established in the lexicon of developers that I see no way MFU will replace it. However, I think it is important that you occasionally refer to bugs as MFUs to remind people that bugs are "intentionally" placed in the code due to a developer's mistake.

When developers insist that bugs are not MFUs, you should worry. Why? Because they have abdicated responsibility for ferreting them out and fixing them. This is when it's critical to remind them that they are MFUs. It's much harder for someone to avoid responsibility when are told there is an MFU in their code than when they are told there is a bug in their code.

So what is a bug? Well, if you run the program and it formats your disk (assuming you are not writing a format program), most people will agree that a bug did it. From this point on, opinions start to differ. I hear statements like: "No one uses that feature so it's ok that it will trash the hard disk"; "Just tell them to buy a parallel port printer"; "They can override the result so it's okay that it doesn't add up correctly", and "That's not a bug, that's a feature!" The same people who find this type of thing acceptable in the programs they write will turn around and categorize bad design in a program they bought as a bug.

By my definition, if a reasonable person would expect certain functionality, or for a program to work in a certain manner, and it doesn't—there is a bug.

Where bugs come from

How do bugs get into a program? Very simply: You put them there. With reckless disregard, you sit down and type some buggy code into your program. It does not sneak in on its own. You might mistype a line. You might use a wrong variable. You might copy over some code from elsewhere that doesn't quite work. In any case, it is YOU YOU YOU.

Every time you code, you insert more bugs. This is one of the dirty little secrets of programming. Programmers write code and create bugs every day. Good coders are generally really good at creating bugs. That way our employers have to keep us around to fix all of the bugs later (COBOL programmers learned this years ago).

…Chapter continued in No Bugs!

What every developer should know about bitmaps

Virtually every developer will use bitmaps at times in their programming. Or if not in their programming, then in a website, blog, or family photos. Yet many of us don't know the trade-offs between a GIF, JPEG, or PNG file – and there are some major differences there. This is a short post on the basics which will be sufficient for most, and a good start for the rest. Most of this I learned as a game developer (inc. Enemy Nations) where you do need a deep understanding of graphics.

Bitmaps fundamentally store the color of each pixel. But there are three key components to this:

1. Storing the color value itself. Most of us are familiar with RGB where it stores the Red, Green, & Blue component of each color. This is actually the least effective method as the human eye can see subtle differences on some parts of the color spectrum more than others. It's also inefficient for many common operations on a color such as brightening it. But it is the simplest for the most common programming tasks and so has become the standard.
2. The transparency of each pixel. This is critical for the edge of non-rectangular images. A diagonal line, to render best, will be a combination of the color from the line and the color of the underlying pixel. Each pixel needs to have its level of transparency (or actually opacity) set from 0% (show the underlying pixel) to 100% (show just the pixel from the image).
3. The bitmap metadata. This is informat about the image which can range from color tables and resolution to the owner of the image.

Compression

Bitmaps take a lot of data. Or to be more exact, they can take up a lot of bytes. Compression has been the main driver of new bitmap formats over the years. Compression comes in three flavors, palette reduction, lossy & lossless.

In the early days palette reduction was the most common approach. Some programs used bitmaps that were black & white, so 1 bit per pixel. Now that's squeezing it out. And into the days of Windows 3.1 16 color images (4 bits/pixel) were still in widespread use. But the major use was the case of 8-bits/256 colors for a bitmap. These 256 colors would map to a palette that was part of the bitmap and that palette held a 24-bit color for each entry. This let a program select the 256 colors out of the full spectrum that best displayed the picture.

This approach was pretty good and mostly failed for flat surfaces that had a very slow transition across the surface. It also hit a major problem early on with the web and windowed operating systems – because the video cards were also 8-bit systems with a single palette for the entire screen. That was fine for a game that owned the entire screen, but not for when images from different sources shared the screen. The solution to this is a standard web palette was created and most browsers, etc. used that palette if there was palette contention.

Finally, there were some intermediate solutions such as 16-bits/pixel which did provide the entire spectrum, but with a coarse level of granularity where the human eye could see jumps in shade changes. This found little usage because memory prices dropped and video cards jumped quickly from 8-bit to 24-bit in a year.

Next is lossy compression. Compression is finding patterns that repeat in a file and then in the second case just point back to the first run. What if you have a run of 20 pixels where the only difference in the second run is two of the pixels are redder by a value of 1? The human eye can't see that difference. So you change the second run to match the first and voila, you can compress it. Most lossy compression schemes let you set the level of lossiness.

This approach does have one serious problem when you use a single color to designate transparency. If that color is shifted by a single bit, it is no longer transparent. This is why lossy formats were used almost exclusively for pictures and never in games.

Finally comes lossless. This is where the program compresses the snot out of the image with no loss of information. I'm not going to dive into what/how of this except to bring up the point that compressing the images takes substantially more time than decompressing them. So displaying compressed images – fast. Compressing images – not so fast. This can lead to situations where for performance reasons you do not want to store in a lossless format on the fly.

Transparency

Transparency comes in three flavors. (If you know an artist who creates web content – have them read this section. It's amazing the number who are clueless on this issue.) The first flavor is none – the bitmap is a rectangle and will obscure every pixel below it.

The second is a bitmap where a designated color value (most use magenta but it can be any color) means transparent. So other colors are drawn and the magenta pixels are not drawn so the underlying pixel is displayed. This requires rendering the image on a selected background color and the edge pixels that should be partially the image and partially the background pixel then are partially the background color. You see this in practice with 256 color icons where they have perfect edges on a white background yet have a weird white halo effect on their edges on a black background.

The third flavor is 8 bits of transparency (i.e. 256 values from 0 – 100%) for each pixel. This is what is meant by a 32-bit bitmap, it is 24-bits of color and 8 bits of transparency. This provides an image that has finer graduations than the human eye can discern. One word of warning when talking to artists – they can all produce "32-bit bitmaps." But 95% of them produce ones where every pixel is set to 100% opacity and are clueless about the entire process and the need for transparency. (Game artists are a notable exception – they have been doing this forever.) For a good example of how to do this right take a look at Icon Experience – I think their bitmaps are superb (we use them in AutoTag).

Resolution

Many formats have a resolution, normally described as DPI (Dots Per Inch). When viewing a photograph this generally is not an issue. But take the example of a chart rendered as a bitmap. You want the text in the chart to be readable, and you may want it to print cleanly on a 600 DPI printer, but on the screen you want the 600 dots that take up an inch to display using just 96 pixels. The resolution provides this ability. The DPI does not exist in some formats and is optional in others (note: it is not required in any format, but it is unusual for it to be missing in PNG).

The important issue of DPI is that when rendering a bitmap the user may want the ability to zoom in on and/or to print at the printer's resolution but display at a lower resolution – you need to provide the ability for the calling program to set the DPI. There's a very powerful charting program that is useless except for standard viewing on a monitor – because it renders at 96 DPI and that's it. Don't limit your uses.

File formats

Ok, so what file formats should you use? Let's go from most to least useful.

PNG – 32-bit (or less), lossless compression, small file sizes – what's not to like. Older versions of some browsers (like Internet Explorer) would display the transparent pixels with an off-white color but the newer versions handle it properly. Use this (in 32-bit mode using 8 bits for transparency) for everything.

ICO – This is the icon file used to represent applications on the desktop, etc. It is a collection of bitmaps which can each be of any resolution and bit depth. For these build it using just 32-bit png files from 16x16 up to 256x256. If your O/S or an application needs a lesser bit depth, it will reduce on the fly – and keep the 8 bits of transparency.

JPEG – 24-bit only (i.e. no transparency), lossy, small file sizes. There is no reason to use this format unless you have significant numbers of people using old browsers. It's not a bad format, but it is inferior to PNG with no advantages.

GIF – 8-bit, lossy, very small file sizes. GIF has two unique features. First, you can place multiple GIF bitmaps in a single file with a delay set between each. It will then play through those giving you an animated bitmap. This works on every browser back to the 0.9 versions and it's a smaller file size than a flash file. On the flip side it is only 8 bits and in today's world that tends to look poor (although some artists can do amazing things with just 8 bits). It also has a set color as transparent so it natively supports transparency (of the on/off variety). This is useful if you want animated bitmaps without the overhead of flash or if bandwidth is a major issue.

BMP (also called DIB) – from 1 up to 32-bit, lossless, large file sizes. There is one case to use this – when speed is the paramount issue. Many 2-D game programs, especially before the graphics cards available today, would store all bitmaps as a BMP/DIB because no decompression was required and that time saving is critical when you are trying to display 60 frames/second for a game.

TIFF – 32-bit (or less), lossless compression, small file sizes – and no better than PNG. Basically the government and some large companies decided they needed a "standard" so that software in the future could still read these old files. This whole argument makes no sense as PNG fits the bill. But for some customers (like the federal government), it's TIFF instead of PNG. Use this when the customer requests it (but otherwise use PNG).

Everything Else – Obsolete. If you are creating a bitmap editor then by all means support reading/writing every format around. But for other uses – stick to the 2+4 formats above.

Previous articles:

• What every developer should know about character encoding
• What every developer should know about time
• Everything you ever wanted to know about reporting (but were afraid to ask).

What every developer should know about character encoding

I wrote an earlier post like this about timezones assuming 20 people would read it (and 3 would find it useful). And the first day I had over 9,000 page views. So here I am with the same attempt on character encoding. If you write code that touches a text file, you probably need this.

Lets start off with two key items

1. Unicode does not solve this issue for us (yet).
2. Every text file is encoded. There is no such thing as an unencoded file or a "general" encoding.

And lets add a codacil to this – most Americans can get by without having to take this in to account – most of the time. Because the characters for the first 127 bytes in the vast majority of encoding schemes map to the same set of characters (more accurately called glyphs). And because we only use A-Z without any other characters, accents, etc. – we're good to go. But the second you use those same assumptions in an HTML or XML file that has characters outside the first 127 – then the trouble starts.

The computer industry started with diskspace and memory at a premium. Anyone who suggested using 2 bytes for each character instead of one would have been laughed at. In fact we're lucky that the byte worked best as 8 bits or we might have had fewer than 256 bits for each character. There of course were numerous charactersets (or codepages) developed early on. But we ended up with most everyone using a standard set of codepages where the first 127 bytes were identical on all and the second were unique to each set. There were sets for America/Western Europe, Central Europe, Russia, etc.

And then for Asia, because 256 characters were not enough, some of the range 128 – 255 had what was called DBCS (double byte character sets). For each value of a first byte (in these higher ranges), the second byte then identified one of 256 characters. This gave a total of 128 * 256 additional characters. It was a hack, but it kept memory use to a minimum. Chinese, Japanese, and Korean each have their own DBCS codepage.

And for awhile this worked well. Operating systems, applications, etc. mostly were set to use a specified code page. But then the internet came along. A website in America using an XML file from Greece to display data to a user browsing in Russia, where each is entering data based on their country – that broke the paradigm.

Fast forward to today. The two file formats where we can explain this the best, and where everyone trips over it, is HTML and XML. Every HTML and XML file can optionally have the character encoding set in it's header metadata. If it's not set, then most programs assume it is UTF-8, but that is not a standard and not universally followed. If the encoding is not specified and the program reading the file guess wrong – the file will be misread.

Point 1 – Never treat specifying the encoding as optional when writing a file. Always write it to the file. Always. Even if you are willing to swear that the file will never have characters out of the range 1 – 127.

Now lets' look at UTF-8 because as the standard and the way it works, it gets people into a lot of trouble. UTF-8 was popular for two reasons. First it matched the standard codepages for the first 127 characters and so most existing HTML and XML would match it. Second, it was designed to use as few bytes as possible which mattered a lot back when it was designed and many people were still using dial-up modems.

UTF-8 borrowed from the DBCS designs from the Asian codepages. The first 128 bytes are all single byte representations of characters. Then for the next most common set, it uses a block in the second 128 bytes to be a double byte sequence giving us more characters. But wait, there's more. For the less common there's a first byte which leads to a sersies of second bytes. Those then each lead to a third byte and those three bytes define the character. This goes up to 6 byte sequences. Using the MBCS (multi-byte character set) you can write the equivilent of every unicode character. And assuming what you are writing is not a list of seldom used Chinese characters, do it in fewer bytes.

But here is what everyone trips over – they have an HTML or XML file, it works fine, and they open it up in a text editor. They then add a character that in their text editor, using the codepage for their region, insert a character like ß and save the file. Of course it must be correct – their text editor shows it correctly. But feed it to any program that reads according to the encoding and that is now the first character fo a 2 byte sequence. You either get a different character or if the second byte is not a legal value for that first byte – an error.

Point 2 – Always create HTML and XML in a program that writes it out correctly using the encode. If you must create with a text editor, then view the final file in a browser.

Now, what about when the code you are writing will read or write a file? We are not talking binary/data files where you write it out in your own format, but files that are considered text files. Java, .NET, etc all have character encoders. The purpose of these encoders is to translate between a sequence of bytes (the file) and the characters they represent. Lets take what is actually a very difficlut example – your source code, be it C#, Java, etc. These are still by and large "plain old text files" with no encoding hints. So how do programs handle them? Many assume they use the local code page. Many others assume that all characters will be in the range 0 – 127 and will choke on anything else.

Here's a key point about these text files – every program is still using an encoding. It may not be setting it in code, but by definition an encoding is being used.

Point 3 – Always set the encoding when you read and write text files. Not just for HTML & XML, but even for files like source code. It's fine if you set it to use the default codepage, but set the encoding.

Point 4 – Use the most complete encoder possible. You can write your own XML as a text file encoded for UTF-8. But if you write it using an XML encoder, then it will include the encoding in the meta data and you can't get it wrong. (it also adds the endian preamble to the file.)

Ok, you're reading & writing files correctly but what about inside your code. What there? This is where it's easy – unicode. That's what those encoders created in the Java & .NET runtime are designed to do. You read in and get unicode. You write unicode and get an encoded file. That's why the char type is 16 bits and is a unique core type that is for characters. This you probably have right because languages today don't give you much choice in the matter.

Point 5 – (For developers on languages that have been around awhile) – Always use unicode internally. In C++ this is called wide chars (or something similar). Don't get clever to save a couple of bytes, memory is cheap and you have more important things to do.

Wrapping it up

I think there are two key items to keep in mind here. First, make sure you are taking the encoding in to account on text files. Second, this is actually all very easy and straightforward. People rarely screw up how to use an encoding, it's when they ignore the issue that they get in to trouble.

Previous articles:

• What every developer should know about bitmaps
• What every developer should know about time
• Everything you ever wanted to know about reporting (but were afraid to ask).

Feedback on our new ribbon design

We need your feedback on our new design for making maximum use of the Office Ribbon. Please note that this exists only in Office 2007 & 2010 so if you are using an earlier version of Office, you will not be able to test this.

Download at: http://www.windwardreports.com/downloads/beta/New_Ribbon_UI/ (if you need a license key, I posted one below in a comment).

We have two different approaches and both are in the ribbon. Please try both and let us know which you prefer. In addition, any suggestions as to how best to improve either approach, or a suggested different approach, is very welcome.

Please post all comments, suggestions, etc. in my blog post at Windward Wrocks so that others can read your suggestions. Discussion usually leads to even better suggestions.

Approach 1:

All tag editing is in this one tab. You select a tag from the gallery and it inserts a tag at that location. Once the tag is inserted, or anytime the caret is on a tag, you will then get the tag properties in the ribbon. You may also click the little box at any time to get the tag builder.

Approach 2:

With this approach you have two tabs. First is the above tab where every tag is listed directly in the ribbon. You can click a button in the top half and it will insert a tag of that type. Or you can click the drop down and for the Wizard choice, it will bring up the tag builder.

Once you have created a tag, or the caret is on a tag, you then get a second tab in the ribbon. This is similar to when you create or are in a table. When you are not on a tag, this additional tab goes away.

Your Evaluation

Please try each approach for at least 30 minutes so you have a good feel for what is smoothest/easiest on an ongoing basis. Also, please note the next/previous arrows at the far right in all the ribbons – that will walk you through all of your tags.

Excel Users

Special note for Excel users. Excel does not provide a means to determine where the caret is inside a cell. (In fact, when you are editing a cell most Excel functionality is turned off – take a look at the standard Excel buttons in the ribbon.) We track a pseudo location as you click next/previous. But if you then go edit a cell or change the cell selection yourself, we have to start over at the first tag in the cell presently selected.

This is why we have to ask you where in a cell to edit/insert a tag.

Thank You

Thank you for taking the time to look at this new beta and give us your feedback. If you wish to discuss these changes directly, please call or email me.

Thanks – dave

David Thielen
CTO/founder
david@windward.net
303-499-2544 x1185

Ensuring complete code coverage

The way I ensure complete code coverage (got this idea from Gordon Letwin many many years ago) is to place a trap in my code at the start of every new section of code, in every if & else, after a while & for, and anywhere else appropiate.

Then when the trap hits, it drops me in the debugger and I single step through the code until I get to code I've already walked through. This way I end up single stepping through every line of code I write. And by doing a grep for trap I can find code that has never been executed.

Here's the code to make this work in C#. To call it you just call Trap.trap();

using System;

namespace WindwardBear.utils

{

/// <summary>

/// This code will force a break in to the debugger for debug builds. For release builds they do nothing.

/// </summary>

public sealed class Trap

{

 

// can't create this

private Trap()

{

}

 

/// <summary>Will break in to the debugger (debug builds only).</summary>

public static void trap()

{

#if DEBUG

System.Diagnostics.Debugger.Break();

#endif

}

 

/// <summary>Will break in to the debugger if breakOn is true (debug builds only).</summary>

/// <param name="breakOn">Will break if this boolean value is true.</param>

public static void trap(Boolean breakOn)

{

#if DEBUG

if (breakOn)

System.Diagnostics.Debugger.Break();

#endif

}

}

}

Should I approve this expense report?

 

Ok to approve?

What every developer should know about time

With daylight savings time ending today, I thought this was a good time for a post. How to handle time is one of those tricky issues where it is all too easy to get it wrong. So let's dive in. (Note: We learned these lessons when implementing the scheduling system in Windward Reports.)

First off, using UTC (also known as Greenwich Mean Time) is many times not the correct solution. Yet many programmers think if they store everything that way, then they have it covered. (This mistake is why several years ago when Congress changed the start of DST in the U.S. you had to run a hotfix on Outlook for it to adjust reoccurring events.)

So let's start with the key question – what do we mean by time? When a user says they want something to run at 7:00 am, what do they mean? In most cases they mean 7:00 am where they are located – but not always. In some cases, to accurately compare say web server statistics, they want each "day" to end at the same time, unadjusted for DST. At the other end, someone who takes medicine at certain times of the day and has that set in their calendar, will want that to always be on local time so a 3:00pm event is not 3:00am when they have travelled half way around the world.

So we have three main use cases here (there are some others, but they can generally be handled by the following):

1. The same absolute (for lack of a better word) time.
2. The time in a given time zone, shifting when DST goes on/off (including double DST which occurs in some regions).
3. The local time.

The first is trivial to handle – you set it as UTC. By doing this every day of the year will have 24 hours. (Interesting note, UTC only matches the time in Greenwich during standard time. When it is DST there, Greenwich and UTC are not identical.)

The second requires storing a time and a time zone. However, the time zone is the geographical zone, not the present offset (offset is the difference with UTC). In other words, you store "Mountain Time," not "Mountain Standard Time" or "Mountain Daylight Savings Time." So 7:00 am in "Mountain Time" will be 7:00 am in Colorado regardless of the time of year.

The third is similar to the second in that it has a time zone called "Local Time." However, it requires knowing what time zone it is in in order to determine when it occurs.

Outlook now has a means to handle this. Click the Time Zones button:

And you can now set the time zone for each event:

When I have business trips I use this including my flight times departing in one zone and arriving in another. Outlook displays everything in the local timezone and adjusts when that changes. The iPhone on the other hand has no idea this is going on and has everything off when I'm on a trip that is in another timezone (and when you live in Colorado, almost every trip is to another timezone).

Putting it to use

Ok, so how do you handle this? It's actually pretty simple. Every time needs to be stored one of two ways:

1. As UTC. Generally when stored as UTC, you will still set/display it in local time.
2. As a datetime plus a geographical timezone (which can be "local time").

Now the trick is knowing which to use. Here are some general rules. You will need to figure this out for additional use cases, but most do fall in to these categories.

1. When something happened – UTC. This is a singular event and regardless of how the user wants it displayed, when it occurred is unchangeable.
2. When the user selects a timezone of UTC – UTC.
3. An event in the future where the user wants it to occur in a timezone – datetime plus a timezone. Now it might be safe to use UTC if it will occur in the next several months (changing timezones generally have that much warning - although sometimes it's just 8 days), but at some point out you need to do this, so you should do it for all cases. In this case you display what you stored.
4. For a scheduled event, when it will next happen – UTC. This is a performance requirement where you want to be able to get all "next events" where their runtime is before now. Much faster to search against dates than recalculate each one. However, this does need to recalculate all scheduled events regularly in case the rules have changed for an event that runs every quarter.
   1. For events that are on "local time" the recalculation should occur anytime the user's timezone changes. And if an event is skipped in the change, it needs to occur immediately.

.NET DateTime

Diving in to .NET, this means we need to be able to get two things which the standard library does not provide:

1. Create a DateTime in any timezone (DateTime only supports your local timezone and UTC).
2. For a given Date, Time, and geographical timezone, get the UTC time. This needs to adjust based on the DST rules for that zone on that date.

Fortunately there's a solution to this. We have open sourced out extensions to the DateTime timezone functionality. You can download WindwardTimeZone here. This uses registry settings in Windows to perform all calculations for each zone and therefore should remain up to date.

Browser pain

The one thing we have not figured out is how to know a user's location if they are using a browser to hit our web application. For most countries the locale can be used to determine the timezone – but not for the U.S. (6 zones), Canada, or Russia (11 zones). So you have to ask a user to set their timezone – and to change it when they travel. If anyone knows of a solution to this, please let me know.

Update: I received the following from Justin Bonnar (thank you):

   <input id="timezone_offset" type="hidden" name="timezone_offset" value="">
   <script type="text/javascript">
  document.getElementById('timezone_offset').value = new Date().getTimezoneOffset();
   </script>

Using that plus the suggestion of the geo location for the IP address mentioned below will get you close. But it's not 100%. The time offset does not tell you if you for example if you are in Arizona (they & Hawaii do not observer daylight savings time) vs Pacific/Mountain (depending on DST) time zone. You also depend on javascript being on although that is true for 99% of the users out there today.

The geo location based on IP address is also iffy. I was at a hotel in D.C. when I got a report of our demo download form having a problem. We pre-populate the form with city, state, & country based on the geo of the IP address. It said I was in Cleveland, OH. So again, usually right but not always.

My take is we can use the offset, and for cases where there are multiple timezones with that offset (on that given day), follow up with the geo of the IP address. But I sure wish the powers that be would add a tz= to the header info sent with an HTML request.

Did I miss anything?

If I missed an interesting use case, please let me know. Comments as always are very welcome.

Previous articles:

• What every developer should know about bitmaps
• What every developer should know about character encoding
• Everything you ever wanted to know about reporting (but were afraid to ask).