Fun With Testing DateUtils.pas #6

 

Okay, so when we last left off, IncMillisecond was still failing in certain circumstances.  Let’s take a look at that.  Note, too, that I have this crazy notion that if you have a function called IncMillisecond, then it should be able to, you know, increment a millisecond.  

Here is the IncMilliseconds that you very likely have on your computer:

function IncMilliSecond(const AValue: TDateTime;
  const ANumberOfMilliSeconds: Int64): TDateTime;
begin
  if AValue > 0 then
    Result := ((AValue * MSecsPerDay) + ANumberOfMilliSeconds) / MSecsPerDay
  else
    Result := ((AValue * MSecsPerDay) - ANumberOfMilliSeconds) / MSecsPerDay;
end;

Now that probably works just fine for you — as long as you don’t have a date that has a value less than the epoch. Below the epoch, and particularly in that magic “48 Hours” area right around the epoch itself, things go horribly awry. As we saw last time, this test will fail:

TestDate := 0.0;
  TestResult := IncMillisecond(TestDate, -1);
  Expected := EncodeDateTime(1899, 12, 29, 23, 59, 59, 999);
  CheckTrue(SameDateTime(Expected, TestResult), 'IncMillisecond failed
    to subtract 1ms across the epoch');

It fails because of a number of reasons actually. The first is precision. The current implementation of IncMillisecond does division using a very small number in the denominator.  In the case of this test the numerator is a really big number multiplied by a really small number.  All of this cries out “precision error!”. (You should thank me – I almost used the <blink> tag there.  Phew!)  And that is basically what happens.  IncMillisecond isn’t precise enough to “see” the difference.

Plus, if you do things around the value of zero, it gets really weird.  For instance, check out the output of this console application:

program IncMillisecondTest;

{$APPTYPE CONSOLE}

uses
  SysUtils, DateUtils;

var
  TestDate: TDateTime;
  TestResult: TDateTime;
  DateStr: string;

begin
  TestDate := 0.0;
  TestResult := IncMilliSecond(TestDate, 1001);
  DateStr := FormatDateTime('dd mmmm, yyyy hh:mm:ss:zzz',  TestResult);
  WriteLn(DateStr);
  TestResult := IncMilliSecond(TestDate, -1001);
  DateStr := FormatDateTime('dd mmmm, yyyy hh:mm:ss:zzz',  TestResult);
  WriteLn(DateStr);
  ReadLn;
end.

I think it is safe to say that something is amiss.

So finally, it is time to rework IncMillisecond, because this pesky little routine is actually at the heart of a bunch of issues with DateUtils.pas. As it will turn out, if you call any of the IncXXXX routines, it all ends up as a call to IncMilliseconds, so this needs to be right.

Okay, so I started out writing this really cool implementation that checked for before and after the epoch, and divided large increments into years and months and days to make sure that their was no loss of precision.  I spent a lot of time on it, and had  whole bunch of tests written and passing with it.   But then it suddenly occurs to me that the trusty TTimeStamp data type and its accompanying conversion routines can once again come to the rescue:

function IncMilliSecond(const AValue: TDateTime;
  const ANumberOfMilliSeconds: Int64 = 1): TDateTime;
var
  TS: TTimeStamp;
  TempTime: Comp;
begin
  TS := DateTimeToTimeStamp(AValue);
  TempTime := TimeStampToMSecs(TS);
  TempTime := TempTime + ANumberOfMilliSeconds;
  TS := MSecsToTimeStamp(TempTime);
  Result := TimeStampToDateTime(TS);
end;

And here is the cool thing:  I was able to change from my sweet but overly complicated version to the new version above without worrying too much about it, because when I made the switch – all of the tests that I had written for my original version still passed.  This was so cool – I could make the change with confidence because of the large set of tests that I had that exercised all aspects on IncMillisecond.

Anywhow….  Again, the TTimeStamp type is precise, and easy. No need to do direct arithmetic on the TDateTime itself. Instead, we can deal with integers and get the exact answer every time no matter how many milliseconds you pass in. You can pass in 5000 years worth of milliseconds, and all will be well. For instance, this test passes just fine.

TestDate := EncodeDate(2010, 4, 8);
MSecsToAdd := Int64(5000) * DaysPerYear[False] * HoursPerDay * MinsPerHour *
    SecsPerMin *  MSecsPerSec; // 1.5768E14 or 157680000000000
TestResult := IncMilliSecond(TestDate, MSecsToAdd);
Expected := EncodeDate(7010, 4, 8);
ExtraLeapDays := LeapDaysBetweenDates(TestDate, Expected);
Expected := IncDay(Expected, -ExtraLeapDays);
CheckTrue(SameDate(Expected, TestResult), 'IncMillisecond failed to
   add 5000 years worth of milliseconds.');

And for you curious folks, here the implementation for the helper function LeapDaysBetweenDates:

function TDateUtilsTests.LeapDaysBetweenDates(aStartDate,
       aEndDate: TDateTime): Word;
var
  TempYear: Integer;
begin
  if aStartDate > aEndDate then
    raise Exception.Create('StartDate must be before EndDate.');
  Result := 0;
  for TempYear := YearOf(aStartDate) to YearOf(aEndDate) do
  begin
    if IsLeapYear(TempYear) then
      Inc(Result);
  end;
  if IsInLeapYear(aStartDate) and
     (aStartDate > EncodeDate(YearOf(aStartDate), 2, 29)) then
    Dec(Result);
  if IsInLeapYear(aEndDate) and
     (aEndDate < EncodeDate(YearOf(aEndDate), 2, 29)) then
    Dec(Result);
end;

From there, the rest of the IncXXXXX routines are simple –- they merely multiply by the next “level up” of time intervals, and call the previous one.  I’ve marked them all inline so that it all happens in one need function call.  Thus, we have:

 

function IncHour(const AValue: TDateTime;
  const ANumberOfHours: Int64 = 1): TDateTime;
begin
  Result := IncMinute(AValue, ANumberOfHours * MinsPerHour);
end;

function IncMinute(const AValue: TDateTime;
  const ANumberOfMinutes: Int64 = 1): TDateTime;
begin
  Result := IncSecond(AValue, ANumberOfMinutes * MinsPerHour);
end;

function IncSecond(const AValue: TDateTime;
  const ANumberOfSeconds: Int64 = 1): TDateTime;
begin
  Result := IncMilliSecond(Avalue, ANumberOfSeconds * MSecsPerSec);
end;

 

One thing to note: DateUtils.pas will only handle years from 1 to 9999. TDateTime won’t handle any date less than midnight on January 1, 0001 nor a date larger than December 31, 9999. So if you are using Delphi to track specific dates in dates before that (or if you plan on doing some time travel into the far future) you’ll have to use some other data type to keep track of dates.

Now, once you’ve done the above, it is tempting to say “Hey, for IncDay, I’ll just add the days to the value passed in.  I mean, that’s all you are really doing.  Well guess what!  You can’t do that!  If you have this for your IncDay:

 

function IncDay(const AValue: TDateTime;
  const ANumberOfDays: Integer = 1): TDateTime;
begin
  Result := AValue + ANumberOfDays;
end;

 

Then this test will not pass because of the strange “48 hour” deal we talked about last post:

 

TestDate := EncodeDateTime(1899, 12, 30, 1, 43, 28, 400);
  TestResult := IncDay(TestDate, -1);
  Expected := EncodeDateTime(1899, 12, 29, 1, 43, 28, 400);
  CheckTrue(SameDate(Expected, TestResult), 'IncDay failed to
    decrement one day from the epoch');

 

Instead, you have to send it all the way back to milliseconds via IncHour, IncMinute, and IncSecond:

 

function IncDay(const AValue: TDateTime;
  const ANumberOfDays: Integer = 1): TDateTime;
begin
  Result := IncHour(AValue, ANumberOfDays * HoursPerDay);
end;

 

Once you put those changes in, well, things get a lot greener.  I have now written a very thorough set of unit test  for testing all of the IncXXXX routines, adding and subtracting dates for both before and after the epoch.  I also test very carefully incrementing and decrementing across the epoch and inside that crazy little 48 hour spot.  They are all passing.

I’ll create a unit with these new fixes in it that you can use if you want.  I’ll also publish the unit that includes these tests that I’ve written.  (When you look at it, be nice.  It’s not very pretty, but it gets the job done.)  As I continue through, I’ll update that file with any other fixes and changes that get made.

Fun With Testing DateUtils.pas #5

Note: This is a “reprint” of content from my blog on Embarcadero.com when I was working there.  They’ve since shut down my blog and the content is gone.  I’m republishing it here.  See the main article for more information.

Okay, so when I left you hanging in the last post, I promised I’d explain what was up with IncMillisecond.  But before I do that, I have to explain a bunch of stuff about TDateTime. And as it turns out, we’ll have to take a detour, and we won’t exactly get to IncMillisecond this time around. 

Most of you probably know how TDateTime works.  TDateTime is a Double that keeps track of minutes in the “front” of the decimal and seconds in the fraction, or the “back”.  The key thing to know is the value of the “epoch” that I mentioned previously.  For TDateTime, the epoch is 0.0, which corresponds to exactly 00:00:00.000 (midnight) on December 30, 1899. (For can read up on all the gory details about why it is December 30, 1899, and not December 31, 1899) 

What this means is that a date time of 2.0 is January 1, 1900 at midnight.  2.5 would be noon on January 1, 1900.  1000.25 would be one thousand days and six hours past December 30, 1899, or September 26, 1903 at 6:00:00 AM.  It also means that –1 is December 29, 1899.  and –1000.25 is Sunday, April 4, 1898 at 6:00:00 AM. 

Now, that last one was a bit tricky if you look carefully at it.  The days part was negative (-1000) but the hours part was not.  Remember, the left part of the double is the number of days before the epoch, but the decimal part – the part to the right, if you will – is always a positive value starting at midnight of the day in question.   I emphasized that last part pretty strongly because once a date goes negative, a counter intuitive thing happens.  The negative part only really applies to the left portion of the value.  The decimal value represents a positive value from midnight.  So to do the last calculation above, I actually had to subtract 999 days and 18 hours to get the right answer.  And there in lies the heart of the problem that we have run into with incrementing milliseconds (and seconds and minutes and hours, as it turns out) for days before the epoch. 

Here’s another way to think about it:  what is the date time value for –0.5?  Well, the correct answer is noon on 29 December 1899.  But look at the left part of the value – it is still zero, which is, of course, 30 December 1899!  And what if you make the call Frac(-0.5) to that value?  You get – ready for it? — -0.5!  And I just got done telling you that you can’t have a “negative” time value.  Time values always are positive values from midnight.  And herein lies our problem. 

Another interesting note:  In the particular world of TDateTime, 0 has an unusual “feature”.  When viewed as the “left” side of a TDateTime, it actually represents a span of time just a hair less than 48 hours.  According to the pure mathematical formula for managing dates and times in Delphi, December 30, 1899 actually has 48 hours.  That is, it stretches from –0.999… to 0.999…. in time.  This is weird, huh?  Never really thought about that, did you?  Well, the whole Date/Time system has to account for this little anomaly. 

So, we have two related issues here:  Time values for negative TDateTime values are really positive, and this weird 48 hour day thing right at the epoch.  Well, frankly I didn’t think about or know about either one when I started out writing my unit tests (until they revealed this issue to me.  Unit testing rocks…) and I am very sad to say that the original author of DateUtils.pas didn’t either.  Both of these errors manifest themselves when calculating times at and before the epoch.  That’s the bad part.    And I know all of this because of unit testing.  That’s the good part. 

But wait, there is more.  As it turns out, all of the time calculations in DateUtils.pas are based on floating point values.  Very, very small floating point values, in fact.  For instance, take a look at the current implementation of IncMillisecond:

 

function IncMilliSecond(const AValue: TDateTime;
  const ANumberOfMilliSeconds: Int64): TDateTime;
begin
  if AValue > 0 then
    Result := ((AValue * MSecsPerDay) + ANumberOfMilliSeconds) / MSecsPerDay
  else
    Result := ((AValue * MSecsPerDay) - ANumberOfMilliSeconds) / MSecsPerDay;
end;

 

The value for MSecsPerDay is pretty large — 86,400,000 – and when you start dividing small numbers by really big numbers you get even smaller numbers –numbers so small that they lose precision.  Now, you can see that our developer at least recognized that something  was going a little goofy with the dates before zero, but the current implementation has the error we are currently looking at.  Alas.

Or even better, go to SysUtils.pas and take a look at TryEncodeTime, which really does some arithmetic fraught with the possibilities for errors and inaccuracies:

 

 

function TryEncodeTime(Hour, Min, Sec, MSec: Word; out Time: TDateTime): Boolean;
begin
  Result := False;
  if (Hour < HoursPerDay) and (Min < MinsPerHour)
    and (Sec < SecsPerMin) and (MSec < MSecsPerSec) then
  begin
    Time := (Hour * (MinsPerHour * SecsPerMin * MSecsPerSec) +
             Min * (SecsPerMin * MSecsPerSec) +
             Sec * MSecsPerSec +
             MSec) / MSecsPerDay;
    Result := True;
  end;
end;

 

That will create some seriously small values, won’t it, given data near midnight on either side?  I’ve subsequently reworked this routine to be more precise.  (I’ll post all this new code for your real soon now.)

Okay, so where to turn in all of this?  The first thing I did was to rewrite IncMilliseconds.  But as you’ll see, even this was really, really tricky and fraught with peril as well.

Okay, so I thought – I’m doing all this test driven development; what I need to do right now is to write some test cases that I know should pass before I even start.  First, I thought that if you have a function called IncMillisecond, then it ought to at least have enough accuracy and precision to at the very least create a different date/time combination, right?

 

 

TestDate := 0.0;
TestResult := IncMillisecond(TestDate);
CheckFalse(SameDateTime(TestDate, TestResult), 'IncMilliseocnd failed to
     change the given date');

 

And of course, this fails.  Good – I expected it to. But after a few hours of writing code, and wondering why it keeps failing, I suddenly realize thatSameDateTime is the problem here!  Argh!

And then it hits me – Uh oh.  I’ve started pulling on a thread, and if I keep pulling on it, it is going to keep unraveling and unraveling….  And that is exactly what happened.

Checkout your SameDateTime:

 

function SameDateTime(const A, B: TDateTime): Boolean;
begin
  Result := Abs(A - B) < OneMillisecond;
end;

 

Now, that looks all well and good. Take the absolute value of the difference, and as long as it is less than 1ms, then the times are effectively the same.OneMillisecond is defined as: OneMillisecond = 1 / MSecsPerDay, or 1.15740741 × 10-8. And in the world of computers, that is a pretty small number. So small, in fact, that it is pretty easy to have small values not register. In our simple test here, the A value is 0, and the B value -1.1574074074e-08. And guess what, that difference is not quite enough to get SameDateTime to return False. It returns True instead.

So, let’s follow this loose thread a bit more, and then we’ll quit for today. We need a SameDateTime function (and, as it turns out, a SameTime function) that returns a correct answer for dates that actually are OneMillisecond apart. We need something that gives answers based on real number so of milliseconds.  And SysUtils.pas has the answer:  TTimeStamp

TTimeStamp is declared as follows:

 

{ Date and time record }
  TTimeStamp = record
    Time: Integer;      { Number of milliseconds since midnight }
    Date: Integer;      { One plus number of days since 1/1/0001 }
  end;

 

Now, that is more like it — integers and not these fuzzy floating point numbers! The accompanying DateTimeToTimeStamp function is exactly what we need. Now, we can write a very precise SameDateTime and SameDate functions:

 

function SameDateTime(const A, B: TDateTime): Boolean;
var
  TSA, TSB: TTimeStamp;
begin
  TSA := DateTimeToTimeStamp(A);
  TSB := DateTimeToTimeStamp(B);
  Result := (TSA.Date = TSB.Date) and (TSA.Time = TSB.Time);
end;

function SameTime(const A, B: TDateTime): Boolean;
begin
  Result := (DateTimeToTimeStamp(A).Time = DateTimeToTimeStamp(B).Time);
end;

 

Those two new implementations will, in fact, return correct results for two dates one millisecond apart.  And let’s just say that TTimeStamp is going to be making more appearances in the new, updated DateUtils.pas in the future.

Okay, so our original, simple test above passes now. But guess what: this second one still doesn’t:

 

 

TestDate := 0.0;
  TestResult := IncMillisecond(TestDate, -1);
  CheckFalse(SameDateTime(TestDate, TestResult), 'IncMilliseocnd failed
    to change the given date');
  Expected := EncodeDateTime(1899, 12, 29, 23, 59, 59, 999);
  CheckTrue(SameDateTime(Expected, TestResult), 'IncMillisecond failed
    to subtract 1ms across the epoch');

 

So next time, we’ll get cracking on that.

Fun With Testing DateUtils.pas #4

Note: This is a “reprint” of content from my blog on Embarcadero.com when I was working there.  They’ve since shut down my blog and the content is gone.  I’m republishing it here.  See the main article for more information.

First, an admin note: I’ve adjusted the color of strings in my code. I was optimizing the colors for reading on my blog proper as opposed to the main site (hadn’t even thought of it, actually, sorry.), and someone pointed out that the colors weren’t working on the main site at all. Hope that this post is better. I changed the last post from Yellow to Lime. If you have a better color suggestion, please let me know. I’ve also endeavored to wrap those long code lines. The code won’t compile as shown, but I trust that you guys can figure it out……

Okay back to the topic at hand.

So things are rolling along. I’ve been writing tons of tests, they are all passing, things are going well, and it’s been fun. But if you have any flair for the dramatic, you can see where this is going….

So there I was rolling along, writing tests for WeeksInAYear (bet you didn’t know that according to ISO 8601, some years have 53 weeks in them, did you. 1981 has 53 weeks, for example) Today, Yesterday – you know, normal stuff. I’m checking edge conditions, standard conditions, all kinds of years, every year. You know, really exercising things. All was rolling along smoothly.

For instance, here are the tests for Yesterday. Not too hard to test, as there is really only one thing you can do:

procedure TDateUtilsTests.Test_Yesterday;
var
  TestResult: TDateTime;
  Expected  : TDateTime;
begin
  TestResult := Yesterday;
  Expected   := IncDay(DateOf(Now), -1);
  CheckEquals(TestResult, Expected, 'The Yesterday function failed to' +
    'return the correct value.');

  TestResult := Yesterday;
  Expected   := DateOf(Now);
  CheckFalse(SameDate(TestResult, Expected), 'The Yesterday function' + 
  'thinks Yesterday is Today, and means that Einstein was totally wrong.');

end;

Just a couple of tests that you can do – or at least what I can think of. (Anyone have any other ideas?) The fun part is that these tests will fail if IncDayand DateOf fail to perform as advertised, we get triple the testing! Sweet!

Things were going along swimmingly, and then all of a sudden, out of left field, all this unit testing stuff suddenly proved to be as valuable as everyone says it is.

Here’s how it happened: I was going along, writing tests, and I wrote this one:

procedure TDateUtilsTests.Test_EndOfTheDay;
var
  TestDate  : TDateTime;
  TestResult: TDateTime;
  i         : Integer;
  Expected  : TDateTime;
begin
  for i        := 1 to 500 do
  begin
    TestDate   := CreateRandomDate(False, 100, 2500);

    TestResult := EndOfTheDay(TestDate);
    // First, don't change the date
    CheckEquals(DayOf(TestDate), DayOf(TestResult), Format('EndOfTheDay changed'
      + ' the day for test date: %s (Result was: %s)', [DateTimeToStr(TestDate),
      DateTimeToStr(TestResult)]));

    // Next, is it really midnight?
    Expected := DateOf(TestDate);
    Expected := IncMillisecond(Expected, -1);
    Expected := IncDay(Expected);
    CheckTrue(SameDateTime(TestResult, Expected), Format('EndOfTheDay didn''t'
      + ' return midnight for test date: %s (Result was: %s, Expected was: %s)',
      [DateTimeToStr(DateOf(TestDate)), DateTimeToStr(TestResult),
       DateTimeToStr(Expected)]));

  end;
end;

Pretty simple and straightforward. But — BOOM – this thing fails. Badly. If you run this test on your computer, the second check, the call to CheckTrue, will pretty quickly fail and you’ll get a message something like:

Test_StartEndOfTheDay: ETestFailure at  $0051FF06 EndOfTheDay 
didn’t return midnight for test date: 5/12/0366 (Result was: 
5/12/0366 11:59:59 PM, Expected was: 5/14/0366 11:59:59 PM), 
expected: <True> but was: <False>

Since the test is creating random dates, you’ll never get the exact same error, but pretty soon I figured out that it only failed for dates before the epoch – that is, for dates that have a negative value and are thus earlier than 30 December 1899.

Naturally, I was left scratching my head. The first inclination is that the test is somehow not correct. But I stared at it for a good long while and came to the conclusion that the test wasn’t the problem.

The first check is fine – the call to EndOfTheDay doesn’t actually change the date as it shouldn’t. But the second test is where the trouble started.

EndOfTheDay is a pretty simple function; it returns the very last millisecond of the date for the date/time combination passed to it – that is, 11:59.999pm for the day in question. It is implemented like so:

// From DateUtils.pas
function EndOfTheDay(const AValue: TDateTime): TDateTime;
begin
  Result := RecodeTime(AValue, 23, 59, 59, 999);
end;

So the natural thing is to actually check to see if the result is indeed that value. So, I did the natural thing: I set the expected date to midnight on the date of the value to be tested, decremented one millisecond, and since that changed the date back one day, I moved it forward again with IncDay. Then I checked to see if they were indeed the same date/time combination. Well, guess what. They weren’t.

I originally had a single line of code combining the three that set the value for Expected. A quick look at the debugger told me that the Expected result wasn’t getting properly calculated. Breaking it down quickly pointed to a strange phenomenon: for dates before the epoch, the IncMillisecond call was actually moving the date portion forward by two days if the date was before the epoch. (Mysteriously, dates after epoch all worked fine. Weird.) That, of course, is a big bad bug.

And this is the part where using the library itself to test other parts of the library is helpful. Because I used IncMillisecond in my test forEndOfTheDay, I found a bug in IncMillisecond. If I hadn’t done so, the problem might have been left lurking for a while longer. Or maybe it never would have revealed itself, depending on how diligent my testing of it ended up once I actually got there.

Luckily, it would appear that not too many of you are manipulating milliseconds for dates before the epoch, because there hasn’t been a big hue and cry about this problem. There have been some QC reports about it, though. But clearly something is dreadfully wrong here.

In the next post, we’ll take a look at just what that is.

Fun With Testing DateUtils.pas #3

Note: This is a “reprint” of content from my blog on Embarcadero.com when I was working there.  They’ve since shut down my blog and the content is gone.  I’m republishing it here.  See the main article for more information.

Okay, things have settled down again, and it is time to get back to my adventure in TDateTime and DateUtils.pas.

When we last left off, I had started at the top of DateUtils, and just started working my way down.  I had written some tests for DateOf and TimeOf, and tried to write tests that pretty thoroughly exercised those functions.  I tried to hit the edges and boundaries, and to test all the different permutations and combinations of a date only, a time only, and both together. 

From there, I worked my way down the list, writing tests for IsLeapYear, IsPM, etc. 

One thing I did was to add IsAM to DateUtils.pas and simply implemented it as:

function IsAM(const AValue: TDateTime): Boolean;
begin
  Result := not IsPM(AValue);
end;

Now, that is really simple.  Shoot, you don’t really need to write tests for that, right?  I mean, I wrote a whole suite of tests for IsPM, and so how could IsAMgo wrong? Well, any number of ways – but the main one is that some day in the future, someone might come along and try to get cute or super-smart or something and change the implementation.  So I went ahead and wrote a whole bunch of tests for IsAM anyway.  Now, if something changes, or if someone changes something, the tests should be able to recognize that. 

Philosophical Note: As I’m doing this, I’m seeing more clearly than ever that writing tests is all about confidence moving forward.  Once you have taken the effort to write thorough, complete suites of unit tests, you can move forward with confidence.  You can make changes and fixes while feeling confident that if your change has unintended consequences, you’ll likely know about it. If you do find a bug, you write a test that “reveals” it, fix the bug so the test passes, and then you can move forward confident that you’ll know right away if that bug comes back to haunt you.  Confidence is a really good thing when it comes to writing code.

So, for instance, let’s look at the tests for IsInLeapYear.  Leap years are a bit funky.  Some years that you think are leap years are not – Quick:  Was 1600 a leap year?  What about 1900?  Wikipedia actually has a good page on leap years.  (Did you know that leap years are also called “intercalary years”? I sure didn’t.)  The actual calculation of a leap year is a bit more complicated that “Is it divisible by 4?”. 

function IsLeapYear(Year: Word): Boolean;
begin
  Result := (Year mod 4 = 0) and ((Year mod 100 <> 0) or (Year mod 400 = 0));
end;

Examine the code, you can see that the answer to the questions above are Yes and No.  (As a side note, our QA Manager is a “Leapling”, born on February 29th.  He’s really only 12 years old.) So, how do you test something called IsInLeapYear?  The declaration is actually quite simple:

function IsInLeapYear(const AValue: TDateTime): Boolean;
begin
  Result := IsLeapYear(YearOf(AValue));
end;

But just because it is simple doesn’t mean that you shouldn’t thoroughly test it!  So I wrote a whole bunch of tests. First, I checked that random dates in years I know are leap years were properly identified as being in a leap year:

  TestDate   := EncodeDate(1960, 2, 29);
  TestResult := IsInLeapYear(TestDate);
  CheckTrue(TestResult, Format('%s is in a leap year, but IsInLeapYear'
    + ' says that it isn''t.  Test #1', [DateToStr(TestDate)]));

  TestDate   := EncodeDate(2000, 7, 31);
  TestResult := IsInLeapYear(TestDate);
  CheckTrue(TestResult, Format('%s is in a leap year, but IsInLeapYear'
    +' says that it isn''t.  Test #2', [DateToStr(TestDate)]));

  TestDate   := EncodeDate(1600, 7, 31);
  TestResult := IsInLeapYear(TestDate);
  CheckTrue(TestResult, Format('%s is in a leap year, but IsInLeapYear'
  + ' says that it isn''t.  Test #4', [DateToStr(TestDate)]));

  TestDate   := EncodeDate(1972, 4, 5);
  TestResult := IsInLeapYear(TestDate);
  CheckTrue(TestResult, Format('%s is in a leap year, but IsInLeapYear'
    +' says that it isn''t.  Test #5', [DateToStr(TestDate)]));

  TestDate   := EncodeDate(1888, 2, 29);
  TestResult := IsInLeapYear(TestDate);
  CheckTrue(TestResult, Format('%s is in a leap year, but IsInLeapYear'
    + ' says that it isn''t.  Test #7', [DateToStr(TestDate)]));

  TestDate   := EncodeDate(2400, 2, 29);
  TestResult := IsInLeapYear(TestDate);
  CheckTrue(TestResult, Format('%s is in a leap year, but IsInLeapYear'
    + ' says that it isn''t.  Test #8', [DateToStr(TestDate)]));

Note that I checked “normal” dates, but also dates in the far future (including the tricky 2400) as well as dates before the epoch (which is December 30, 1899, or a datetime value of 0.0). I’ll talk a little more about the epoch in a future post because the epoch is really, really important to TDateTime. It is also really, really troublesome.   

Another thing to note is that this code uses (and thus tests) EncodeDate. And IsInLeapYear itself will exercise YearOf and IsLeapYear indirectly.  If a test in IsInLeapYear fails indirectly because of one of these, you’ll be able to figure that out pretty quickly, write tests specifically to reveal those problems, fix the problems, and then move forward with confidence that you’ve resolved the issues.

Anyway, I also wrote some negative test cases, checking to see that it returned False for dates that most definitely were not in leap years.   I also wrote tests for dates in years that many folks might thing are leap years but are in fact not leap years:

  // Years that end in 00 are /not/ leap years, unless divisible by 400
  TestDate   := EncodeDate(1700, 2, 28);
  TestResult := IsInLeapYear(TestDate);
  CheckFalse(TestResult, Format('%s is in a leap year, but IsInLeapYear says
    that it isn''t.  Test #6', [DateToStr(TestDate)]));

  TestDate   := EncodeDate(1900, 2, 28);
  TestResult := IsInLeapYear(TestDate);
  CheckFalse(TestResult, Format('%s is in a leap year, but IsInLeapYear says
    that it isn''t.  Test #7', [DateToStr(TestDate)]));

Now that might seem like overkill for a simple function like IsInLeapYear, but I don’t think so. I am now really confident that, since we will be running these tests almost continuously on our Hudson server, no one can mess or alter or change or otherwise break the way leap years are calculated without us knowing about it immediately. And that’s sort of the whole point, right?

Fun With Testing DateUtils.pas #2

Note: This is a “reprint” of content from my blog on Embarcadero.com when I was working there.  They’ve since shut down my blog and the content is gone.  I’m republishing it here.  See the main article for more information.

First of all, thanks for the help improving the CreateRandomDate function.  I confess that I didn’t spend enough time thinking on it, and I’ll also confess that you guys are way smarter than I am.  ðŸ˜‰  I’ll post the “updated” version for your perusal in a separate post. 

In addition, you all were right – that code formatting sucks.   The plug-in I got for Live Writer was not very configurable.  I am now using John Kasters “YAPP” tool, and it all looks a lot better. 

Okay, back to the show…..

So to get going with unit testing DateUtils.pas, I naturally simply “plugged in” to our existing RTL unit test framework.  We have an existing RTL set of unit tests for running DUnit tests on the RTL.  I simply added the unit UnitTest.DateUtils.pas to the project, created the new class:

TDateUtilsTests = class(TTestCase) 
end;

and I was in business.  From there, it’s merely a matter of declaring published methods that run the DUnit tests.

So, I started right at the top with DateOf and TimeOf. So, what to test? Well, the most obvious thing: Does DateOf actually return the date portion of a given TDateTime?  Well, lets create a TDateTime with a random time, then, lets create a TDate with the same date but no time at all, and see if DateOf can do it’s magic?

procedure TDateUtilsTests.Test_DateOf; 
var
  TestDate: TDateTime;
  Expected: TDateTime; 
  Result : TDateTime; 
begin
  TestDate := EncodeDateTime(1945, 12, 1, 1, 46, 13, 112);
  Expected := EncodeDate(1945, 12, 1);
  Result := DateOf(TestDate); 
  CheckTrue(SameDate(Result, Expected), 
       'Test date and Expected date were not the same.'
     + ' DateOf function failed. Test #1'); 
end;



So this is a pretty straightforward test – you create two dates, and see if they are the same after the call to DateOf.  Simple.

What if you, say, increment the time part by one millisecond.  Come on, that can’t hurt anything right?  Better make sure:

  
// This test will fail if it gets run at 23:59.999 
// at night. I'm willing to gamble that this 
// will never happen. 
TestDate := Now;
Expected := IncMillisecond(TestDate); 
Result := DateOf(TestDate);
CheckTrue(SameDate(Result, Expected),
  'Test date and Expected date were not the same.' 
+ ' DateOf function failed. Test #2');


Okay, those are some “positive test cases”.  (I have a bunch more different ones along these lines….)  What about testing the negative case?  That is, test where we know that the two dates should be different after the call, and we test to make sure that they are, indeed different.

  
  TestDate := Now; 
  Expected := DateOf(IncDay(TestDate)); 
  Result := DateOf(TestDate); 
  CheckFalse(SameDate(Result, Expected),
     'Test date and Expected date should have'
   + ' been different but they weren''t. Test #2');


I have a similar set of tests for TimeOf.  These are pretty basic, but that is where you start, right?  With the basics?  From there, I wrote tests that change only the milliseconds, the seconds, the minutes and the hours.  All should never allow the DateOf function to return anything other than the date.  For TimeOf, I do the same – change the year, month, and date and make sure the time is the same.  Then I purposefully change the time and make sure that the function actually does change the time. 

Now, some of you are going to chastise me for using other DateUtils.pas functions to write tests.  Two schools of thought on that.  One says that you should never rely on anything outside of the actually call being tested.  Another says to use those library functions because they’ll get tested all the more when used in other tests. I’m going to be following the latter philosophy, and as well see in a later post, this way of doing things actually will reveal a pretty significant bug in a routine that was used to test another routine.

Fun With Testing DateUtils.pas #1

Note: This is a “reprint” of content from my blog on Embarcadero.com when I was working there.  They’ve since shut down my blog and the content is gone.  I’m republishing it here.  See the main article for more information.

Okay, so I’m a Development Manager. My job is to see to the health, welfare, productivity, effectiveness, and proper tasking of a big chunk of the RAD Studio development team. I share these duties with the excellent and capable Mike Devery.  I mainly manage the guys that work on the IDE, the RTL, and the frameworks.  I do things like make sure they are working on the right thing via our SCRUM interations, that they have good machines, nice chairs, vacations when they want them, the right keyboard, etc.  I manage the development process in that we on the “War Team” spend a lot of time triaging bugs, managing and defining requirements, tracking progress, finding better ways to write better code – you know, development manager stuff.

But, like all good development managers, my heart really is in coding.  And I don’t do much of that anymore.  So I’ve tried to keep my fingers in things by taking on some small development tasks where I can keep my skills up, stimulate my brain in that way, and not cause too much damage. 

What better way to do that than to write unit tests?  I’m a bit weird in that I actually like to write unit tests. I find them challenging and enjoy the “puzzle solving” aspect of it.  I like to try to find corner and edge cases where the tests might fail.  I like knowing that ever test I write means less work down the road because any regressions will be found sooner – hopefully immediately.  And I can write tests to my hearts content without worrying about breaking the product.   

So I started in on DateUtils.pas.  This is a pretty cool unit with a lot of good functionality, and it’s ripe for expanding on our unit tests.  It was written a while ago, and its unit test coverage wasn’t where it should be.  So in my “spare time” (hehe….) I’ve been writing unit tests for the routines in that unit.  It’s been pretty fun, and I think, too, that it’s been illustrative of how beneficial unit testing can be.  So I thought I’d write a series of blog posts about it, and this is the first one. 

So, one of the first things I realized was that I needed to be able to generate dates.  Now, I realized that you don’t want that many non-deterministic tests (or maybe you don’t want any at all – it depends).  But I need to be sure that many of the DateUtils.pas routines can pass with any date.  So I wrote the following routine to generate a legitimate but random date:

 

/// <summary>
///   This creates a random, valid date from year 1 to aYearRange
/// </summary>
function CreateRandomDate(aMakeItLeapYear: Boolean = False; 
        aYearRange: Word = 2500): TDateTime;
var
  AYear, AMonth, ADay, AHour, AMinute, ASecond, AMilliSecond: Word;
begin
  AYear := Random(aYearRange) + 1;
  if (not IsLeapYear(AYear)) and (aMakeItLeapYear) then
  begin
    repeat
      Inc(AYear);
    until IsLeapYear(AYear);
  end;
  AMonth := Random(MonthsPerYear) + 1;
  if IsLeapYear(AYear) and (AMonth = 2) then
  begin
    ADay := Random(29) + 1;
  end else begin
    ADay := Random(28) + 1;
  end;
  AHour := Random(HoursPerDay) - 1;
  AMinute := Random(MinsPerHour) - 1;
  ASecond := Random(SecsPerMin) - 1;
  AMilliSecond := Random(MSecsPerSec);
  Result := EncodeDateTime(AYear, AMonth, ADay, AHour, 
                 AMinute, ASecond, AMilliSecond);
end;

 

Now I’ll bet that you guys can come up with a better algorithm, but this works just fine for testing purposes and is pretty clear in what it does.  I use it to test, say, 1000 random dates against a routine that takes a Date as a parameter.  (By the way, it uses some constants from SysUtils and from DateUtils.)  I mix that in with tests that used constant dates every time.  I debated whether to do the random date thing (if a test fails, you can’t necessarily reproduce it), but I decided in favor of it because I’ll make sure all the tests clearly report the date that was failing, and because I wanted to test to make sure that any date would be handled correctly, and you simply can’t do that with a limited, fixed set of dates.  Constantly running a large set of random dates is as close as you can come to testing “every” date.

Another thing that I knew I’d need was to generate to do date testing is valid “Leap Days’”, that is, a valid February 29 date.  When you unit test, you are constantly looking for corner cases, and Leap Days are a corner case for dates.  Naturally, I’ll utilize CreateRandomDate to help out:

 

function GetRandomLeapDay: TDate;
begin
  Result := DateOf(CreateRandomDate(True));
  Result := RecodeDate(Result, YearOf(Result), 2, 29);
end;

 

So, now I can generate random dates, and easily create corner case LeapDays. I’ll probably also eventually add a GenerateRandomTime routine as well. That’s it for now – next time I’ll talk about the basics of how I got tests up and running. By the way, I am using DUnit.  We use DUnit extensively internally.

My Embarcadero Blog Content

While I was at Embarcadero, I started a series on what I considered “Pretty Good Development Practices”.  Sadly, my tenure ended before I could continue the series further.  Then, mystifyingly, Embarcadero shut down my blog, making all my content unaccessible except via the wonders of the Google Cache. Actually, now it seems that my old blog is back up and running.  Yay!  Well, there was a lot of good content on my blog, if I do say so myself, and so I’m trying to capture as much of it as I can and republish it here.  I’ve grabbed all the “Delphi Pretty Good Practices” series and the “Fun with Testing DateUtils.pas” series as well.  I’ll reformat and republish that content here.  Sadly, I’m not really able to reproduce the great comments that you all left, but I’ll try to reference the Google Cache versions which have them.  (How long does a page stay in the Google Cache — forever?).  Below, then, is an index to as much of my old stuff as I could find/pull together.  If you have any requests, please remind me, and I’ll track it down:

Fun with Testing DateUtils.pas

Fun with Testing DateUtils #1 

Fun with Testing DateUtils #2

Fun with Testing DateUtils #3

Fun with Testing DateUtils #4

Fun with Testing DateUtils #5

Fun with Testing DateUtils #6

Fun with Testing DateUtils #7

Fun with Testing DateUtils #8

Delphi Pretty Good Practices

Delphi Pretty Good Practices #1

Delphi Pretty Good Practices #2

 
 

Delphi Open Source Projects

This page is a listing of a number of high quality Delphi Open Source Projects.

The projects listed on this page are:

  1. Vetted and looked at by me and deemed “worthy” of being here.  I’m happy to take recommendations for listing.  I’ll try not to be too snooty about it, but I don’t want to list every single project out there — there has to be some minimum standard, right?
  2. Available for download via either Subversion (svn) or Mercurial (hg) or another popular open source SCM tool.

If you know if a good, well-maintained, and useful library that deserves to be here, let me know.

Project Name  Short Description 
Delphi Spring Framework A Delphi version of the well known Java Spring Framework.  It includes a Dependency Injection framework, a lovely set of generic collections, encryption libraries, and more.
Internet Direct Indy is a complete socket library for building clients and servers using the standard Internet protocols
Delphi Relax Marco Cantu’s extensions for Delphi Datasnap REST
DUnitX Testing Framework A new, more modern unit testing framework for Delphi
Emballo

Though not currently maintained, I’m including it here anyway.  This is a lovely library, including a Dependency Injection Container and a Mock library, as well as the ability to bind a DLL to an interface.  The community would do well to figure out a way to get this project going again.

Delphi Mocks

A complete mocking framework for Delphi that makes building mock objects drop dead simple.  

Delphi Sorcery

A very nice, powerful framework for Delphi providing data-binding, a dependency injection container, an MVVM framework, a mock object, and more.  

DuckDuckDelphi

A powerful yet easy to use duck typing library for Delphi

About This Blog

I am pretty surprised at how little effort it took to get this site up and running.  From start to finish, including research, set up, and deploying took a Sunday afternoon.  Here are a few notes about the site.

Why?

Well, having your own blog with your own name in the domain is pretty cool. It’s a great place to express ideas, share information, and generally be utterly self-absorbed.  ðŸ˜‰

Blog Software

The blog engine I use for nickhodges.com is BlogEngine.net.  I chose this for a number of reasons:

  • Steve Trefethen recommends it and Steve is a pretty smart guy.
  • It appears to be a fairly robust community of developers, but not so robust that there isn’t room to do something of my own.
  • It’s written in ASP.NET, and thus I feel confident that I can update it, add to it, and fix it if need be.  That relates back to the point above; since there aren’t too many plug-ins for it yet, it leave rooom for me to do some work with it. 
  • It’s a good vehicle to hone my C# skills.  Plus, I bet I can figure out a way to integrate Delphi Prism into the mix as well.

Hosting Service

 

The hosting service I chose was DiscountASP.NET.  The reasons I chose them are as follows:

  • Steve Trefethen recommends it and Steve is a pretty smart guy.  Plus, I could help Steve out by signing up through his referral link so that he gets a kickback.  Sweet!
  • They are obviously very ASP.NET friendly, and my blog engine uses ASP.NET.
  • They are pretty cheap, and they have a really nice set of features.  
  • Their administrative interface is really easy to use. 
  • It was pathetically easy to get my new email address (nick at nickhodges dot com) up and running.  Very nice.
  • It was clear that I could be up and running in no time.
  • The support is very good.  They were really helpful when I had some initial snags, solving the problem on a Sunday.

Please note that I am also part of the referral program, and so if you decide to use  DiscountASP.NET, please feel free to click through on this link, or the banner above.  I’ll get a referral fee, and you’ll have my eternal gratitude.  

Theme/Layout

This was actually the hardest part.  (For the record, Steve Trefethen had nothing to do with this part. Tongue out) BlogEngine.net provides a few themes, but I poked around and soon found a large collection of BlogEngine.Net themes generiously donated by the community.  Someone kindly put them all together on CodePlex.  I downloaded them, and then loaded them all up.  Then, I went through them one by one.  It was a lot of work, actually, but basically I deleted the theme that I knew right away I didn’t want, and then slowly got more critical until I arrived at the Vertigo2 theme. At first I was looking hard for a three column feed, but I liked Vertigo2 so much I decided to forgo the three columns and go with two. This decision was mitigated by the nice footer area.  By the way, the “big N” on the site is a result of “N” being the first letter of my name — it is placed there automatically.

Process

The actual process of getting the blog up and running was pretty easy.  I first set it up locally, got IIS going, and then looked at it on http://localhost.  From there, I tweaked things a bit, got it set up like I wanted, and decided on using XML as the datastore.  BlogEngine.net can use any number of databases, but XML is the simplest and cheapest.  I’ve used XML-based blogs before and they’ve worked out really well.  .Net’s XML handling is really good, so it’s pretty staight-forward to use it.

Once I got things set up like I wanted, it was just a matter of using Filezilla to move the files over to ftp.nickhodges.com, and that was pretty much it.  My domain name is managed by EasyDNS.com, and so I simply used their online interface to point the domain to the DiscountASP.NET name servers, and that was it.  Once the site is up and running, it can be totally administered within itself.  I can still pull a complete copy and play around with it locally if I want to try something new out like adding a widget or an extension.  But generally, any changes like adding posts, etc. is all done within the site itself.

And finally, I am using Windows Live Writer to write most of the posts for the site.  I like it because it allows me to keep and work on drafts offline.