I wrote about digitally signing programs for Windows a few years ago. Microsoft announced last year that windows would no longer trust files signed with the SHA-1 algorithm after 1 January 2017. This causes some problems with older operating systems (like XP SP2 and Vista) as they do not support the SHA256 algorithm for certificates used to sign programs/apps. To maintain compatibility with ealier versions Microsoft suggests dual signing with both the SHA1 and SHA256 certificates. It turns out my certificate (from Comodo and issued in mid-2015) supports both the SHA1 and SHA256 algorithm so it’s not a big hassle for me. However, some older certificates (that have not expired) may need to be re-issued by the issuing authority, some certificate issuers such as K-Software are issuing replacements for free.

Here’s what I had to do to sign my EXE files with both SHA1 and SHA256 versions of my certificate.

1. Download an up-to-date version of the signtool.exe file from Microsoft. Such as this one from the Windows 8.1 SDK.
2. My certificate was installed automatically by Comodo when I purchased it so it needed to be exported to a PFX file. You’ll need to know what the password for your certificate was when you purchased it. There’s a decent tutorial covering this process here.
3. Work out the new commands to dual sign your EXE files. In my case they look something like this.

//Code sign with SHA1
signtool.exe sign /f "c:\path\to\pfx-file\my-pfx-file.pfx" /p mypfxpassword /t http://timestamp.comodoca.com /v c:\Path\To\File\somefile.exe

//Code sign with SHA256
signtool.exe sign /f "c:\path\to\pfx-file\my-pfx-file.pfx" /p mypfxpassword /fd sha256 /tr http://timestamp.comodoca.com/?td=sha256 /td sha256 /as /v c:\Path\To\File\somefile.exe

It’s important to use the /fd and /td switches on the second call to ensure both the file and date/time stamp are SHA256 signed. I actually wrote a little batch file to automate the dual signing of files. I call the file from my various build scripts to sign both my program executables and installers. Here’s that file:

@echo off
echo ************************************************ 
echo Running %0
IF %1=="" GOTO InvalidParameter
IF not exist %1 goto InvalidFile  
echo Signing %1
echo Signing with sha1

"d:\code signing 256\signtool.exe" sign /f "c:\path\to\pfx-file\my-pfx-file.pfx" /p mypfxpassword /t http://timestamp.comodoca.com /v %1

echo Signing with sha256
"d:\code signing 256\signtool.exe" sign /f "c:\path\to\pfx-file\my-pfx-file.pfx" /p mypfxpassword /fd sha256 /tr http://timestamp.comodoca.com/?td=sha256 /td sha256 /as /v %1
echo Signing completed Successfully
goto eof

:InvalidParameter
echo You must pass this script a file to sign
goto eof

:InvalidFile
echo %1 does not exist
goto eof

:eof
echo ************************************************ 

You can easily check if you’ve dual signed correctly from within Windows 10 by right clicking on your signed EXE file, selecting Properties and then clicking the Digital Signatures tab. If it’s done right it should look something like the screen below. Note that both the SHA1 and SHA256 certificates are present.

Dual Code Signed EXE File

Recently I’ve been working with a local truck re-finishing company. They work for their customers on a job-by-job basis, providing fixed priced quotes for each job. Obviously their profitability relies on them controlling both the time and materials spent on those jobs. Their billing, invoicing and payroll is controlled with MYOB, while employee time and attendance and tracking time spent on jobs is done via wall mounted Fingertec Timeline 100 time clock and Fingertec’s TCMSV2 time and attendance software. The time clock only allows for tracking of two digit job numbers which means job numbers must be re-used. There is no link between the TCMS software and MYOB.

The Client’s Fingertec Timeline 100 Time Clock

The Problem

The owner of the business has little or no idea of how much time employees have spent on jobs until the job is complete. This has lead to losses being made on jobs where better control could have resulted in less time being spent by employees. It has also lead to problems in quoting as it has been difficult for the owner to access historical cost information for jobs of a similar type. The reasons for the delay in reporting can be pinned down to the following root causes:

1. The employee responsible for monitoring time and attendance and generating job reports had little understanding of the TCMS software and was not willing to learn more. As a result reporting was largely a process of printing hard copy reports and manual calculations that were either wrong or severely delayed.

2. The TCMS limitation of two digit job numbers meant that the historical information stored by the system was useless. For example, job 89 in May could easily be a completely different job in December. The client controlled what each job code represented via a descriptor the TCMS software could store with each code. When a number was re-used the descriptor would be changed. However, the TCMS software only discriminated by job number.

3. Delays in reporting meant the owner was loathe to ask for reports as they took so long to generate and (appeared) to require a lot of work to generate.

The Goal

The goal of the project was to provide the owner of the business with a simple tool that could report on the time his employees spent working on customer jobs. The tool should be easy to use by admin staff and be able to generate reports in minutes rather than hours or days.

The Solution

I spent several hours familiarising myself with the TCMS software. The reason for this was two-fold. First to understand what data integration was possible, and secondly to investigate the TCMS software for automation opportunities to speed up the time and attendance reporting and payroll processing procedures. The second task lead to immediate cost savings within the business via the following:

1. Configuring TCMS to automatically round times to the nearest 15 minutes (previously done manually)
2. Set up overtime within the software to automatically determine daily overtime on week days and penalty overtime for employees working on weekends.

These two changes have immediately reduced the time taken to process payroll by 2-3 hours per week.

The TCMS reporting system can produce reports in a variety of formats. I was hoping the software could generate reports directly to MS Excel as this would have made my job a lot simpler. However, the Excel reporting system in TCMS is severely flawed resulting in a single report taking more than an hour to generate (I suspect due to not setting the Calculation property of Excel to xlCalculateManual). Further digging around indicated that CSV reports could be generated. In particular the TCMS “Job Cost Analysis” report could be generated in CSV format. The format was a simple flat file that included a record for each time punch pair, the employee, the date, and the recorded job number and job descriptor tag.

The Excel Job Reporter

I developed an Excel spreadsheet that could import the Job Cost Analysis CSV file from TCMS and then present the data to the business owner in a useful format. The spreadsheet allows the construction of a master list of current jobs so that monitoring of current spend of jobs is quick and easy. You can see what this job sheet looks like below. This report is generated daily and made available to staff so that they can see how much budget is available on their current jobs.

Excel Job Tracker – Current Job List

The system also allows the owner to easily see the time spent on any historical job by any employee. Incredibly useful for quoting of new jobs. You can see that report below.

Excel Job Tracker – Detailed Job Report

A number of other reports are also available allowing the owner to see who has been spending time on what in the preceding day, week, or month.

The Benefits

The entire system was developed from start to finish in under 20 hours. The cost of the project is going to be recouped in lower payroll processing costs alone in about 3 months. The opportunities for better job cost control are likely to give multiples of project cost in savings and better quoting each month. All of this with some simple business process analysis, better use of the time and attendance software the client already owned, and making use of Excel.

In late 2012 one of my software products started failing on start up on certain computers.  The product is programmed in VB6 and uses a number of 3rd party controls. The concerning thing about it was that the product would run for months or even years on a computer and then one day, without warning, the software would start up, and display the dreaded “The Application has stopped working” message.  I must confess that for a long time I believed the issue was a minor one, and probably caused by a conflict with an anti virus program or similar.  I tended to (rather poorly) recommend rolling back Windows to an earlier point.  As the months rolled by I saw the issue on Windows Vista, Windows 7 (32 and 64 bit) and Windows 8 and 8.1.  Every 10 or 12 weeks I’d spend a day digging around on Google trying to get to the bottom of it with no luck.  For three reasons, firstly I had no real idea what I was looking for so my searching was rather aimless.  Secondly, I could never EVER ever reproduce the problem on any of the half dozen computers in my office, nor on innumerable VirtualBox machines.  And finally, I never really paid the problem the attention it deserved, which is entirely my fault.

However, in the last month the issue seems to have become more common, with the occasional person reporting it in my software uninstall surveys, and long term users reporting the sudden failure of the software on their computers.  Of course, if it was failing for trial users it was costing me money, but I couldn’t measure the loss so I ignored it.  That changed in the last month when I had to give a number of refunds to users who bought software that suddenly stopped working.  Ouch.  So the bug was really starting to concern me and to be honest, I’d started losing sleep over it because I just couldn’t work out what was going on.  I was even considering re-coding the entire product in C#, not a small task with 75,000+ LOC in the Visual Basic code.  I’d be looking at 6 to 12 months to completion and then at least 6 months of quick release cycles to get the bugs ironed out.  Not an exciting prospect and I’d still have the bug gnawing away at my sales while I was beavering away on the re-write.  And I’ll be honest I didn’t want to move the application to .NET.  I’ve ported a VB6 application to C# before, and while the coding structure was much nicer, the actual speed of the application was FAR worse than the VB6 app and I’ve had to resort to all sorts of caching chicanery to get the performance to what is a barely acceptable level.

Matters Come to a Head

Things came to a head yesterday, with two new people informing me of the error and it was looking like I’d have to refund both of them for purchases made in the last few months.  I spent some time poking around on Google again yesterday and finally hit on something useful, how to create application crash logs.  It turns out getting an application to create a crash dump is as simple as adding one new Windows registry key:

HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindowsWindows Error ReportingLocalDumps

When the application fails it creates a dump file here:

%LOCALAPPDATA%CrashDumps

I tested this on my local dev machine by triggering an application failure and creating a dump file, which I then could open in VS 2010. I couldn’t make much sense of it though. All I needed now was a machine that was experiencing the crash to create a dump file for me and perhaps I could get to the bottom of things.

Along Comes a White Knight

I had the first glimmers of the possibility of getting to the bottom of this vexing issue last night, when a long time user of my product reported that he was experiencing the issue. Recently my usual approach has been to ask if I can get a short time to access the problem computer remotely using my remote access tool of choice, Teamviewer. Sometimes people say yes, sometimes they so no. In the case of this particular customer, he said sure you can have access, and I’ll leave it on ALL NIGHT for you so you can take as long as you like. HUZZAH!

Grabbing the Crash Dump

I logged into the customer’s PC remotely and ran my software. Sure enough, it crashed on startup. Interestingly when it crashed I saw a flash of the main application window before the troublesome error box appeared. I then tried starting up the software with the application window minimized, and hey presto it ran. But as soon as I activated the window the software crashed again. This confirmed what I’d already suspected, that the crash had something to do with the first redraw/paint of the application window. Examining the log files my software creates showed what it usually did, the software crashes somewhere AFTER the first form resize event. Anyway, I’d confirmed that the crash I was seeing was the same one other people had seen. Time to create the Windows Registry key I mentioned earlier to have the software create a crash dump. I did that, ran the software to crash, and looked in the %LOCALAPPDATA%CrashDumps folder and there was my crash dump. Transferring the file to my own computer I wondered what to do next.

WinDBG is King

I’d already opened up a crash log in VS2010 and to be honest it meant nothing to me. So I tooled about for a bit on Google and it appeared that the tool of choice was “WinDBG”. I found an installer for it on windbg.org. Once installed it fired up and I loaded up my crash log and was shown this:

Crash Dump in WinDBG

All very interesting but it doesn’t tell me much and the reason is on the 5th line of text “Symbol search path: *** Invalid ***”. Now, time for full disclosure, I am no computer scientist and my programming coursework was confined to PASCAL and I have little idea of computer architecture and a computer stack to me, is just a pile of computers. So, when I saw this message I had no idea what a “Symbol” was and had even less idea how WinDBG was going to help me. 10 minutes of research later I’d worked out that Symbols are used to decode the crash dump (which is essentially a memory dump) and allow WinDBG to display various function calls and function sources associated with the dump. Giving the debugger a symbol path is simple, just run the following command in WinDBG:

.sympath SRV*f:localsymbols*http://msdl.microsoft.com/download/symbols

Once that was done I could analyze the dump file again with:

!analyze -v

The analysis took a minute or so and the first thing of interest that popped up was:

*** ERROR: Symbol file could not be found.  Defaulted to export symbols for ciaXPLabel30.ocx - 
***** OS symbols are WRONG. Please fix symbols to do analysis.

ciaXPLabel30.ocx is a third party text label control used on the main form of the application. The same form that would partially re-draw before the fatal crash.

The actual call stack was many lines long, but basically was a repetitive loop that looked like this:

0009750c 72a09a7b msvbvm60!BASIC_DISPINTERFACE_GetTypeInfo+0x2aa
0009754c 72a09c2c msvbvm60!EVENT_SINK_Invoke+0x50
0009757c 758c370d oleaut32!CFontEventsCP::Notify+0x9f
000975d4 7589c30e oleaut32!CFont::DiscardFont+0x5d
000975e8 758c41e5 oleaut32!CFont::put_Size+0x76
000975fc 729932c4 msvbvm60!IID_IVbaHost+0x24c84
00097630 72973db1 msvbvm60!IID_IVbaHost+0x5771
00097684 729c1e19 msvbvm60!IID_IVbaHost+0x537d9
000976c0 729acdb0 msvbvm60!IID_IVbaHost+0x3e770
000976fc 729ad0a1 msvbvm60!IID_IVbaHost+0x3ea61
00097730 72980eed msvbvm60!IID_IVbaHost+0x128ad
00097990 4fbadfdb ciaxplabel30!DllCanUnloadNow+0x10efd
00097a28 4fbaa41a ciaxplabel30!DllCanUnloadNow+0xd33c
00097a64 75873e75 oleaut32!DispCallFunc+0x165
00097a80 72a16ef5 msvbvm60!_vbaAptOffset+0x68b

You’ll note in there that the ciaxplabel30 control has raised it’s head again. To my mind this was enough evidence to remove the label controls of that type from my form and replace them with vanilla Visual Basic 6 text labels. That took me about 20 minutes, I ran my automated build script and transferred the new executable file to my helpful user’s computer. And ran the application. And it ran. AND IT DIDN’T CRASH. So, two years of frustration and fruitless work followed by half a morning of focused effort and I’d fixed up my problem.

Where I Went Wrong

My main failing in this whole sorry saga was not paying the bug the attention it deserved. I dithered and procrastinated with it, and it wasn’t until it was costing me real money that I became truly motivated to solve it. Don’t get me wrong, I’d spent time looking into it but it was rather half-hearted. Shame on me.

Why I Went Wrong

There’s two main reasons outside of my control that contributed to the time taken to solve this problem. First, I work in a rather isolated fashion with no other programmers that I can bounce ideas off of. Add that to the fact that my technical understanding of crash dumps and dump analysis borders on non-existent and you have a situation that isn’t going to result in quick fixes. To remedy this in the future I’ll resolve to make better use of online resources like StackOverflow. If I’d asked a question there I am sure someone would have pointed me to the right path in short order. The second contributing factor was my inability to replicate the issue on one of my own dev computers. If could have done that I would have solved the problem through trial and error. Ipso facto, removing all the controls from the form one at a time until the application stopped crashing and solving the problem by trial and error.

Where I Went Right

I didn’t do any of this very well. I guess I did bumble onto the solution in the end but it took a long time and I was heavily reliant on a generous user of my software to do so. Without that help I wonder now if I would have ever gotten to the bottom of it all. Part of me doubts it.

Where to from Here

There’s an important new tool in my debugging arsenal now, and the next time (and I’m sure there’ll be a next time) I know were to start with application crashes. If I get stuck I will make better use of online resources and forums. And I will not sit on these problems and hope they resolve themselves.

I’ve been having a bit of fun in the last few weeks developing the next major release of Time Clock MTS. Part of this release includes a system that sends out emails based on certain events occurring within the software. To accomplish this I could have included some POP email fields and asked users to enter their mail account details. Or I could have used MAPI email. But both of these approaches are ugly as they require input from the user and don’t work for web mail.

So I decided on an approach that goes something like this:

1. The existing application generates an XML file when an event email is required and saves it into the appropriate application data folder.
2. A process monitors the data folder and uploads the files to a web server as it finds them. Each upload process is executed on its’ own thread.
3. The web server processes the XML file and sends emails as required.

Step 1 was easy enough even in Visual Basic 6 (which Time Clock MTS is developed in). However step 2 was always going to be tough so I decided to build a small application in c# to perform this role. The VB6 application would start the C# app on startup and the C# app would shut itself down when the VB6 application closed. Step 3 would be accomplished easily enough with PHP. The key advantage (as I see it) with this approach is that the only requirement for a client computer is an internet connection. It doesn’t require an email client to be installed (like MAPI does) and doesn’t require the user to know and enter some arcane email server settings when they first install the program.

I won’t bore you with Step 1 or 3 but Step 2 had some interesting code that I thought I might share. The C# app is a standard Windows Forms app but I hide the form (except when the debugger is running). I could have made this a windows service but that seemed to introduce a level of complexity (and difficulty in debugging) that was unwarranted. Here’s how I hide the form in the form constructor:

            InitializeComponent();
            if (!Debugger.IsAttached)
            {
                // Prevent the window from showing up in the task bar AND when Alt-tabbing
                ShowInTaskbar = false;
                FormBorderStyle = FormBorderStyle.FixedToolWindow;

                // Move it off-screen
                StartPosition = FormStartPosition.Manual;
                Location = new Point(SystemInformation.VirtualScreen.Right + 10, SystemInformation.VirtualScreen.Bottom + 10);
                Size = new System.Drawing.Size(1, 1);
            }

The work that the form does is handled by a WebExtensionsService class that includes a couple of timers. The class is initialized in the form constructor and the timers are started at the same time. The first timer checks for the existence the parent VB6 program process. I’m sure there’s better ways of doing this but the method I used below works fairly well with little or no CPU overhead and raises an event.

        private void timCheckForClose_Elapsed(object sender, ElapsedEventArgs e)
        {
            //writeLog("timCheckForClose update");
            Process[] adminProcesses = Process.GetProcessesByName("appname");

            if (adminProcesses.Count() == 0)
            {
                if (_bAppRunning == true)
                {
                    _bAppRunning = false;
                    OnAppNotRunning(EventArgs.Empty);
                }
            }
            else
            {
                if (!_bAppRunning )
                {
                    _bAppRunning = true;
                    OnAppNotRunning(EventArgs.Empty);

                }

            }
            

        }

Then I could consume this event in the C# form with this:

        private void WebExtensionsService_AppStopped(object sender, EventArgs e)
        {
            Application.Exit();
        }

I used the Command Line Parser Library to read in a range of command line options that were passed to the C# application when it was started by the parent VB6 application. These settings were used by the second timer to do the real work of this C# forms app. Namely scanning a folder looking for XML files and then uploading them to a server. The second timer in the WebExtensionsService class does this:

        private void timCheckFiles_Elapsed(object sender, ElapsedEventArgs e)
        {
            this.checkForXMLFiles();
        }

        public void checkForXMLFiles()
        {
            Classes.Uploader uploader;

            if (uploaderThreads.Count < this.MaxThreads)
            {
                if (bDirectoryExists(this.XMLFolder))
                {
                    DirectoryInfo info = new DirectoryInfo(this.XMLFolder);
                    FileInfo[] files = info.GetFiles().OrderBy(p => p.CreationTime).ToArray();
                    writeLog("checkForXMLFile:: " + files.Count().ToString() + " files found");
                    foreach (FileInfo file in files)
                    {
                        if (!uploaderThreads.ContainsKey(file.Name))
                        {
                            uploader = new Classes.Uploader();
                            uploader.Filename = file.Name;
                            Thread workerThread = new Thread(uploader.DoWork);
                            uploaderThreads.Add(file.Name, workerThread);
                            workerThread.Start();
                            writeLog("Starting uploader thread " + uploaderThreads.Count + " of " + this.MaxThreads);
                        }
                        if (uploaderThreads.Count >= this.MaxThreads)
                        {
                            writeLog("checkForXMLFile:: MaxThreads " + this.MaxThreads.ToString() + " are busy");
                            break;
                        }
                    }

                }
                else
                {
                    writeLog("checkForXMLFile::"+this.XMLFolder+" doesn't exist");
                }
            }
            else
            {
                writeLog("checkForXMLFile:: MaxThreads " + this.MaxThreads.ToString() + " are busy");
            }
        }

The Uploader class is my worker class that is used to upload a file in it’s own thread. Note that I have a MaxThreads setting and maintain a collection of UploadedThreads to make sure I don’t try to upload the same file twice.

And for interests sake here’s the method from the Uploader class that does the actual work. It’s worth noting that the WebExtensionsService class is a singleton and contains all of the global settings I need. I’m sure this isn’t the most elegant way of doing things but it’s always worked extremely well for me in the past.

        private bool uploadFile()
        {
            bool bReturn = false;
            string sResponse = "No Response";
            WebClient wClient = new WebClient();


            try
            {
                byte[] byte_response = wClient.UploadFile(WebExtensionsService.UploadLocation, "POST", WebExtensionsService.XMLFolder + "\\" + this.Filename);
                if (byte_response != null)
                {
                    sResponse = System.Text.ASCIIEncoding.ASCII.GetString(byte_response);
                    WebExtensionsService.writeLog("Upload " + this.Filename + " Reponse :: " + sResponse);
                    if (sResponse == "1")
                    {
                        bReturn = true;
                    }
                }

            }
            catch (Exception e)
            {
                WebExtensionsService.writeError(e);
            }
            if (bReturn)
            {
                sResult += " UPLOAD SUCCESSFUL";
            }
            else
            {
                sResult += " UPLOAD FAILED RESPONSE " + sResponse;
            }
            

            return bReturn;

        }

Right now the uploadFile method waits for the correct response from the web server. I am thinking about not just uploading the file but also uploading a checksum of the file contents so that the web server can check that the file has not been corrupted in the upload process. But that’s something for another day.

The final step in the application is how to close it out cleanly. I showed earlier how the Application.exit() method was called when the parent VB6 program process was no longer present. Here’s the code that makes sure that all the uploader threads are closed out before the application exits.

        private static void OnApplicationExit(object sender, EventArgs e)
        {
            Classes.WebExtensionsService WebExtensionsService;
            WebExtensionsService = Classes.WebExtensionsService.Instance;
            WebExtensionsService.writeLog("Closing TimeClockMTSWebExtensions");

            foreach (KeyValuePair<string, System.Threading.Thread> entry in WebExtensionsService.UploaderThreads) //wait for uploaded threads to finish
            {
                System.Threading.Thread thread = entry.Value;
                thread.Join();
                WebExtensionsService.writeLog("Waiting for " + entry.Key + " uploader thread to finish");
            }
            WebExtensionsService.writeLog("Closed TimeClockMTSWebExtensions");
        }

The key here is the thread.Join() call. This blocks the main C# program thread until the child threads (stored in my UploaderThreads collection) have completed their work.

Doing this work in C# is far easier than trying to do the same thing in VB6. The C# network code is about a million times easier and more robust. Part of me would LOVE to port the entire VB6 application to .NET but there’s three things stopping me. Firstly, the sheer amount of work. We’re talking about 80,000 lines of code that need porting. Not a trivial task. Second, ADO code in .NET simply isn’t as good as it is in VB6. It’s slower, handles connection pooling very poorly in comparison with VB6, and file sharing of a simple database like MS Access just doesn’t work as well. In the past I’ve had to perform tricky pre-caching of ADO data in .NET to get it performing even remotely as quickly as VB6 code. And finally, it scares me witless mainly because of all the reasons discussed in this great article by Joel Spolsky.

That being said stepping back to VB6 to do development work instead of using C# is like owning a 2013 model car and being forced to drive a leaky 1969 Volkswagen Beetle every second day. It works but it’s not fun. As a result I’m trying to build new features and move older non database reliant features (such as web camera image capture and network time checking code) into COM visible .NET code. However, because of the problems with database handling in .NET I don’t think I’ll ever get rid of VB6 altogether unless Windows drops compatibility for it.