Author Archives: markn

About markn

Mark is the owner and founder of Timesheets MTS Software, an mISV that develops and markets employee timesheet and time clock software. He's also a mechanical engineer, father of four, and a lifelong lover of gadgets.

    Find more about me on:
  • googleplus
  • linkedin

Compiled HTML Help and Online Help Documents

All my products ship with documentation in the form of a compiled HTML (CHM) file. I also provide the same content online. Maintaining the two different HTML projects was always a pain which I could have solved through the use of a third party tool like Help and Manual. But I am cheap and a few years ago I hit on a solution that allows me to maintain a single copy of my documentation and build it to a compiled help file or upload it as HTML to a website.

Tag the HTML

My individual help files (which are all just HTML files) are littered with custom tags encapsulated within HTML comment tags. So, for example when my help files are displayed online I like to display a simple menu at the top of each page, but I do not want this appearing in the CHM version. So the top of the HTML body in all my documentation HTML files looks something like this:

<body> 
<table width='100%' border=0 bgcolor=#6699cc> 
  <tr> 
    <td width='100%'> 
    <h1>Help Topic Name Goes Here</h1> 
  </td> 
  </tr>
<!--insert php here-->
</table>

Note the “insert php here” tag in the HTML comments.

Search and Replace at Build Time

When I am ready to build a new version of the software (and compile a new CHM file and upload documentation in HTML form) I just need to search and replace the HTML using the free tool FART (Find And Replace Tool). Here’s some of the relevant lines out of my build script (which is just a DOS batch file).

@ECHO BUILDING HELP FILE
"d:\Program Files (x86)\HTML Help Workshop\hhc" "d:\Path\To\HTML\HelpProject\help.hhp"
ECHO Deleting old online help files
delete "d:\Path\To\Online\Help\Files\*.htm"
ECHO Copying help files to online help directory
copy "d:\Path\To\HTML\HelpProject\html" "d:\Path\To\Online\Help\Files\"
ECHO Replacing place holders with PHP
fart -q "d:\Path\To\Online\Help\Files\*.htm" "<!--insert php here-->" "<?php include('inc/header.php');?>"
fart -q "d:\Path\To\Online\Help\Files\*.htm" "</head>" "<?php include('inc/google-tracking.php');?>"
fart -q "d:\Path\To\Online\Help\Files\*.htm" "<script src='scripts/footer.js'></script>" "<?php include('inc/footer.php');?>"
@ECHO FTPing FILES TO WEB SERVER
"d:\Path\To\WinSCP\winscp.exe" /console /script=winscp-ftp-help-file.txt ftp_login:ftp_password@ftp.somesite.com

Line 1 compiles the CHM file. I use good old HTML Help Workshop from Microsoft to do this. It works well and but I maintain all of the project files in a text editor because the GUI is awful.
Line 4 deletes the old temporary copy of the online HTML files.
Line 6 makes a copy of the HTML files into the temporary online HTML location.
Lines 8-10 uses the FART tool to do the search and replaces. Note that I’ve replaced a custom tag with a php_include, replaced the </head> tag with a php include that generates Google Analytics code (and puts the </head> back), and finally replaced a js footer with a PHP equivalent.
Line 12 uses the very neat WINSCP command line SFTP tool to upload the files.

So, there you have it. One set of HTML files can be used to generate a CHM file using HTML Help Workshop and to generate online HTML files. It works very well and automating the process using tools like FART and WINSCP have made my build and release process a snap.

Checking and Setting an INI Entry in Innosetup

Here’s a handy bit of code I wrote today for checking and setting an INI entry using the free setup compiler, Innosetup.

procedure CurStepChanged(CurStep: TSetupStep);
  var 
    sFirstRun: string;
begin
    if CurStep = ssDone then
      begin
          sFirstRun:=GetIniString('FirstRun','FirstRun','0',ExpandConstant('{commonappdata}')+'\AppName\appname.ini');
          if sFirstRun<>'1' then //we are doing a clean install of version 5 set version 5 database updates to 1 to stop upgrade prompt.
            begin
              SetIniString('IniSection','KeyName','1',ExpandConstant('{commonappdata}')+'\AppName\appname.ini');
            end;
      end;
end;

A key thing to note here are that the code is living in the CurStepChanged event and I’m checking that CurStep=ssDone. ssDone represents the successful completion of the installation process. Another item of interest is the ExpandConstant function which is used to access the in-built constants that Innosetup offers.

The reason I’m using this code is to differentiate between a clean install of a program and an install over the top of an existing installation. When any of my programs runs for the first time I write a key (usually FirstRun into section FirstRun) into an ini file. If this key is present then the software has been run at least once. If it’s not present then the installation is a clean one. I can then write an ini entry if it’s a clean install.

Dredd Movie Review

Dredd 2012 Movie Poster

The 2012 movie Dredd came and went having barely registered a blip on my radar. This was for a couple of reasons. It was a big time for comic cross-over films (think The Avengers and The Amazing Spiderman) and I can distinctly remember being quite bored with most of them. That, and the fact that I loathed the 1995 “Judge Dredd” starring Sly Stallone left me less than motivated to see this new imagining of Dredd about which I had heard nothing. A friend of mine went along to see it and afterwards told me that I should take a look. It took me more than a year but last December I watched the DVD and was (very) impressed. I watched it again yesterday and was impressed again, impressed enough to pen this review.

The movie itself depicts a dystopian, post nuclear holocaust future where mankind is huddled into “mega-cities” with populations of nearly a billion people. “Mega-City One” is depicted in the movie as a sprawling walled city that looks for all the world like Los Angeles but sprinkled with towering 200 story apartment blocks that house 50,000 plus people. As you’d imagine such an environment has rampant crime (with more than 17,000 crimes reported daily according to a voice-over), extreme poverty and unemployment, and plot-device appropriate highly addictive narcotics. Law in Mega City One is enforced by “judges” who act as police officer, juror, judge, and in some cases executioner when a “perp” deserves such justice. These one man (or woman) armies ride around on motorcycles (the Lawmaster) dispensing justice with their sophisticated side-arm (the Lawgiver). They wear black armour and a face covering helmet that makes them look for all the world like slightly grungy Darth Vader wannabees.

That all sounds pretty grim and with almost all comic book cross-over movies depicting a some sort of bleak future with a troubled hero it seems almost de-rigueur. But unlike so many other movies Dredd is faithful to the source material, and if anything, guilty of perhaps underselling how bad life in a mega city might be. And Judge Dredd (for whom the movie is named) is not your typical angsty troubled super-hero of the last 15 years. He’s a cold, imposing bad-ass. Karl Urban (who we know from the Lord of the Rings trilogy, the Star Trek reboots, and a couple of the Bourne movies) plays Dredd as an anonymous, incorruptible, unstoppable figure who cuts a path through this movie like an elemental force. When the helmeted Urban (we never see his entire face during the movie) announces over a PA system that “I am the Law” I grinned in anticipation at the mayhem that would ensue. When Sylvester Stallone uttered the same words in the execrable 1995 “Judge Dredd” he sounded like a brain damaged Rocky Balboa. While we’re talking about actors, there’s a couple of others I recognised in this movie. There’s Lena Headley (from The 300 and The Game of Thrones) who plays the scar-faced gang leader “Ma-Ma” and Olivia Thirlby who we saw in Juno and the neat Russian sci-fi movie, The Darkest Hour. Thirlby plays Dredd’s psychic rookie partner “Judge Anderson” who is a recurring character in the 2000AD source material.

The movie itself has a basic plot (which I won’t bother talking about) and a grim and grimy appearance. Most of the action takes place in one of the 200 story mega towers called Peach Trees and when I say action I mean action. There’s a lot of it and most of it is both visually spectacular and confronting. This is not a movie for the squeamish. Some of the deaths depicted are fairly horrific. The special effects appeared to be mostly practical (not much CGI to see here) with costumes, props, and weapons being fairly recognisable. It’s not hard to see the technology depicted in this movie being available just 10 or 20 years from now. This makes it a lot more involving for me, as compared with for example, the technology depicted the recent (and enjoyable) movie, Oblivion.

Two viewings of the movie have left me wanting to get my hands on some 2000AD comics. I read a number of them in my early teens and I can remember them being grim, gory affairs and quite distinct to the other comics I happened to be reading at the time. It’s also left me wanting to watch the movie again because I actually found the second viewing quite a bit more fun than the first. A sequel would be great too but given the poor box-office that’s highly unlikely.  There is bit of a campaign by fans to get a sequel made including a Make a Dredd Sequel Facebook page and I can only wish them best of luck. Dredd is a look at what is an unlikely future but a future that is easily understood by anyone from modern western society. It’s visually confronting and has a superb depiction of a classic comic book hero. Sure the plot is simple but one cannot help but enjoy the ride on the shoulders of Judge Dredd as he dispenses justice without bias or favour. If you’re a fan of sci-fi and don’t mind a bit of gore then I cannot recommend Dredd any more highly.

WordPress vs Movable Type Six Weeks Later

It’s now been almost 6 weeks since I moved this blog from the Movable Type platform to WordPress. I made the move mainly because I prefer WP. However, I was interested at the time to see if WP performed any better with regards to Google organic search traffic because people always rave about how wonderfully WP can perform in SERPS when used with the Yoast SEO plugin. I must confess that I didn’t do much (or any really) optimization when I moved the blog content so I didn’t really expect any improvement. And sure enough that’s what I saw. No improvement. So what does the WordPress vs Movable Type traffic look like? Here’s the Google organic traffic comparison since the migration and the same period prior.

WordPress vs Movable Type

Sure there’s a 50% improvement but the actual traffic this blog receives is so pathetic that I’m going to happily call that “noise”. Perhaps no-one is interested in what I write (so sad) or perhaps there’s a lot more to SEO than on-site factors.

What Next?

I guess as I find time I’ll work my way back through older posts and make use of the Yoast SEO plugin a little better. In the past I’ve generated posts with little (actually no) regard to SEO at all. That’s because this blog is really just a little bit of fun for me. There’s a learning opportunity though, if I can make this blog perform better in the SERPS just through the use of the WordPress blogging platform and a bit of effort with the Yoast plugin then it may be worth the gargantuan effort of migrating one of my product websites to the same platform. Because of that, I’ll have to make sure I find the time to improve my post archive and see what benefits I see over the old Movable Type platform.

Installing the PHP MCRYPT Extension on Ubuntu

I use the PHP mycrypt extension to generate some encrypted URLs for one of my online applications. I had reason to test some of this functionality on my local development web server recently and the following error was thrown by PHP:

Call to undefined function mcrypt_create_iv()

Turns out that I didn’t actually have the MCRYPT extension installed on my little Ubuntu web server. Doing so was simply a matter of typing this at a command prompt:

sudo apt-get install php5-mcrypt

And that’s it. No need to re-start Apache or anything else. Groovy.

Load XML File into Associative Array in PHP

Here’s a neat bit of code to load an XML file into an associative array in PHP. Note that if an XML field is empty it creates a zero element array for that field (for some reason) so I’m checking for this and catching it.

$xml_obj=simplexml_load_file($filename);
$json=json_encode($xml_obj);
$this->file_contents=json_decode($json,true);
			
foreach ($this->file_contents as $key=>$value)
{
  if (is_array($value))
  {
    $value='';
    $this->file_contents[$key]='';
  }
  echo "field::$key value:: ".$value."<br />";
}		

C# MD5 File Hash and PHP Comparison

Further to my recent formless multi-threaded C# application blog post I’ve just implemented a simple MD5 hashing system to ensure that files are uploaded without the contents being changed. I’ve created a new method in my File Uploader class to calculate the MD5 hash of the files I’m uploading. You wouldn’t want to do this on the fly for larger files because of the time taken but the XML files I’m dealing with are less than 2kb in size so it’s not an issue. Here’s the MD5 method for my uploader class.

        private string md5Hash
        {
            get
            {
                using (var md5 = MD5.Create())
                {
                    using (var stream = File.OpenRead(WebExtensionsService.XMLFolder + "\\" + this.Filename))
                    {
                        byte[] md5_bytes=md5.ComputeHash(stream);
                        return BitConverter.ToString(md5_bytes).Replace("-","");
                    }
                }
            }
        }

Note that you need to convert the byte stream to a string and you’re going to have to strip out all the “-” characters to make the MD5 string look normal.

Once I had the MD5 hash I simply appended it to the URL I’m uploading the file to. Like this:

    byte[] byte_response = wClient.UploadFile(WebExtensionsService.UploadLocation+"?md5="+md5, "POST", WebExtensionsService.XMLFolder + "\\" + this.Filename);

The final step is to calculate the MD5 of the uploaded file in PHP and then compare this with the passed MD5 hash. I do this in the code below as well as a couple of other checks to make sure the file is of the correct type and isn’t too big.

	if (isset($_GET['md5']))
	{
		$md5=$_GET['md5'];
	}
	else
	{
		echo "Error : No md5 hash";
		die();		
	}

	if (substr($_FILES["file"]["name"],strlen($_FILES["file"]["name"])-4)=="xml")
	{
		echo "Error : Invalid file type";
		die();
	}
        if ($_FILES["file"]["size"]/1024>1000)
	{
		echo "Error : File too large";
		die();		
	}
	
	$tmp_name=$_FILES["file"]["tmp_name"];
	
	$calculated_md5=md5_file($tmp_name);
	
	if (strtoupper($md5)!=strtoupper($calculated_md5))
	{
		echo "Error : MD5 File Hash Mismatch";
		die();
	}

Help Brian Buy New Computers

Brian Lunduke has a cool blog that I read. He’s a Linux advocate, sometimes tech journalist, and writer of children’s and comic books. And sadly someone just robbed him and stole all of his computers. Rather than appeal for cash to get new ones he’s encouraging people to spread the word about his books (which can be bought here) in the hope of selling enough to pay for new computers. As a prize he’s going to randomly choose someone who helps spread the word to become a character in his next book entitled “Steve’s Laptop”. Now I don’t particularly care about winning anything but I would like a copy of “Linux is Badass” so I’m hoping that one of my readers will pony up the $2 to buy it for me. I’d do it myself but I am cheap.

C# Formless Application with Multi Threading

I’ve been having a bit of fun in the last few weeks developing the next major release of Time Clock MTS. Part of this release includes a system that sends out emails based on certain events occurring within the software. To accomplish this I could have included some POP email fields and asked users to enter their mail account details. Or I could have used MAPI email. But both of these approaches are ugly as they require input from the user and don’t work for web mail.

So I decided on an approach that goes something like this:

1. The existing application generates an XML file when an event email is required and saves it into the appropriate application data folder.
2. A process monitors the data folder and uploads the files to a web server as it finds them. Each upload process is executed on its’ own thread.
3. The web server processes the XML file and sends emails as required.

Step 1 was easy enough even in Visual Basic 6 (which Time Clock MTS is developed in). However step 2 was always going to be tough so I decided to build a small application in c# to perform this role. The VB6 application would start the C# app on startup and the C# app would shut itself down when the VB6 application closed. Step 3 would be accomplished easily enough with PHP. The key advantage (as I see it) with this approach is that the only requirement for a client computer is an internet connection. It doesn’t require an email client to be installed (like MAPI does) and doesn’t require the user to know and enter some arcane email server settings when they first install the program.

I won’t bore you with Step 1 or 3 but Step 2 had some interesting code that I thought I might share. The C# app is a standard Windows Forms app but I hide the form (except when the debugger is running). I could have made this a windows service but that seemed to introduce a level of complexity (and difficulty in debugging) that was unwarranted. Here’s how I hide the form in the form constructor:

            InitializeComponent();
            if (!Debugger.IsAttached)
            {
                // Prevent the window from showing up in the task bar AND when Alt-tabbing
                ShowInTaskbar = false;
                FormBorderStyle = FormBorderStyle.FixedToolWindow;

                // Move it off-screen
                StartPosition = FormStartPosition.Manual;
                Location = new Point(SystemInformation.VirtualScreen.Right + 10, SystemInformation.VirtualScreen.Bottom + 10);
                Size = new System.Drawing.Size(1, 1);
            }

The work that the form does is handled by a WebExtensionsService class that includes a couple of timers. The class is initialized in the form constructor and the timers are started at the same time. The first timer checks for the existence the parent VB6 program process. I’m sure there’s better ways of doing this but the method I used below works fairly well with little or no CPU overhead and raises an event.

        private void timCheckForClose_Elapsed(object sender, ElapsedEventArgs e)
        {
            //writeLog("timCheckForClose update");
            Process[] adminProcesses = Process.GetProcessesByName("appname");

            if (adminProcesses.Count() == 0)
            {
                if (_bAppRunning == true)
                {
                    _bAppRunning = false;
                    OnAppNotRunning(EventArgs.Empty);
                }
            }
            else
            {
                if (!_bAppRunning )
                {
                    _bAppRunning = true;
                    OnAppNotRunning(EventArgs.Empty);

                }

            }
            

        }

Then I could consume this event in the C# form with this:


        private void WebExtensionsService_AppStopped(object sender, EventArgs e)
        {
            Application.Exit();
        }

I used the Command Line Parser Library to read in a range of command line options that were passed to the C# application when it was started by the parent VB6 application. These settings were used by the second timer to do the real work of this C# forms app. Namely scanning a folder looking for XML files and then uploading them to a server. The second timer in the WebExtensionsService class does this:


        private void timCheckFiles_Elapsed(object sender, ElapsedEventArgs e)
        {
            this.checkForXMLFiles();
        }

        public void checkForXMLFiles()
        {
            Classes.Uploader uploader;

            if (uploaderThreads.Count < this.MaxThreads)
            {
                if (bDirectoryExists(this.XMLFolder))
                {
                    DirectoryInfo info = new DirectoryInfo(this.XMLFolder);
                    FileInfo[] files = info.GetFiles().OrderBy(p => p.CreationTime).ToArray();
                    writeLog("checkForXMLFile:: " + files.Count().ToString() + " files found");
                    foreach (FileInfo file in files)
                    {
                        if (!uploaderThreads.ContainsKey(file.Name))
                        {
                            uploader = new Classes.Uploader();
                            uploader.Filename = file.Name;
                            Thread workerThread = new Thread(uploader.DoWork);
                            uploaderThreads.Add(file.Name, workerThread);
                            workerThread.Start();
                            writeLog("Starting uploader thread " + uploaderThreads.Count + " of " + this.MaxThreads);
                        }
                        if (uploaderThreads.Count >= this.MaxThreads)
                        {
                            writeLog("checkForXMLFile:: MaxThreads " + this.MaxThreads.ToString() + " are busy");
                            break;
                        }
                    }

                }
                else
                {
                    writeLog("checkForXMLFile::"+this.XMLFolder+" doesn't exist");
                }
            }
            else
            {
                writeLog("checkForXMLFile:: MaxThreads " + this.MaxThreads.ToString() + " are busy");
            }
        }

The Uploader class is my worker class that is used to upload a file in it’s own thread. Note that I have a MaxThreads setting and maintain a collection of UploadedThreads to make sure I don’t try to upload the same file twice.

And for interests sake here’s the method from the Uploader class that does the actual work. It’s worth noting that the WebExtensionsService class is a singleton and contains all of the global settings I need. I’m sure this isn’t the most elegant way of doing things but it’s always worked extremely well for me in the past.

        private bool uploadFile()
        {
            bool bReturn = false;
            string sResponse = "No Response";
            WebClient wClient = new WebClient();


            try
            {
                byte[] byte_response = wClient.UploadFile(WebExtensionsService.UploadLocation, "POST", WebExtensionsService.XMLFolder + "\\" + this.Filename);
                if (byte_response != null)
                {
                    sResponse = System.Text.ASCIIEncoding.ASCII.GetString(byte_response);
                    WebExtensionsService.writeLog("Upload " + this.Filename + " Reponse :: " + sResponse);
                    if (sResponse == "1")
                    {
                        bReturn = true;
                    }
                }

            }
            catch (Exception e)
            {
                WebExtensionsService.writeError(e);
            }
            if (bReturn)
            {
                sResult += " UPLOAD SUCCESSFUL";
            }
            else
            {
                sResult += " UPLOAD FAILED RESPONSE " + sResponse;
            }
            

            return bReturn;

        }

Right now the uploadFile method waits for the correct response from the web server. I am thinking about not just uploading the file but also uploading a checksum of the file contents so that the web server can check that the file has not been corrupted in the upload process. But that’s something for another day.

The final step in the application is how to close it out cleanly. I showed earlier how the Application.exit() method was called when the parent VB6 program process was no longer present. Here’s the code that makes sure that all the uploader threads are closed out before the application exits.

        private static void OnApplicationExit(object sender, EventArgs e)
        {
            Classes.WebExtensionsService WebExtensionsService;
            WebExtensionsService = Classes.WebExtensionsService.Instance;
            WebExtensionsService.writeLog("Closing TimeClockMTSWebExtensions");

            foreach (KeyValuePair<string, System.Threading.Thread> entry in WebExtensionsService.UploaderThreads) //wait for uploaded threads to finish
            {
                System.Threading.Thread thread = entry.Value;
                thread.Join();
                WebExtensionsService.writeLog("Waiting for " + entry.Key + " uploader thread to finish");
            }
            WebExtensionsService.writeLog("Closed TimeClockMTSWebExtensions");
        }

The key here is the thread.Join() call. This blocks the main C# program thread until the child threads (stored in my UploaderThreads collection) have completed their work.

Doing this work in C# is far easier than trying to do the same thing in VB6. The C# network code is about a million times easier and more robust. Part of me would LOVE to port the entire VB6 application to .NET but there’s three things stopping me. Firstly, the sheer amount of work. We’re talking about 80,000 lines of code that need porting. Not a trivial task. Second, ADO code in .NET simply isn’t as good as it is in VB6. It’s slower, handles connection pooling very poorly in comparison with VB6, and file sharing of a simple database like MS Access just doesn’t work as well. In the past I’ve had to perform tricky pre-caching of ADO data in .NET to get it performing even remotely as quickly as VB6 code. And finally, it scares me witless mainly because of all the reasons discussed in this great article by Joel Spolsky.

That being said stepping back to VB6 to do development work instead of using C# is like owning a 2013 model car and being forced to drive a leaky 1969 Volkswagen Beetle every second day. It works but it’s not fun. As a result I’m trying to build new features and move older non database reliant features (such as web camera image capture and network time checking code) into COM visible .NET code. However, because of the problems with database handling in .NET I don’t think I’ll ever get rid of VB6 altogether unless Windows drops compatibility for it.

Take a Deep Breath

Sometimes being the place where all bucks end up stopping can be a little wearisome. I’ve been tail up working on a major release for one of my products for a few weeks now and this morning I was keen to get a large chunk of a new feature polished off. However, as I am responsible for both technical support and new product development (and everything else in my mISV) I try to get support issues resolved before I start development for the day. Usually this takes an hour or two and leaves me the majority of the day to work on development but this morning I just couldn’t seem to get (what I thought) was a fairly basic point across to a user of one of my products. The email chain went something like this:

To: Support
From: User

My employee can’t clock in.

From: Support
To: User

What product are you using? What version? What browser (and browser version) are you using. Are you getting any error messages when your employee tries to clock in?

To: Support
From: User

It tells them they are not authorized to log in from that computer and I have to manually enter their times.

From: Support
To: User

Oh, you’re using ProductNameRemoved. You’ve set up IP login restrictions to stop people from logging in from un-authorized devices (such as their cell phone). Just log into ProductNameRemoved as the administrator and go to this screen (URL REMOVED) and you’ll be able to adjust the allowed computer IP addresses for your account.

To: Support
From: User

We never set that up.

From: Support
To: User

I’ve just logged into your account and taken a look and you’ve restricted logging into your account to the following IP addresses (ADDRESSES REMOVED)

To: Support
From: User

Oh. Yes I did set that up. My employee can’t clock in.

From: Support
To: User

Why don’t you check what the IP address is on the computer the employee is trying to clock in from. Perhaps it has changed. Once you work that out you can adjust your restriction rules.

To: Support
From: User

Why did you change the computer IP address?

From: Support
To: User

We didn’t. Perhaps your internet service provider has changed your IP address? Unless you have a contract that specifies a fixed IP they can certainly change.

To: Support
From: User

I will check into that. I will give them a call. Thank you for your help!

By this time I’d wasted nearly an hour with this back and forth and (in the words of Homer Simpson) my urge to kill was rising. Thankfully my partner told me to take a deep breath and step away from the computer and go and do something else. So I ducked out, got some sushi, calmed down and have returned for an afternoon that is being entirely more productive.