{"id":81,"date":"2005-09-11T02:40:04","date_gmt":"2005-09-10T16:40:04","guid":{"rendered":"http:\/\/www.somethinkodd.com\/oddthinking\/?p=81"},"modified":"2006-03-18T17:52:47","modified_gmt":"2006-03-18T06:52:47","slug":"mail-merge-a-war-story","status":"publish","type":"post","link":"https:\/\/www.somethinkodd.com\/oddthinking\/2005\/09\/11\/mail-merge-a-war-story\/","title":{"rendered":"Mail-Merge: A War Story"},"content":{"rendered":"<p><!-- UnMarkedDown_2_01132526384--><\/p>\n<h3>The Problem<\/h3>\n<p>It&#8217;s true: We were sending over 1500 emails. But, it wasn&#8217;t spam, honestly! We had a prior relationship with each of the recipients, and they had opted-in to the product announcement mailing list. It was time to survey them.<\/p>\n<p>Depending on which random web-site that you believe, scientific research shows that a personalised salutation (e.g. <em>Dear Fred<\/em>, rather than <em>Dear Sir\/Madam<\/em>) either <a href=\"http:\/\/questionpro.com\/akira\/showArticle.do?articleID=deploy01\">increases response rates by at least 5%<\/a> or <a href=\"http:\/\/www.audiencedialogue.org\/kya8.html\">almost makes no difference<\/a>. I suspect the real answer is somewhere in between, and decided to personalise the greeting.<\/p>\n<p>While the URL pointing to the on-line survey included a coded, personally-identifying field &#8211; including their name, we weren&#8217;t betraying their trust, honestly!  We informed them that their identity was being encoded. It not only allowed me to personalise the greeting, but to get a picture of responses by subgroup (e.g. by country, or by version of the product).<\/p>\n<h3>The Solution to the Problem<\/h3>\n<p>So, how do you send 1500 slightly-different emails?  Microsoft Word includes mail-merge software that will talk directly to Microsoft Outlook to send the emails, reading the details from a database or spreadsheet.<\/p>\n<p>I ran a little test against four email addresses, and the mail-merge worked fine. All four emails went out in a few seconds. <\/p>\n<h3>The Problem with the Solution to the Problem<\/h3>\n<p>However, the URL appeared as a long string of text, rather than as a link that could be clicked. The recipient would have to cut-and-paste the result into a browser address bar, rather than just clicking on the link to start the survey. How much would that little bit of extra effort affect the response rate?  I shuddered to think.<\/p>\n<h3>The Solution to the Problem with the Solution to the Problem.<\/h3>\n<p>I spent some time trying to make the URL actually appear as a link with Microsoft Word&#8217;s mail-merge but I was unsuccessful. I discussed it with my colleagues, and the suggestion was made to move from HTML-formatted mail to plain-text formatted mail.  This step is counter-intuitive (URLs should work better in HTML!) but made sense, because rather than sending the wrong HTML (i.e. missing the anchor tag), I was leaving it to the email client to recognise the URL, and to automatically mark it up. Many email clients (including Microsoft Outlook, in use by many of the product&#8217;s user-base) will do this automatically.<\/p>\n<h3>The New Problem<\/h3>\n<p>I steadied myself, and started the mail-merge.<\/p>\n<p>Up popped a dialog box:<\/p>\n<p><img decoding=\"async\" src=\"http:\/\/www.somethinkodd.com\/oddthinking\/images\/mailmerge\/image1.PNG\" alt=\"Dialog Box\" \/><\/p>\n<p>Nice one, Microsoft Office developers! No, seriously! I think making this check was probably very successful in slowing the spread of a number of email-worms. I mean, don&#8217;t get me wrong. The dialog box could be better in several key aspects:<\/p>\n<ul>\n<li>Don&#8217;t offer to approve for 1 minute, 2 minutes, 5 minutes, etc. Does anyone care if they only time-limit period is 10 minutes? That would <a href=\"http:\/\/www.joelonsoftware.com\/uibook\/chapters\/fog0000000059.html\">remove another unnecessary choice from the user<\/a>.<\/li>\n<li>On the other hand, I do want to say &#8220;This is an application I use every day. You can trust it permanently.&#8221;<\/li>\n<li>It could contain some information about the process trying to read my email address. Am I being asked to approve Microsoft Word or some new amazing worm that waits for me to start an mail merge and <em>then<\/em> latches on to Outlook in the confusion!<\/li>\n<li>It was also a little confusing in context &#8211; Word should not have been trying to read my email addresses stored in Outlook, but just send some email to the addresses I had already provided in Excel. Maybe it <em>was<\/em> a new amazing worm!<\/li>\n<\/ul>\n<p>But no matter &#8211; I selected Yes.<\/p>\n<p>Up popped another dialog box:<\/p>\n<p><img decoding=\"async\" src=\"http:\/\/www.somethinkodd.com\/oddthinking\/images\/mailmerge\/image2.PNG\" alt=\"Dialog Box\" \/><\/p>\n<p>Well, this dialog box is more accurate, at least. It described exactly the situation I was expecting. However, this security is less powerful. Any self-respecting email-based worm would carry its own SMTP-stack around with it, and avoid this security point.<\/p>\n<p>Notice the progress bar. It deliberately forces the user to wait for five seconds before the Yes button appears:<\/p>\n<p><img decoding=\"async\" src=\"http:\/\/www.somethinkodd.com\/oddthinking\/images\/mailmerge\/image3.PNG\" alt=\"Dialog Box\" \/><\/p>\n<p>I clicked Yes, and to my horror, another dialog box popped up &#8211; just like the previous one.<\/p>\n<p>It expected me to click once for every email I wanted to send &#8211; at a maximum rate of one every five  seconds. Let&#8217;s see, that&#8217;s 1500 * 5 \/ 3600&#8230; over 2 hours of repeated clicking, even at the optimal rate!<\/p>\n<p>I started to panic, and came up with some wild ideas to try (with the assistance of my bemused colleagues.)  <\/p>\n<ul>\n<li>\n<p><a href=\"http:\/\/www.dimastr.com\/Redemption\/\"><em>Redemption<\/em><\/a>, a separate library to avoid the restrictions? Useful if I was building my own worm (and couldn&#8217;t be bothered implementing my own SMTP-stack) but no good here.  Office didn&#8217;t use this library.<\/p>\n<\/li>\n<li>\n<p>A <a href=\"http:\/\/www.mapilab.com\/outlook\/security\/\">hack<\/a> that offers better security pop-ups from Outlook and permanent approval?  Installed it but coudn&#8217;t get it to have any effect. (A hint to the product manager: Don&#8217;t let the developers write the user documentation, without assistance!)<\/p>\n<\/li>\n<li>\n<p>Software version differences between the machine I ran the original test on and machine I was running the final mail-merge? I did the test on machine in my study at home, and the actual merge on my machine at work. I raced home to re-try it, but it was now prompting for the same security approval there!<\/p>\n<\/li>\n<\/ul>\n<p>I wondered what I had changed since the test run&#8230; and could only think of one thing.<\/p>\n<h3>The New Problem = the Solution to the Problem with the Solution to the Problem.<\/h3>\n<p>I changed the format of the email back from plain-text to HTML, and re-launched the mail-merge. I can hear you scoffing at my superstition; what could possibly relevant about the <em>format<\/em> of the plain-text email that would affect the security of the system?<\/p>\n<p>Well may you scoff, but it sent out all the emails in HTML format, without complaint, at a rate of about 100 per minute.<\/p>\n<p>That&#8217;s right. The Office developers have put <em>tighter<\/em> security around sending <strong>plain-text<\/strong> than they have sending <strong>HTML<\/strong> email! <\/p>\n<p>Crazy.<\/p>\n<p>p.s. Clicking <em>No<\/em> on the dialog box aborts the sending of one piece of email, only. There&#8217;s no other way to gracefully quit from the middle of a mail-merge. I had to kill the process, or click past the dialog 1500 times!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>1500 mouse-clicks at 5 second intervals = over 2 hours of mouse-clicking.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_s2mail":"","footnotes":""},"categories":[23,25],"tags":[],"class_list":["post-81","post","type-post","status-publish","format-standard","hentry","category-based-on-a-true-story","category-insufficiently-advanced-technology"],"_links":{"self":[{"href":"https:\/\/www.somethinkodd.com\/oddthinking\/wp-json\/wp\/v2\/posts\/81","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.somethinkodd.com\/oddthinking\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.somethinkodd.com\/oddthinking\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.somethinkodd.com\/oddthinking\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.somethinkodd.com\/oddthinking\/wp-json\/wp\/v2\/comments?post=81"}],"version-history":[{"count":0,"href":"https:\/\/www.somethinkodd.com\/oddthinking\/wp-json\/wp\/v2\/posts\/81\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.somethinkodd.com\/oddthinking\/wp-json\/wp\/v2\/media?parent=81"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.somethinkodd.com\/oddthinking\/wp-json\/wp\/v2\/categories?post=81"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.somethinkodd.com\/oddthinking\/wp-json\/wp\/v2\/tags?post=81"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}