Monitor Broken Links Using MCC Level Scripts

Note: I recommend you take a look at the official solution from the AdWords Scripts team before implementing this solution.

For anyone who didn't know, they are finally here in Beta form. AdWords scripts are now available at the MCC level. If you want access to these beta features, all you need to do is apply here and wait for the team at Google to give you access.

So what's new with MCC level scripting? The full details are at the Google Developers page, but here is a summary. You can now kick off selectors on Accounts by using the MccApp object. Filtering by stats is the same as other selectors only now you run it at the account level.  As you work through each Account, once you set the account you want to work on using MccApp.select(), everything works just like it used to.

The one new function you will probably want to take advantage of is executeInParallel() which allows you to execute the same code across up to 50 accounts at the same time. So you can kick off a reporting script to run across all of your accounts, then collect the results and send a single email or store it to a single spreadsheet. Also, scripts can now run up to 60 minutes using this method since you get 30 minutes of execution time to run the code on each account and 30 minutes to collect the results from the callback function.

To get you started with the new MccApp object, I thought I would take one of my most popular posts and rewrite it to run at the MCC level. Finding Broken Urls in your Account is a great example of how you can leverage the new executeInParallel() function to improve the monitoring of your MCC.

This script works very similarly to the previous script but has a few added features.  This script will check all your Keyword and Ad urls once per day. When you install this script, you should schedule it to run hourly in case there is a large account which can't be processed in the allotted timeframe, it can pick up from where it left off and continue processing. It controls this internally using labels.

Also, the results of this script are stored in a new spreadsheet for each run. I had issues with the last one where the script would overwrite the values in the spreadsheet before I had a chance to look at them. This eliminates that issue.  The spreadsheet is accessed from a summary email that looks like this, with each row containing a link to the spreadsheet tab with that account's results.

An Example Email from the Script

There are a few other features such as the ability to notify you when errors occur, report on redirects, and setting any number of response codes you are looking for.  Take it for a test drive and let me know what you think in the comments.

Also, just a quick note that I am getting back in the swing of things after getting married at the end of March so look for a much more frequent posting schedule. I am speaking at the Marketing Festival in Brno, Czech Republic at the end of October so I look forward to meeting anyone who can make it.

Thanks,
Russ

/******************************************
* Monitor Broken Links Using MCC Level Scripts
* Version 1.5
* Changelog v1.5
*   - Additional fixes from copy and paste errors
* Changelog v1.4
*   - Fixed INVALID_QUERY error
* Changelog v1.3
*   - Added previous version of report api to script until
*     I update my urls.
* Changelog v1.2
*   - Fixing INVALID_PREDICATE_ENUM_VALUE
* Changelog v1.1
*   - Stopped timeouts 
* Created By: Russ Savage
* FreeAdWordsScripts.com
******************************************/
var SCRIPT_NAME = 'Broken Url Checker';
var LOG_LEVEL = 'error'; //change this to debug if you want more logging
var NOTIFY = [''];
var SPREADSHEET_PREFIX = 'Broken Url Details'; // A timestamp is appended 
var NOTIFY_ON_ERROR = [''];
var STRIP_QUERY_STRING = true; //Drop everything after the ? in the url to speed things up
var REPORT_ON_REDIRECTS = true; //If you want to be able to track 301s and 302, turn this on
var VALID_RESPONSE_CODES = [200,301,302];
var URLS_CHECKED_FILE_NAME = 'UrlsAlreadyChecked-'+AdWordsApp.currentAccount().getCustomerId()+'.json';
var DONE_LABEL_PREFIX = 'All Urls Checked - ';
  
function main() {
  MccApp.accounts().withLimit(50).executeInParallel('checkUrls', 'reportResults'); 
}
    
function checkUrls() {
  try {
    debug('Processing account: '+AdWordsApp.currentAccount().getName());
      
    debug('Checking to see if we finished processing for today.');
    var dateStr = Utilities.formatDate(new Date(), AdWordsApp.currentAccount().getTimeZone(), 'yyyy-MM-dd');
    var finishedLabelName = DONE_LABEL_PREFIX+dateStr;
    var alreadyDone = AdWordsApp.labels().withCondition("Name = '"+finishedLabelName+"'").get().hasNext();
    if(alreadyDone) {
      info('All urls have been checked for today.');
      return '';
    }
    var labelIter = AdWordsApp.labels().withCondition("Name STARTS_WITH '"+DONE_LABEL_PREFIX+"'").get();
    while(labelIter.hasNext()) { labelIter.next().remove(); }
      
    debug('Checking for previous urls.');
    var urlsAlreadyChecked = readValidUrlsFromJSON();
    info('Found '+Object.keys(urlsAlreadyChecked).length+' urls already checked.');
      
    var toReportKeywords = [];
    var toReportAds = [];
    var didExitEarly = false;
    var keywordUrls = getKeywordUrls();
    for(var key in keywordUrls) {
      var kwRow = keywordUrls[key];
      var final_urls = kwRow.FinalUrls.split(';');
      for(var i in final_urls) {
        var url = cleanUrl(final_urls[i]);
        verifyUrl(kwRow,url,urlsAlreadyChecked,toReportKeywords);
        if(shouldExitEarly()) { didExitEarly = true; break; }
      }
    }
    if(!didExitEarly) {
      var adUrls = getAdUrls();
      for(var i in adUrls) {
        var adRow = adUrls[i];
        if(adRow.CreativeFinalUrls) {
          var final_urls = adRow.CreativeFinalUrls.split(';');
          for(var x in final_urls) {
            var url = cleanUrl(final_urls[x]);
            verifyUrl(adRow,url,urlsAlreadyChecked,toReportAds);
          }
        }
        if(shouldExitEarly()) { didExitEarly = true; break; }
      }
    }
    var returnData = {
      accountId : AdWordsApp.currentAccount().getCustomerId(),
      accountName : AdWordsApp.currentAccount().getName(),
      uniqueUrlsChecked : Object.keys(urlsAlreadyChecked).length,
      brokenKeywords : toReportKeywords,
      brokenAds : toReportAds,
      didExitEarly : didExitEarly
    };
    if(didExitEarly) {
      writeValidUrlsToJSON(urlsAlreadyChecked);
    } else {
      AdWordsApp.createLabel(finishedLabelName, 'Label created by '+SCRIPT_NAME, '#C0C0C0');
      writeValidUrlsToJSON({});
    }
    return JSON.stringify(returnData);
  } catch(e) {
    // This error handling helps notify you when things don't work out well.
    error(e);
    if(MailApp.getRemainingDailyQuota() >= NOTIFY_ON_ERROR.length) {
      var acctName = AdWordsApp.currentAccount().getName();
      var acctId = AdWordsApp.currentAccount().getCustomerId();
      for(var i in NOTIFY_ON_ERROR) {
        info('Sending mail to: '+NOTIFY_ON_ERROR[i]);
        MailApp.sendEmail(NOTIFY_ON_ERROR[i], 'ERROR: '+SCRIPT_NAME+' - '+acctName+' - ('+acctId+')', e);
      }
    } else {
      error('Out of email quota for the day. Sending a carrier pigeon.'); 
    }
    return '';
  }
    
  function shouldExitEarly() {
    return (AdWordsApp.getExecutionInfo().getRemainingTime() < 60);
  } 
  
  function verifyUrl(row,url,urlsAlreadyChecked,toReport) {
    if(!urlsAlreadyChecked[url]) {
      info('Checking url: ' + url);
      var urlCheckResults = checkUrl(url);
      if(!urlCheckResults.isValid) {
        row['cleanUrl'] = url;
        row['responseCode'] = urlCheckResults.responseCode;
        toReport.push(row);
      }
      urlsAlreadyChecked[url] = urlCheckResults;
    } else {
      if(!urlsAlreadyChecked[url].isValid) {
        row['cleanUrl'] = url;
        row['responseCode'] = urlsAlreadyChecked[url].responseCode;
        toReport.push(row);
      }
    }
  }
  
  function checkUrl(url) {
    var retVal = { responseCode : -1, isValid: false };
    var httpOptions = {
      muteHttpExceptions:true,
      followRedirects:(!REPORT_ON_REDIRECTS)
    };
    try {
      retVal.responseCode = UrlFetchApp.fetch(url, httpOptions).getResponseCode();
      retVal.isValid = isValidResponseCode(retVal.responseCode);
    } catch(e) {
      warn(e.message);
      //Something is wrong here, we should know about it.
      retVal.isValid = false;
    }
    return retVal;
  }
    
  function isValidResponseCode(resp) {
    return (VALID_RESPONSE_CODES.indexOf(resp) >= 0);
  }
    
  //Clean the url of query strings and valuetrack params  
  function cleanUrl(url) {
    if(STRIP_QUERY_STRING) {
      if(url.indexOf('?')>=0) {
        url = url.split('?')[0];
      }
    }
    if(url.indexOf('{') >= 0) {
      //Let's remove the value track parameters
      url = url.replace(/\{[^\}]*\}/g,'');
    }
    return url;
  }
    
  //Use the reporting API to pull this information because it is super fast.
  //The documentation for this is here: http://goo.gl/IfMb31
  function getKeywordUrls() {
    var OPTIONS = { includeZeroImpressions : true };
    var cols = ['CampaignId','CampaignName',
                'AdGroupId','AdGroupName',
                'Id','Criteria','KeywordMatchType',
                'IsNegative','FinalUrls','Impressions'];
    var report = 'KEYWORDS_PERFORMANCE_REPORT';
    var query = ['select',cols.join(','),'from',report,
                 'where CampaignStatus = ENABLED',
                 'and AdGroupStatus = ENABLED',
                 'and Status = ENABLED',
                 'during','LAST_7_DAYS'].join(' ');
    var results = {};
    var reportIter = AdWordsApp.report(query, OPTIONS).rows();
    while(reportIter.hasNext()) {
      var row = reportIter.next();
      if(row.IsNegative === 'true') { continue; }
      if(!row.FinalUrls) { continue; }
      if(row.KeywordMatchType === 'Exact') {
        row.Criteria = ['[',row.Criteria,']'].join('');
      } else if(row.Criteria === 'Phrase') {
        row.Criteria = ['"',row.Criteria,'"'].join('');
      }
      var rowKey = [row.CampaignId,row.AdGroupId,row.Id].join('-');
      results[rowKey] = row;
    }
    return results; 
  }
    
  //Use the reporting API to pull this information because it is super fast.
  //The documentation for this is here: http://goo.gl/8RHTBj
  function getAdUrls() {
    var OPTIONS = { includeZeroImpressions : true };
    var cols = ['CampaignId','CampaignName',
                'AdGroupId','AdGroupName',
                'AdType',
                'Id','Headline','Description1','Description2','DisplayUrl',
                'CreativeFinalUrls','Impressions'];
    var report = 'AD_PERFORMANCE_REPORT';
    var query = ['select',cols.join(','),'from',report,
                 'where CampaignStatus = ENABLED',
                 'and AdGroupStatus = ENABLED',
                 'and Status = ENABLED',
                 'during','TODAY'].join(' ');
    var results = {};
    var reportIter = AdWordsApp.report(query, OPTIONS).rows();
    while(reportIter.hasNext()) {
      var row = reportIter.next();
      if(!row.CreativeFinalUrls) { continue; }
      var rowKey = [row.CampaignId,row.AdGroupId,row.Id].join('-');
      results[rowKey] = row;
    }
    return results;
  }
    
  //This function quickly writes the url data to a file
  //that can be loaded again for the next run
  function writeValidUrlsToJSON(toWrite) {
    var file = getFile(URLS_CHECKED_FILE_NAME,false);
    file.setContent(JSON.stringify(toWrite));
  }
    
  //And this loads that stored file and converts it to an object
  function readValidUrlsFromJSON() {
    var file = getFile(URLS_CHECKED_FILE_NAME,false);
    var fileData = file.getBlob().getDataAsString();
    if(fileData) {
      return JSON.parse(fileData);
    } else {
      return {};
    }
  }
}
  
//This is the callback function that collects all the data from the scripts
//that were run in parallel on each account. More details can be found here:
// http://goo.gl/BvOPZo
function reportResults(responses) {
  var summaryEmailData = [];
  var dateTimeStr = Utilities.formatDate(new Date(), AdWordsApp.currentAccount().getTimeZone(), 'yyyy-MM-dd HH:m:s');
  var spreadsheetName = SPREADSHEET_PREFIX+' - '+dateTimeStr;
  for(var i in responses) {
    if(!responses[i].getReturnValue()) { continue; }
    var res = JSON.parse(responses[i].getReturnValue());
    var sheetUrl = writeResultsToSpreadsheet(res,spreadsheetName);
    summaryEmailData.push({accountId:res.accountId,
                           accountName:res.accountName,
                           didExitEarly:res.didExitEarly,
                           uniqueUrlsChecked:res.uniqueUrlsChecked,
                           numBrokenKeywords:res.brokenKeywords.length,
                           numBrokenAds:res.brokenAds.length,
                           sheetUrl: sheetUrl});
  }
  if(summaryEmailData.length > 0) {
    sendSummaryEmail(summaryEmailData);
  }
    
  function writeResultsToSpreadsheet(res,name) {
    var file = getFile(name,true);
    var spreadsheet;
    var maxRetries = 0;
    while(maxRetries < 3) {
      try {
        spreadsheet = SpreadsheetApp.openById(file.getId());
        break;
      } catch(e) {
        maxRetries++;
        Utilities.sleep(1000);
      }
    }
    if(!spreadsheet) { throw 'Could not open file: '+name; }
    if(spreadsheet.getSheetByName('Sheet1')) {
      spreadsheet.getSheetByName('Sheet1').setName(res.accountId);
    }
    var sheet = spreadsheet.getSheetByName(res.accountId);
    if(!sheet) {
      sheet = spreadsheet.insertSheet(res.accountId, spreadsheet.getSheets().length);
    }
    var toWrite = [['Type','Clean Url','Response Code','Campaign Name','AdGroup Name','Text','Full Url']];
    for(var i in res.brokenKeywords) {
      var row = res.brokenKeywords[i];
      toWrite.push(['Keyword',
                    row.cleanUrl,
                    row.responseCode,
                    row.CampaignName,
                    row.AdGroupName,
                    row.Criteria,
                    row.FinalUrls]); 
    }
    for(var i in res.brokenAds) {
      var row = res.brokenAds[i];
      toWrite.push([row.AdType,
                    row.cleanUrl,
                    row.responseCode,
                    row.CampaignName,
                    row.AdGroupName,
                    (row.Headline) ? [row.Headline,row.Description1,row.Description2,row.DisplayUrl].join('|') : '',
                    row.CreativeFinalUrls]);
    }
    var lastRow = sheet.getLastRow();
    var numRows = sheet.getMaxRows();
    if((numRows-lastRow) < toWrite.length) {
      sheet.insertRowsAfter(lastRow,toWrite.length-numRows+lastRow);
    }
    var range = sheet.getRange(lastRow+1,1,toWrite.length,toWrite[0].length);
    range.setValues(toWrite);
    if((sheet.getMaxColumns() - sheet.getLastColumn()) > 0) {
      sheet.deleteColumns(sheet.getLastColumn()+1, sheet.getMaxColumns() - sheet.getLastColumn());
    }
    file = DriveApp.getFileById(spreadsheet.getId());
    try {
      file.setSharing(DriveApp.Access.ANYONE_WITH_LINK, DriveApp.Permission.VIEW);
    } catch(e) {
      file.setSharing(DriveApp.Access.DOMAIN_WITH_LINK, DriveApp.Permission.VIEW);
    }
    //This gives you a link directly to the spreadsheet sheet.
    return (spreadsheet.getUrl() + '#gid=' + sheet.getSheetId());
  }
    
  //This function builds the summary email and sends it to the people in
  //the NOTIFY list
  //This function builds the summary email and sends it to the people in
  //the NOTIFY list
  function sendSummaryEmail(summaryEmailData) {
    var subject = SCRIPT_NAME+' Summary Results';
    var body = subject;
    var htmlBody = '<html><body>'+subject;
    htmlBody += '<br/ >Should strip query strings: '+STRIP_QUERY_STRING;
    htmlBody += '<br/ >Report on redirects: '+REPORT_ON_REDIRECTS;
    htmlBody += '<br/ >Valid response codes: '+VALID_RESPONSE_CODES;
    htmlBody += '<br/ ><br/ >';
    htmlBody += '<table border="1" width="95%" style="border-collapse:collapse;">';
    htmlBody += '<tr>';
    htmlBody += '<td align="left"><b>Acct Id</b></td>';
    htmlBody += '<td align="left"><b>Acct Name</b></td>';
    htmlBody += '<td align="left"><b>Exited Early</b></td>';
    htmlBody += '<td align="center"><b>Unique Urls Checked</b></td>';
    htmlBody += '<td align="center"><b># Broken Keyword Urls</b></td>';
    htmlBody += '<td align="center"><b># Broken Ad Urls</b></td>';
    htmlBody += '<td align="center"><b>Full Report</b></td>';
    htmlBody += '</tr>';
    for(var i in summaryEmailData) {
      var row = summaryEmailData[i];
      htmlBody += '<tr><td align="left">'+ row.accountId +
                 '</td><td align="left">' + row.accountName + 
                 '</td><td align="left">' + row.didExitEarly + 
                 '</td><td align="center">' + row.uniqueUrlsChecked + 
                 '</td><td align="center">' + row.numBrokenKeywords + 
                 '</td><td align="center">' + row.numBrokenAds + 
                 '</td><td align="left"><a href="'+row.sheetUrl+'">' + 'Show Details' + 
                 '</a></td></tr>';
    }
    htmlBody += '</table>';
    htmlBody += '<br/ >';
    htmlBody += Utilities.formatDate(new Date(),AdWordsApp.currentAccount().getTimeZone(),'MMMM dd, yyyy @ hh:mma z');
    htmlBody += '.  Completed. '+Object.keys(summaryEmailData).length+' Accounts checked.';
    htmlBody += '</body></html>';
    var options = { htmlBody : htmlBody };
    for(var i in NOTIFY) {
      MailApp.sendEmail(NOTIFY[i], subject, body, options);
    }
  }
}
  
//This function finds a given file on Google Drive
//If it does not exist, it creates a new file
//if isSpreadsheet is set, it will create a new spreadsheet
//otherwise, it creates a text file.
function getFile(fileName,isSpreadsheet) {
  var maxRetries = 0;
  var errors = [];
  while(maxRetries < 3) {
    try {
      var fileIter = DriveApp.getFilesByName(fileName);
      if(!fileIter.hasNext()) {
        info('Could not find file: '+fileName+' on Google Drive. Creating new file.');
        if(isSpreadsheet) {
          return SpreadsheetApp.create(fileName);
        } else {
          return DriveApp.createFile(fileName,'');
        }
      } else {
        return fileIter.next();
      }
    } catch(e) {
      errors.push(e);
      maxRetries++;
      Utilities.sleep(1000);
    }
  }
  if(maxRetries == 3) {
    throw errors.join('. ');
  }
}
  
//Some functions to help with logging
var LOG_LEVELS = { 'error':1, 'warn':2, 'info':3, 'debug':4 };
function error(msg) { if(LOG_LEVELS['error'] <= LOG_LEVELS[LOG_LEVEL]) { log('ERROR',msg); } }
function warn(msg)  { if(LOG_LEVELS['warn']  <= LOG_LEVELS[LOG_LEVEL]) { log('WARN' ,msg); } }
function info(msg)  { if(LOG_LEVELS['info']  <= LOG_LEVELS[LOG_LEVEL]) { log('INFO' ,msg); } }
function debug(msg) { if(LOG_LEVELS['debug'] <= LOG_LEVELS[LOG_LEVEL]) { log('DEBUG',msg); } }
function log(type,msg) { Logger.log(type + ' - ' + msg); }

Beginner's Guide to Javascript You Should Know For AdWords Scripts

I've heard from a few readers that the posts on this blog have inspired them to learn to code. That's awesome! But I've also heard from a few that say they have run into trouble getting through many of the Javascript tutorials out there since they deal mostly with Javascript for web design or Node.js.

So this post is going to attempt to get someone up to speed with Javascript enough that they can at least walk through most of the code I post here and make changes when needed. Also, I aim to at least help you know what to Google when you get stuck.

A few caveats. Is this meant to be a replacement for a full coding class? No. Will I be making generalizations and over-simplifying some extremely complex topics? Yes. Are there mistakes in this? Probably. If you find one, let me know.

/*********************************
* Intro to Javascript For AdWords Scripts
* Version 1.0
* Created By: Russ Savage
* FreeAdWordsScripts.com
*********************************/
function main() {
  // This is a comment. AdWords Scripts ignores this
  /* Here is another way to comment
     that can be used when you need
     to comment multiple lines */
  
  // The main function tells AdWords where to start. You always need
  // at least a main function in your script.
  
  // Let's start with some variables (or primatives)
  // More info on Javascript variables can be found:
  // http://www.tutorialspoint.com/javascript/javascript_variables.htm
  var clubName = 'Fight Club'; // declared with single quotes
  var rule1 = "Don't talk about fight club."; // or double quotes if needed
  var members = 12; // a number, no quotes
  var dues = 3.50; // also a number
  var isAcceptingNewMembers = true; // a boolean, for yes or no answers
  
  // When you need to store multiple values, consider an Array
  // More detailed intro to Arrays can be found here:
  // http://www.w3schools.com/js/js_obj_array.asp
  var memberNames = ['brad','edward','robert'];
  // Which you can access the values with an index
  var coolestMember = memberNames[0]; // pronounced member names sub zero
  // 0 is the index of the first element of the array, 1 for the second, etc.
  // We can use the length property of an array to find out how big it is.
  var numberOfMembers = memberNames.length; // this will be 3
  var dailyFights = numberOfMembers*2; // star ( * ) is an operator for multiply
  // so the total number of fights is 6.
  // More on operators can be found here:
  // http://web.eecs.umich.edu/~bartlett/jsops.html
  
  // If you want to group multiple variables together, you can using an Object.
  // An Object is simply a grouping of common variables (and other stuff we'll see later)
  var FightClub = { // The curly brace says group these things together. there is another one at the end.
    clubName : 'The Fight Club', // a string variable. In an Object, we use : instead of = for assignment
    rules : ["Don't talk about fight club.",  // each variable is separated by a comma, instead of a semi-colon
             'Do not talk about fight club.'],
    memberNames : ['brad','eddy','robert','phil','dave'],
    dues : 3.50, 
    foundedYear : 1999
  };
  // Now to access the variables inside the object, we use the dot
  Logger.log(FightClub.clubName); // prints The Fight Club
  Logger.log(FightClub.memberNames[0]); // prints brad
  
  // Objects are one of the most important concepts of Javascript and they will come back
  // again and again a little later. More details can be found here:
  // https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Working_with_Objects
  
  // Sidebar: Why do I use camelCase for variable names? Technically
  // I could 
  var UsEWhaTevERIwanteD = 'but camelCase is easier to read';
  // and conforms to the style guide that Google recommends:
  // https://google-styleguide.googlecode.com/svn/trunk/javascriptguide.xml#Naming
  // Follow the style guide. It helps others read your code.
  
  // If statements (or control statements) allow you to split your code path if needed
  if(numberOfMembers > 10) { // if we have more than 10 members
    dues += 1.00; // increase the dues, 
    // plus equals (+=) says "add the value on the right to the value on the left"
  } else { // otherwise
    dues -= 1.00; // decrease the dues
    // there are also -=, *= (multiply), /= (divide by), and %= (modulo equals)
  }
  // Comparison operators like >, <, ==, ===, <=, >= allow you to compare values
  // They return true or false, always
  // Notice the double and triple equal signs. That's not a typo. More info can be found at:
  // http://www.impressivewebs.com/why-use-triple-equals-javascipt/
  
  // You can also have multiple if statements and multiple things to test
  if(dues > 5) { // if dues are over $5
    dailyFights++; // increase the fights
  } else if(dues > 2 && dues <= 5) { // if dues are greater than $2, but less than $5
    dailyFights--; // decrease the fights
  } else { // otherwise
    dailyFights = numberOfMembers*2; // reset the fights
  }
  // You'll probably notice none of this makes sense. it is only for example.
  // Double Ampersand && just means AND, || means OR. So in the statement above,
  // both statements with operators must be true in order for the fights to be decreased.
  // Oh, and ++, -- is shortcut for +=1 and -=1 respectively.
  
  // Ok, now lets talk about loops. 
  // Here are a few different ways to loop through the members
  // This is called a While Loop and while it might be easy to understand,
  // You won't use it nearly as often as the other two.
  var i = 0; // the variable i is what we will use for each indice
  while(i < memberNames.length) { // while i is less than the length of names
    Logger.log(memberNames[i]); // print out the name
    i++; // and increment the index by 1
  }
  // i is a variable that controls the loop. A common issue with While loops
  // is that you will forget to increment the loop control and you get an infinate loop
  
  // This is the classic For loop
  // The declaration, checking, and incrementing are all done 
  // in the first line so it is harder to miss them
  for(var index = 0; index < memberNames.length; index++) {
    Logger.log(memberNames[index]);
  }
  
  // And finally, the easiest loop but hardest to explain, the ForEach loop
  // This is just a variation of the For loop that handles incrementing index
  // behind the scenes so you don't have to.
  for(var index in memberNames) { // declare index, which will be assigned each indice
    Logger.log(memberNames[index]); // Use the indice to print each name
  }
  
  // You can jump out of a loop before it reaches the end by combining the if statement
  for(var index in memberNames) { 
    if(memberNames[index] === 'edward') {
      break; // break is a keyword you can use to break out of the loop.
    }
    Logger.log(memberNames[index]); 
  }
  // In this case, only the first name is printed because we broke out once we had the 
  // second name. More on break and its partner, continue, check out:
  // http://www.tutorialspoint.com/javascript/javascript_loop_control.htm
  
  // Now let's talk about functions. We have already seen a function in action: main()
  // Functions are groupings of useful code that you can call over and over again easily
  function fight(player1, player2) {
    if(Math.random() < .5) {
      return player1;
    } else {
      return player2; // return means we are going to send player2 back 
                      // to the code that called the function
    }
  }
  // This code can be called over and over again using a loop
  for(var player1 in memberNames) { // Loop through each member
    for(var player2 in memberNames) { // Then loop through again 
      if(player1 !== player2) { // Players can't fight themselves so check for that
        Logger.log(fight(player1,player2)); // Then call the function we defined earlier
      }
    }
  }
  // This code calls fight() for:
  //    brad vs. edward, brad vs. robert
  //    edward vs. brad, edward vs. robert
  //    robert vs. brad, robert vs. edward
  // Some other functions we have been calling are Logger.log() and Math.random()
  // The cool thing is that as callers of the function, we only need to know how
  // to call the function, we don't need to know how it works behind the scenes
  // For example:
  //   var answer = LargeHadronColider.simulateEleventhDimensionalQuantumThingy(47);
  // Who knows how this works. All we need to know is to send it a number and expect a
  // number back.
  
  // I hope you've been noticing all of the Objects we have been using here. Logger is one,
  // Math is another one (and LargeHadronColider is a fake one). Along with variables, we 
  // can also put functions in there as well:
  var FightClub = { 
    // ... all that other stuff
    chant : function() { 
      Logger.log('His name is Robert Paulson.'); 
    },
    totalMembers : 5
  };
  // Whoa trippy. So what happens when I call 
  FightClub.chant();
  // It's going to print His name is Robert Paulson
  
  // The thing that makes Google AdWords Scripts different from writing just regular Javascript
  // is all of the pre-defined Objects that use functions to interact with AdWords.
  AdWordsApp.currentAccount();
  Utilities.jsonParse('{}');
  AdWordsApp.keywords().withLimit(10).get();
  // How does the above statement work?
  AdWordsApp  // this is a predefined object in AdWords Scripts
    .keywords() // which has a function called keywords() that returns a KeywordSelector object
    .withLimit(10) // which has a function withLimit() that returns the same KeywordSelector object
    .get(); // which has a function get() that returns a KeywordIterator object.
  // Check out the AdWords Scripts documentation to find the objects and classes that make up these calls
  // https://developers.google.com/adwords/scripts/docs/reference/adwordsapp/adwordsapp
  // https://developers.google.com/adwords/scripts/docs/reference/adwordsapp/adwordsapp#keywords_0
  // https://developers.google.com/adwords/scripts/docs/reference/adwordsapp/adwordsapp_keywordselector
  // https://developers.google.com/adwords/scripts/docs/reference/adwordsapp/adwordsapp_keywordselector#withLimit_1
  // https://developers.google.com/adwords/scripts/docs/reference/adwordsapp/adwordsapp_keywordselector#get_0
  
  // So I think that just about does it for this tutorial.  If you made it this far, awesome! Post a comment to ask
  // any questions you might have.
  
  // Thanks,
  // Russ
}

Make Calls And Send Text Messages To Your Phone From AdWords Scripts

Many of the scripts that I write are about alerting SEM managers when there are issues with an account they manage. It is pretty easy to notify yourself and others through email using the MailApp, but what if you can't access your email for some reason?

To solve this, I have put together a very simple Javascript object that allows you to send SMS messages and make phone calls using Twilio. If you are unfamiliar with Twilio, they are a company that provides developers with an easy way to work telephony into their applications. Their API is extremely easy to use and their documentation is excellent.

In order to use this script, there are a few things you need to do. First, Sign Up for a Twilio Account. Once you do so, you should have a phone number that you can use to play around with as well as an Account SID and an Auth Token. Twilio might not be available in your country yet but hopefully they will be there soon. Make sure you are using the full phone numbers (including counry codes) when making your requests. If you are just using the free version, there may be usage limits but I'm not sure.

The code is set up so that it can be copied into any script where you need to send notifications. Once you've copied the Twilio object into your script, whenever you want the notification to be sent, you should add the following code:

  ...
  var sid = 'YOUR ACCOUNT SID GOES HERE';
  var auth = 'YOUR AUTH TOKEN GOES HERE';
  //First, create a new Twilio client
  var client = new Twilio(sid,auth);
  //Here is how you send a text message
  // First number is the receiver (most likely, your cell phone)
  // Second number is where is it coming from, which is the free number you got when
  //   you registered in Twilio
  // The third parameter is what you want the text or voice message to say
  client.sendMessage('+17245551234','+14155554321','WARNING: Your AdWords Account Is Not Serving Ads.');
  client.makeCall('+17245551234','+14155554321',
      'This is an automated call to warn you that your AdWords account is no longer serving ads.');
  ...

Of course, the sid, auth, and client can be global variables which would allow you to have a single line in your code to make phone calls or send messages. You could also set up some sort of escalation chain in case people miss the call or text.

This is just a simple example of starting to use UrlFetchApp to integrate AdWords scripts with 3rd party apps. If you have 3rd party apps you'd like me to try out, leave a note in the comments.

Thanks,
Russ

/*********************************
* Twilio Client Library
* Based on the Twilio REST API: https://www.twilio.com/docs/api/rest
* Version 1.0
* Created By: Russ Savage
* FreeAdWordsScripts.com
*********************************/
function Twilio(accountSid, authToken) {
  this.ACCOUNT_SID = accountSid;
  this.AUTH_TOKEN = authToken;
  
  this.MESSAGES_ENDPOINT = 'https://api.twilio.com/2010-04-01/Accounts/'+this.ACCOUNT_SID+'/Messages.json';
  this.CALLS_ENDPOINT = 'https://api.twilio.com/2010-04-01/Accounts/'+this.ACCOUNT_SID+'/Calls.json';

  this.sendMessage = function(to,from,body) {
    var httpOptions = {
      method : 'POST',
      payload : {
        To: to,
        From: from,
        Body: body
      },
      headers : getBasicAuth(this)
    };
    var resp = UrlFetchApp.fetch(this.MESSAGES_ENDPOINT, httpOptions).getContentText();
    return JSON.parse(resp)['sid'];
  }
  
  this.makeCall = function(to,from,whatToSay) {
    var url = 'http://proj.rjsavage.com/savageautomation/twilio_script/dynamicSay.php?alert='+encodeURIComponent(whatToSay);
    var httpOptions = {
      method : 'POST',
      payload : {
        To: to,
        From: from,
        Url: url
      },
      headers : getBasicAuth(this)
    };
    var resp = UrlFetchApp.fetch(this.CALLS_ENDPOINT, httpOptions).getContentText();
    return JSON.parse(resp)['sid'];
  }
  
  function getBasicAuth(context) {
    return {
      'Authorization': 'Basic ' + Utilities.base64Encode(context.ACCOUNT_SID+':'+context.AUTH_TOKEN)
    };
  }
}

Automated Creative Testing With Statistical Significance

In the spirit of the Holidays, I've got a little gift for everyone. This script will monitor your AdGroups for creative tests that have hit statistical significance and notify you with an email so that you can take action. The script keeps track of changes to any AdGroups so that it always knows when a new test has started without you having to keep track. It applies labels to your Ads and then notifies you via email when the tests have completed.

This script is based on my previous Ad Testing Script but has some additional features. Namely, instead of pausing stats based purely on a metric, this script calculates the statistical significance of the test before making a decision. The calculations are based on a blog post and spreadsheet from VisualWebsiteOptimizer that I modified to work with AdWords Ads.

I also incorporated the ability to link directly to the Campaign or AdGroup from the notification email which I posted about before. All you need to do is fill in the __c and __u parameters to enable that feature.

And finally, since this is probably the largest and most complicated script I've shared here, I used the logic from my post about Google Analytics reporting to fire a beacon whenever the script is ran. You are more than welcome to disable it (remove beacon(); from the second to the last line of the main function), but I would love to get an idea of just how many people are using this script.

Since this script is pretty big, and the formatting here can be a little iffy sometimes, you can also download it from GitHub.

Thanks as always for reading and have a fun and safe holiday.

Thanks,
Russ

/*********************************************
* Automated Creative Testing With Statistical Significance
* Version 2.1
* Changelog v2.1
*   - Fixed INVALID_PREDICATE_ENUM_VALUE
* Changelog v2.0
*   - Fixed bug in setting the correct date
*   - Script now uses a minimum visitors threshold 
*        per Ad instead of AdGroup
*   - Added the ability to add the start date as a label to AdGroups
*   - Added ability to check mobile and desktop ads separately
* Changelog v1.1.1 - Fixed bug with getDisplayUrl
* Changelog v1.1 
*   - Added ability to only run on some campaigns
*   - Fixed bug in info logging
* Russ Savage
* FreeAdWordsScripts.com
**********************************************/
var EXTRA_LOGS = true;
var TO = ['user@email.com'];
var CONFIDENCE_LEVEL = 95; // 90%, 95%, or 99% are most common
 
//If you only want to run on some campaigns, apply a label to them
//and put the name of the label here.  Leave blank to run on all campaigns.
var CAMPAIGN_LABEL = '';
 
//These two metrics are the components that make up the metric you
//want to compare. For example, this measures CTR = Clicks/Impressions
//Other examples might be:
// Cost Per Conv = Cost/Conversions
// Conversion Rate = Conversions/Clicks
// Cost Per Click = Cost/Clicks
var VISITORS_METRIC = 'Impressions';
var CONVERSIONS_METRIC = 'Clicks';
//This is the number of impressions the Ad needs to have in order
//to start measuring the results of a test.
var VISITORS_THRESHOLD = 100;

//Setting this to true to enable the script to check mobile ads
//against other mobile ads only. Enabling this will start new tests
//in all your AdGroups so only enable this after you have completed
//a testing cycle.
var ENABLE_MOBILE_AD_TESTING = false;

//Set this on the first run which should be the approximate last time
//you started a new creative test. After the first run, this setting
//will be ignored.
var OVERRIDE_LAST_TOUCHED_DATE = 'Jan 1, 2014';
 
var LOSER_LABEL = 'Loser '+CONFIDENCE_LEVEL+'% Confidence';
var CHAMPION_LABEL = 'Current Champion';

// Set this to true and the script will apply a label to 
// each AdGroup to let you know the date the test started
// This helps you validate the results of the script.
var APPLY_TEST_START_DATE_LABELS = true;
 
//These come from the url when you are logged into AdWords
//Set these if you want your emails to link directly to the AdGroup
var __c = '';
var __u = '';
 
function main() {
  createLabelIfNeeded(LOSER_LABEL,"#FF00FF"); //Set the colors of the labels here
  createLabelIfNeeded(CHAMPION_LABEL,"#0000FF"); //Set the colors of the labels here
   
  //Let's find all the AdGroups that have new tests starting
  var currentAdMap = getCurrentAdsSnapshot();
  var previousAdMap = getPreviousAdsSnapshot();
  if(previousAdMap) {
    currentAdMap = updateCurrentAdMap(currentAdMap,previousAdMap);
  }
  storeAdsSnapshot(currentAdMap);
  previousAdMap = null;
   
  //Now run through the AdGroups to find tests
   var agSelector = AdWordsApp.adGroups()
    .withCondition('CampaignStatus = ENABLED')
    .withCondition('AdGroupStatus = ENABLED')
    .withCondition('Status = ENABLED');
  if(CAMPAIGN_LABEL !== '') {
    var campNames = getCampaignNames();
    agSelector = agSelector.withCondition("CampaignName IN ['"+campNames.join("','")+"']");
  }
  var agIter = agSelector.get();
  var todayDate = getDateString(new Date(),'yyyyMMdd');
  var touchedAdGroups = [];
  var finishedEarly = false;
  while(agIter.hasNext()) {
    var ag = agIter.next();

    var numLoops = (ENABLE_MOBILE_AD_TESTING) ? 2 : 1;
    for(var loopNum = 0; loopNum < numLoops; loopNum++) {
      var isMobile = (loopNum == 1);
      var rowKey;
      if(isMobile) {
        info('Checking Mobile Ads in AdGroup: "'+ag.getName()+'"');
        rowKey = [ag.getCampaign().getId(),ag.getId(),'Mobile'].join('-');
      } else {
        info('Checking Ads in AdGroup: "'+ag.getName()+'"');
        rowKey = [ag.getCampaign().getId(),ag.getId()].join('-');
      }

      if(!currentAdMap[rowKey]) {  //This shouldn't happen
        warn('Could not find AdGroup: '+ag.getName()+' in current ad map.');
        continue; 
      }
      
      if(APPLY_TEST_START_DATE_LABELS) {
        var dateLabel;
        if(isMobile) {
          dateLabel = 'Mobile Tests Started: '+getDateString(currentAdMap[rowKey].lastTouched,'yyyy-MM-dd');
        } else {
          dateLabel = 'Tests Started: '+getDateString(currentAdMap[rowKey].lastTouched,'yyyy-MM-dd');
        }

        createLabelIfNeeded(dateLabel,"#8A2BE2");
        //remove old start date
        var labelIter = ag.labels().withCondition("Name STARTS_WITH '"+dateLabel.split(':')[0]+"'")
                                   .withCondition("Name != '"+dateLabel+"'").get();
        while(labelIter.hasNext()) {
          var label = labelIter.next();
          ag.removeLabel(label.getName());
          if(!label.adGroups().get().hasNext()) {
            //if there are no more entities with that label, delete it.
            label.remove();
          }
        }
        applyLabel(ag,dateLabel);
      }
          
      //Here is the date range for the test metrics
      var lastTouchedDate = getDateString(currentAdMap[rowKey].lastTouched,'yyyyMMdd');
      info('Last Touched Date: '+lastTouchedDate+' Todays Date: '+ todayDate);
      if(lastTouchedDate === todayDate) {
        //Special case where the AdGroup was updated today which means a new test has started.
        //Remove the old labels, but keep the champion as the control for the next test
        info('New test is starting in AdGroup: '+ag.getName());
        removeLoserLabelsFromAds(ag,isMobile);
        continue;
      }
      
      //Is there a previous winner? if so we should use it as the control.
      var controlAd = checkForPreviousWinner(ag,isMobile);
      
      //Here we order by the Visitors metric and use that as a control if we don't have one
      var adSelector = ag.ads().withCondition('Status = ENABLED').withCondition('AdType = TEXT_AD');
      if(!AdWordsApp.getExecutionInfo().isPreview()) {
        adSelector = adSelector.withCondition("LabelNames CONTAINS_NONE ['"+[LOSER_LABEL,CHAMPION_LABEL].join("','")+"']");
      }
      var adIter = adSelector.forDateRange(lastTouchedDate, todayDate)
                             .orderBy(VISITORS_METRIC+" DESC")
                             .get();
      if( (controlAd == null && adIter.totalNumEntities() < 2) ||
          (controlAd != null && adIter.totalNumEntities() < 1) )
      { 
        info('AdGroup did not have enough eligible Ads. Had: '+adIter.totalNumEntities()+', Needed at least 2'); 
        continue; 
      }
      
      if(!controlAd) {
        info('No control set for AdGroup. Setting one.');
        while(adIter.hasNext()) {
          var ad = adIter.next();
          if(shouldSkip(isMobile,ad)) { continue; }
          controlAd = ad;
          break;
        }
        if(!controlAd) {
          continue;
        }
        applyLabel(controlAd,CHAMPION_LABEL);
      }
      
      while(adIter.hasNext()) {
        var testAd = adIter.next();
        if(shouldSkip(isMobile,testAd)) { continue; }
        //The Test object does all the heavy lifting for us.
        var test = new Test(controlAd,testAd,
                            CONFIDENCE_LEVEL,
                            lastTouchedDate,todayDate,
                            VISITORS_METRIC,CONVERSIONS_METRIC);
        info('Control - Visitors: '+test.getControlVisitors()+' Conversions: '+test.getControlConversions());
        info('Test    - Visitors: '+test.getTestVisitors()+' Conversions: '+test.getTestConversions());
        info('P-Value: '+test.getPValue());
        
        if(test.getControlVisitors() < VISITORS_THRESHOLD ||
           test.getTestVisitors() < VISITORS_THRESHOLD)
        {
          info('Not enough visitors in the control or test ad.  Skipping.');
          continue;
        }
        
        //Check for significance
        if(test.isSignificant()) {
          var loser = test.getLoser();
          removeLabel(loser,CHAMPION_LABEL); //Champion has been dethroned
          applyLabel(loser,LOSER_LABEL);
          
          //The winner is the new control. Could be the same as the old one.
          controlAd = test.getWinner();
          applyLabel(controlAd,CHAMPION_LABEL);
          
          //We store some metrics for a nice email later
          if(!ag['touchCount']) {
            ag['touchCount'] = 0;
            touchedAdGroups.push(ag);
          }
          ag['touchCount']++;
        }
      }
      
      //Let's bail if we run out of time so we can send the emails.
      if((!AdWordsApp.getExecutionInfo().isPreview() && AdWordsApp.getExecutionInfo().getRemainingTime() < 60) ||
         ( AdWordsApp.getExecutionInfo().isPreview() && AdWordsApp.getExecutionInfo().getRemainingTime() < 10) )
      {
        finishedEarly = true;
        break;
      }
    }
  }
  if(touchedAdGroups.length > 0) {
    sendMailForTouchedAdGroups(touchedAdGroups,finishedEarly);
  }
  beacon();
}
 
// A helper function to return the list of campaign ids with a label for filtering 
function getCampaignNames() {
  var campNames = [];
  var labelIter = AdWordsApp.labels().withCondition("Name = '"+CAMPAIGN_LABEL+"'").get();
  if(labelIter.hasNext()) {
    var label = labelIter.next();
    var campIter = label.campaigns().get();
    while(campIter.hasNext()) {
      campNames.push(campIter.next().getName()); 
    }
  }
  return campNames;
}
 
function applyLabel(entity,label) {
  if(!AdWordsApp.getExecutionInfo().isPreview()) {
    entity.applyLabel(label);
  } else {
    var adText = (entity.getEntityType() === 'Ad') ? [entity.getHeadline(),entity.getDescription1(),
                                                      entity.getDescription2(),entity.getDisplayUrl()].join(' ') 
                                                   : entity.getName();
    Logger.log('PREVIEW: Would have applied label: '+label+' to Entity: '+ adText);
  }
}
 
function removeLabel(ad,label) {
  if(!AdWordsApp.getExecutionInfo().isPreview()) {
    ad.removeLabel(label);
  } else {
    var adText = [ad.getHeadline(),ad.getDescription1(),ad.getDescription2(),ad.getDisplayUrl()].join(' ');
    Logger.log('PREVIEW: Would have removed label: '+label+' from Ad: '+ adText);
  }
}
 
// This function checks if the AdGroup has an Ad with a Champion Label
// If so, the new test should use that as the control.
function checkForPreviousWinner(ag,isMobile) {
  var adSelector = ag.ads().withCondition('Status = ENABLED')
                           .withCondition('AdType = TEXT_AD');
  if(!AdWordsApp.getExecutionInfo().isPreview()) {
    adSelector = adSelector.withCondition("LabelNames CONTAINS_ANY ['"+CHAMPION_LABEL+"']");
  }
  var adIter = adSelector.get();
  while(adIter.hasNext()) {
    var ad = adIter.next();
    if(shouldSkip(isMobile,ad)) { continue; }
    info('Found a previous winner. Using it as the control.');
    return ad;
  }
  return null;
}

function shouldSkip(isMobile,ad) {
  if(isMobile) {
    if(!ad.isMobilePreferred()) {
      return true;
    }
  } else {
    if(ad.isMobilePreferred()) {
      return true;
    }
  }
  return false;
}
 
// This function sends the email to the people in the TO array.
// If the script finishes early, it adds a notice to the email.
function sendMailForTouchedAdGroups(ags,finishedEarly) {
  var htmlBody = '<html><head></head><body>';
  if(finishedEarly) {
    htmlBody += 'The script was not able to check all AdGroups. ' +
                'It will check additional AdGroups on the next run.<br / >' ;
  }
  htmlBody += 'The following AdGroups have one or more creative tests that have finished.' ;
  htmlBody += buildHtmlTable(ags);
  htmlBody += '<p><small>Generated by <a href="http://www.freeadwordsscripts.com">FreeAdWordsScripts.com</a></small></p>' ;
  htmlBody += '</body></html>';
  var options = { 
    htmlBody : htmlBody,
  };
  var subject = ags.length + ' Creative Test(s) Completed - ' + 
    Utilities.formatDate(new Date(), AdWordsApp.currentAccount().getTimeZone(), 'yyyy-MM-dd');
  for(var i in TO) {
    MailApp.sendEmail(TO[i], subject, ags.length+' AdGroup(s) have creative tests that have finished.', options);
  }
}

// This function uses my HTMLTable object to build the styled html table for the email.
function buildHtmlTable(ags) {
  var table = new HTMLTable();
  //CSS from: http://coding.smashingmagazine.com/2008/08/13/top-10-css-table-designs/
  //Inlined using: http://inlinestyler.torchboxapps.com/
  table.setTableStyle(['font-family: "Lucida Sans Unicode","Lucida Grande",Sans-Serif;',
                       'font-size: 12px;',
                       'background: #fff;',
                       'margin: 45px;',
                       'width: 480px;',
                       'border-collapse: collapse;',
                       'text-align: left'].join(''));
  table.setHeaderStyle(['font-size: 14px;',
                        'font-weight: normal;',
                        'color: #039;',
                        'padding: 10px 8px;',
                        'border-bottom: 2px solid #6678b1'].join(''));
  table.setCellStyle(['border-bottom: 1px solid #ccc;',
                      'padding: 4px 6px'].join(''));
  table.addHeaderColumn('#');
  table.addHeaderColumn('Campaign Name');
  table.addHeaderColumn('AdGroup Name');
  table.addHeaderColumn('Tests Completed');
  for(var i in ags) {
    table.newRow();
    table.addCell(table.getRowCount());
    var campName = ags[i].getCampaign().getName();
    var name = ags[i].getName();
    var touchCount = ags[i]['touchCount'];
    var campLink, agLink;
    if(__c !== '' && __u !== '') { // You should really set these.
      campLink = getUrl(ags[i].getCampaign(),'Ad groups');
      agLink = getUrl(ags[i],'Ads');
      table.addCell(a(campLink,campName));
      table.addCell(a(agLink,name));
    } else {
      table.addCell(campName);
      table.addCell(name);
    }
    table.addCell(touchCount,'text-align: right');
  }
  return table.toString();
}

// Just a helper to build the html for a link.
function a(link,val) {
  return '<a href="'+link+'">'+val+'</a>';
}
 
// This function finds all the previous losers and removes their label.
// It is used when the script detects a change in the AdGroup and needs to 
// start a new test.
function removeLoserLabelsFromAds(ag,isMobile) {
  var adSelector = ag.ads().withCondition('Status = ENABLED');
  if(!AdWordsApp.getExecutionInfo().isPreview()) {
    adSelector = adSelector.withCondition("LabelNames CONTAINS_ANY ['"+LOSER_LABEL+"']");
  }
  var adIter = adSelector.get();
  while(adIter.hasNext()) {
    var ad = adIter.next();
    if(shouldSkip(isMobile,ad)) { continue; }
    removeLabel(ad,LOSER_LABEL);
  }
}
 
// A helper function to create a new label if it doesn't exist in the account.
function createLabelIfNeeded(name,color) {
  if(!AdWordsApp.labels().withCondition("Name = '"+name+"'").get().hasNext()) {
    info('Creating label: "'+name+'"');
    AdWordsApp.createLabel(name,"",color);
  } else {
    info('Label: "'+name+'" already exists.');
  }
}
 
// This function compares the previous and current Ad maps and
// updates the current map with the date that the AdGroup was last touched.
// If OVERRIDE_LAST_TOUCHED_DATE is set and there is no previous data for the 
// AdGroup, it uses that as the last touched date.
function updateCurrentAdMap(current,previous) {
  info('Updating the current Ads map using historical snapshot.');
  for(var rowKey in current) {
    var currentAds = current[rowKey].adIds;
    var previousAds = (previous[rowKey]) ? previous[rowKey].adIds : [];
    if(currentAds.join('-') === previousAds.join('-')) {
      current[rowKey].lastTouched = previous[rowKey].lastTouched;
    }
    if(previousAds.length === 0 && OVERRIDE_LAST_TOUCHED_DATE !== '') {
      current[rowKey].lastTouched = new Date(OVERRIDE_LAST_TOUCHED_DATE);
    }
    //if we make it here without going into the above if statements
    //then the adgroup has changed and we should keep the new date
  }
  info('Finished updating the current Ad map.');
  return current;
}
 
// This stores the Ad map snapshot to a file so it can be used for the next run.
// The data is stored as a JSON string for easy reading later.
function storeAdsSnapshot(data) {
  info('Storing the Ads snapshot to Google Drive.');
  var fileName = getSnapshotFilename();
  var file = DriveApp.getFilesByName(fileName).next();
  file.setContent(Utilities.jsonStringify(data));
  info('Finished.');
}
 
// This reads the JSON formatted previous snapshot from a file on GDrive
// If the file doesn't exist, it creates a new one and returns an empty map.
function getPreviousAdsSnapshot() {
  info('Loading the previous Ads snapshot from Google Drive.');
  var fileName = getSnapshotFilename();
  var fileIter = DriveApp.getFilesByName(fileName);
  if(fileIter.hasNext()) {
    return Utilities.jsonParse(fileIter.next().getBlob().getDataAsString());
  } else {
    DriveApp.createFile(fileName, '');
    return {};
  }
}
 
// A helper function to build the filename for the snapshot.
function getSnapshotFilename() {
  var accountId = AdWordsApp.currentAccount().getCustomerId();
  return (accountId + ' Ad Testing Script Snapshot.json');
}
 
// This function pulls the Ad Performance Report which is the fastest
// way to build a snapshot of the current ads in the account.
// This only pulls in active text ads.
function getCurrentAdsSnapshot() {
  info('Running Ad Performance Report to get current Ads snapshot.');
  var OPTIONS = { includeZeroImpressions : true };
  var cols = ['CampaignId','AdGroupId','Id','DevicePreference','Impressions'];
  var report = 'AD_PERFORMANCE_REPORT';
  var query = ['select',cols.join(','),'from',report,
               'where AdType = TEXT_AD',
               'and AdNetworkType1 = SEARCH',
               'and CampaignStatus = ENABLED',
               'and AdGroupStatus = ENABLED',
               'and Status = ENABLED',
               'during','TODAY'].join(' ');
  var results = {}; // { campId-agId : row, ... }
  var reportIter = AdWordsApp.report(query, OPTIONS).rows();
  while(reportIter.hasNext()) {
    var row = reportIter.next();
    var rowKey = [row.CampaignId,row.AdGroupId].join('-');
    if(ENABLE_MOBILE_AD_TESTING && row.DevicePreference == 30001) {
      rowKey += '-Mobile';
    }
    if(!results[rowKey]) {
      results[rowKey] = { adIds : [], lastTouched : new Date() };
    }
    results[rowKey].adIds.push(row.Id);
  }
  for(var i in results) {
    results[i].adIds.sort();
  }
  info('Finished building the current Ad map.');
  return results;
}
 
//Helper function to format the date
function getDateString(date,format) {
  return Utilities.formatDate(new Date(date),AdWordsApp.currentAccount().getTimeZone(),format); 
}
 
// Function to build out the urls for deeplinking into the AdWords account.
// For this to work, you need to have __c and __u filled in.
// Taken from: http://www.freeadwordsscripts.com/2013/11/building-entity-deep-links-with-adwords.html
function getUrl(entity,tab) {
  var customerId = __c;
  var effectiveUserId = __u;
  var decodedTab = getTab(tab);  
    
  var base = 'https://adwords.google.com/cm/CampaignMgmt?';
  var url = base+'__c='+customerId+'&__u='+effectiveUserId+'#';
   
  if(typeof entity['getEntityType'] === 'undefined') {
    return url+'r.ONLINE.di&app=cm';
  }
   
  var type = entity.getEntityType()
  if(type === 'Campaign') {
    return url+'c.'+entity.getId()+'.'+decodedTab+'&app=cm';
  }
  if(type === 'AdGroup') {
    return url+'a.'+entity.getId()+'_'+entity.getCampaign().getId()+'.'+decodedTab+'&app=cm';
  }
  if(type === 'Keyword') {
    return url+'a.'+entity.getAdGroup().getId()+'_'+entity.getCampaign().getId()+'.key&app=cm';
  }
  if(type === 'Ad') {
    return url+'a.'+entity.getAdGroup().getId()+'_'+entity.getCampaign().getId()+'.create&app=cm';
  }
  return url+'r.ONLINE.di&app=cm';
    
  function getTab(tab) {
    var mapping = {
      'Ad groups':'ag','Settings:All settings':'st_sum',
      'Settings:Locations':'st_loc','Settings:Ad schedule':'st_as',
      'Settings:Devices':'st_p','Ads':'create',
      'Keywords':'key','Audiences':'au','Ad extensions':'ae',
      'Auto targets':'at','Dimensions' : 'di'
    };
    if(mapping[tab]) { return mapping[tab]; }
    return 'key'; //default to keyword tab
  }
}
 
// Helper function to print info logs
function info(msg) {
  if(EXTRA_LOGS) {
    Logger.log('INFO: '+msg);
  }
}
 
// Helper function to print more serious warnings
function warn(msg) {
  Logger.log('WARNING: '+msg);
}
 
/********************************
* Track Script Runs in Google Analytics
* Created By: Russ Savage
* FreeAdWordsScripts.com
********************************/
function beacon() {
  var TAG_ID = 'UA-40187672-2';
  var CAMPAIGN_SOURCE = 'adwords';
  var CAMPAIGN_MEDIUM = 'scripts';
  var CAMPAIGN_NAME = 'AdTestingScriptV2_1';
  var HOSTNAME = 'www.freeadwordsscripts.com';
  var PAGE = '/Ad_Testing_Script_v2_1';
  if(AdWordsApp.getExecutionInfo().isPreview()) {
    PAGE += '/preview';
  }
  var DOMAIN_LINK = 'http://'+HOSTNAME+PAGE;
  
  //Pulled from: http://stackoverflow.com/questions/105034/how-to-create-a-guid-uuid-in-javascript
  var uuid = 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, 
    function(c) {var r = Math.random()*16|0,v=c=='x'?r:r&0x3|0x8;return v.toString(16);});
    
  var url = 'http://www.google-analytics.com/collect?';
  var payload = {
    'v':1,'tid':TAG_ID,'cid':uuid,    
    't':'pageview','cs':CAMPAIGN_SOURCE,'cm':CAMPAIGN_MEDIUM,'cn':CAMPAIGN_NAME,
    'dl':DOMAIN_LINK
  };
  var qs = '';
  for(var key in payload) {
    qs += key + '=' + encodeURIComponent(payload[key]) + '&';
  }
  url += qs.substring(0,qs.length-1);
  UrlFetchApp.fetch(url);
}
 
/*********************************************
* Test: A class for runnning A/B Tests for Ads
* Version 1.0
* Based on VisualWebsiteOptimizer logic: http://goo.gl/jiImn
* Russ Savage
* FreeAdWordsScripts.com
**********************************************/
// A description of the parameters:
// control - the control Ad, test - the test Ad
// startDate, endDate - the start and end dates for the test
// visitorMetric, conversionMetric - the components of the metric to use for the test
function Test(control,test,desiredConf,startDate,endDate,visitorMetric,conversionMetric) {
  this.desiredConfidence = desiredConf/100;
  this.verMetric = visitorMetric;
  this.conMetric = conversionMetric;
  this.startDate = startDate;
  this.endDate = endDate;
  this.winner;
   
  this.controlAd = control;
  this.controlStats = (this.controlAd['stats']) ? this.controlAd['stats'] : this.controlAd.getStatsFor(this.startDate, this.endDate);
  this.controlAd['stats'] = this.controlStats;
  this.controlVisitors = this.controlStats['get'+this.verMetric]();
  this.controlConversions = this.controlStats['get'+this.conMetric]();
  this.controlCR = getConversionRate(this.controlVisitors,this.controlConversions);
   
  this.testAd = test;
  this.testStats = (this.testAd['stats']) ? this.testAd['stats'] : this.testAd.getStatsFor(this.startDate, this.endDate);
  this.testAd['stats'] = this.testStats;
  this.testVisitors = this.testStats['get'+this.verMetric]();
  this.testConversions = this.testStats['get'+this.conMetric]();
  this.testCR = getConversionRate(this.testVisitors,this.testConversions);
   
  this.pValue;
   
  this.getControlVisitors = function() { return this.controlVisitors; }
  this.getControlConversions = function() { return this.controlConversions; }
  this.getTestVisitors = function() { return this.testVisitors; }
  this.getTestConversions = function() { return this.testConversions; }
   
  // Returns the P-Value for the two Ads
  this.getPValue = function() {
    if(!this.pValue) {
      this.pValue = calculatePValue(this);
    }
    return this.pValue;
  };
   
  // Determines if the test has hit significance
  this.isSignificant = function() {
    var pValue = this.getPValue();
    if(pValue && pValue !== 'N/A' && (pValue >= this.desiredConfidence || pValue <= (1 - this.desiredConfidence))) {
      return true;
    }
    return false;
  }
   
  // Returns the winning Ad
  this.getWinner = function() {
    if(this.decideWinner() === 'control') {
      return this.controlAd;
    }
    if(this.decideWinner() === 'challenger') {
      return this.testAd;
    }
    return null;
  };
   
  // Returns the losing Ad
  this.getLoser = function() {
    if(this.decideWinner() === 'control') {
      return this.testAd;
    }
    if(this.decideWinner() === 'challenger') {
      return this.controlAd;
    }
    return null;
  };
   
  // Determines if the control or the challenger won
  this.decideWinner = function () {
    if(this.winner) {
      return this.winner;
    }
    if(this.isSignificant()) {
      if(this.controlCR >= this.testCR) {
        this.winner = 'control';
      } else {
        this.winner = 'challenger';
      }
    } else {
      this.winner = 'no winner';
    }
    return this.winner;
  }
   
  // This function returns the confidence level for the test
  function calculatePValue(instance) {
    var control = { 
      visitors: instance.controlVisitors, 
      conversions: instance.controlConversions,
      cr: instance.controlCR
    };
    var challenger = { 
      visitors: instance.testVisitors, 
      conversions: instance.testConversions,
      cr: instance.testCR
    };
    var z = getZScore(control,challenger);
    if(z == -1) { return 'N/A'; }
    var norm = normSDist(z);
    return norm;
  }
   
  // A helper function to make rounding a little easier
  function round(value) {
    var decimals = Math.pow(10,5);
    return Math.round(value*decimals)/decimals;
  }
   
  // Return the conversion rate for the test
  function getConversionRate(visitors,conversions) {
    if(visitors == 0) {
      return -1;
    }
    return conversions/visitors;
  }
   
  function getStandardError(cr,visitors) {
    if(visitors == 0) {
      throw 'Visitors cannot be 0.';
    }
    return Math.sqrt((cr*(1-cr)/visitors));
  }
   
  function getZScore(c,t) {
    try {
      if(!c['se']) { c['se'] = getStandardError(c.cr,c.visitors); }
      if(!t['se']) { t['se'] = getStandardError(t.cr,t.visitors); }
    } catch(e) {
      Logger.log(e);
      return -1;
    }
     
    if((Math.sqrt(Math.pow(c.se,2)+Math.pow(t.se,2))) == 0) { 
      Logger.log('WARNING: Somehow the denominator in the Z-Score calulator was 0.');
      return -1;
    }
    return ((c.cr-t.cr)/Math.sqrt(Math.pow(c.se,2)+Math.pow(t.se,2)));
  }
   
  //From: http://www.codeproject.com/Articles/408214/Excel-Function-NORMSDIST-z
  function normSDist(z) {
    var sign = 1.0;
    if (z < 0) { sign = -1; }
    return round(0.5 * (1.0 + sign * erf(Math.abs(z)/Math.sqrt(2))));
  }
   
  // From: http://picomath.org/javascript/erf.js.html
  function erf(x) {
    // constants
    var a1 =  0.254829592;
    var a2 = -0.284496736;
    var a3 =  1.421413741;
    var a4 = -1.453152027;
    var a5 =  1.061405429;
    var p  =  0.3275911;
     
    // Save the sign of x
    var sign = 1;
    if (x < 0) {
      sign = -1;
    }
    x = Math.abs(x);
     
    // A&S formula 7.1.26
    var t = 1.0/(1.0 + p*x);
    var y = 1.0 - (((((a5*t + a4)*t) + a3)*t + a2)*t + a1)*t*Math.exp(-x*x);
     
    return sign*y;
  }
}
 
/*********************************************
* HTMLTable: A class for building HTML Tables
* Version 1.0
* Russ Savage
* FreeAdWordsScripts.com
**********************************************/
function HTMLTable() {
  this.headers = [];
  this.columnStyle = {};
  this.body = [];
  this.currentRow = 0;
  this.tableStyle;
  this.headerStyle;
  this.cellStyle;
  
  this.addHeaderColumn = function(text) {
    this.headers.push(text);
  };
  
  this.addCell = function(text,style) {
    if(!this.body[this.currentRow]) {
      this.body[this.currentRow] = [];
    }
    this.body[this.currentRow].push({ val:text, style:(style) ? style : '' });
  };
  
  this.newRow = function() {
    if(this.body != []) {
      this.currentRow++;
    }
  };
  
  this.getRowCount = function() {
    return this.currentRow;
  };
  
  this.setTableStyle = function(css) {
    this.tableStyle = css;
  };
  
  this.setHeaderStyle = function(css) {
    this.headerStyle = css; 
  };
  
  this.setCellStyle = function(css) {
    this.cellStyle = css;
    if(css[css.length-1] !== ';') {
      this.cellStyle += ';';
    }
  };
  
  this.toString = function() {
    var retVal = '<table ';
    if(this.tableStyle) {
      retVal += 'style="'+this.tableStyle+'"';
    }
    retVal += '>'+_getTableHead(this)+_getTableBody(this)+'</table>';
    return retVal;
  };
  
  function _getTableHead(instance) {
    var headerRow = '';
    for(var i in instance.headers) {
      headerRow += _th(instance,instance.headers[i]);
    }
    return '<thead><tr>'+headerRow+'</tr></thead>';
  };
  
  function _getTableBody(instance) {
    var retVal = '<tbody>';
    for(var r in instance.body) {
      var rowHtml = '<tr>';
      for(var c in instance.body[r]) {
        rowHtml += _td(instance,instance.body[r][c]);
      }
      rowHtml += '</tr>';
      retVal += rowHtml;
    }
    retVal += '</tbody>';
    return retVal;
  };
  
  function _th(instance,val) {
    var retVal = '<th scope="col" ';
    if(instance.headerStyle) {
      retVal += 'style="'+instance.headerStyle+'"';
    }
    retVal += '>'+val+'</th>';
    return retVal;
  };
  
  function _td(instance,cell) {
    var retVal = '<td ';
    if(instance.cellStyle || cell.style) {
      retVal += 'style="';
      if(instance.cellStyle) {
        retVal += instance.cellStyle;
      }
      if(cell.style) {
        retVal += cell.style;
      }
      retVal += '"';
    }
    retVal += '>'+cell.val+'</td>';
    return retVal;
  };
}