When you automate your email marketing, it's often referred to as "Set It and Forget It" but this is actually bad advice. You've established a repeatable process and through automation, you're able to deliver a timely, relevant and engaging experience to your audience. This experience might be a single email or a welcome treatment stream - it might include outbound phone calls or printed postal pieces or the shipping of a physical good.
And then it just runs, day in and day out, no more effort needed, right? Wrong. Your automation can and probably will fail at some point. It's up-to-you to regularly audit your own programs and make sure they're still working as intended. (These are also great places to conduct testing - you can let them run until you reach statistical significance, provided you have enough volume to get there in a reasonable amount of time.)
Automations are great. What could possibly go wrong?
Systems Failure
Things happen. Even the best systems will fail from time-to-time. Data that doesn't get sent on time, latency in the network, a software glitch, you name it. You should be receiving regular reports from your service provider as to the performance of your automation, probably on a daily basis, or in a real-time dashboard. If you can swing it, monitoring with proactive escalation may be warranted if you have a high enough volume that any failure causes load on your inboard customer service (people calling for a missing receipt or booking confirmation.)
You want to know that it's still working, that it's still sending the number of emails you expect it to and that those emails are still performing consistently. If you haven't had a sale and suddenly there's a lot more (or less) sent, you'll want to investigate. If you had a pretty consistent open or click rate and you suddenly it changes (and you didn't change anything) you should investigate why.
Relevancy Failure
Many a snarky aside in email or website (or other media) has been rendered "heartless" when events completely outside their control changed the tone of the subject matter. While not email-specific, on 9/11, many news websites broke under the load of traffic and fixed a lot otherwise dynamic content. For several hours as the tragedy unfolded, one news site (part of an entertainment conglomerate) had the unfortunate problem of a single ad stuck at the top of every news story -- it was promoting a new TV show. It showed the a guy from the chest up, leaned back in a chair or on laying in the grass or something (sorry, I can't recall) - but he had his hands clasped behind his head looking quite relaxed with a smile on his face with the tagline proudly promoting "Life is good." Ouch. The TV show didn't make it a season, the news network has changed hands and 14-years later I still remember that unfortunately placed advertisement.
Just as important - some of your automated treatment streams will be encountered by your audience more than once. You can aim for a static experience that never changes (simple, classic elegance) or you can treat it as an opportunity to entertain, engage, excite or upsell your audience. Regularly review to make sure offers are still relevant, or that you're changing it up for each season so there's a new reason for your customer to open that receipt email.
Content Failure
Here's a well-known company well known for its attention to detail. Their shipping update email is marred because of a single pixel transparent image that someone deleted off a server somewhere. (This is a common trick most email marketers use - these small "shims" as a way to force spacing in emails. The trick of borrowing and reusing elements is also pretty common - but there's a risk if someone comes along who's not aware of the purpose of a particular element, especially if it's just hidden amongst a bunch of other stuff that's clearly years old and would never be used again.)
What if it had been their logo that was deleted? Or replaced with one of different dimensions by another designer not realizing it was used in these emails? (Bonus points: this vendor's checkout gives you the option to apply for store credit which gives you 24-months to pay, interest-free and even calculates your monthly payment. Their banking partner who actually extends the credit is actually only giving you 18-months to pay, interest-free. Oops.)
Data Failure
To make automated emails relevant, there's often some logic - either a simple fill-in-the-blanks "you ordered x, we charged your card and will ship it to y" or more complex "if they are this kind of customer, show this kind of data, unless they just did this particular action in the past three days" - any change in your data (or missing data) could break your automation, or cause unintended consequences. It's important to plan for edge cases (or how to fail gracefully with missing or erroneous data - like a string or a null when a number is expected) but it's also important to assess the impact to automations at any time in the future when you change or add new data fields.
Bottom Line: Check It
Your automations are inexpensive customer service touchpoints designed to reinforce your brand, educate (and retain) your customer or encourage them to come back and spend more money with you. They may reduce your customer service calls by proactively addressing issues before the customer calls you or deepens engagement by helping them to explore more of what they've purchased.
In some really streamlined cases, this experience may be the strongest interaction your customer has with your brand, not counting how the UPS-driver treats them when delivering a package.
It is your duty to be very aware of how your automations are working, how they are performing and that they are still relevant.
Daily: Review yesterday's statistics. How many were sent? Does that match expectations? (Is that how many API calls your system made, or how many abandoned or completed shopping carts you recorded?) If you have inbox monitoring, you should also receive a daily report to make sure your emails aren't getting junked.
Weekly: Review performance metrics. What's the open or click rate? Do the numbers fall within the acceptable range? If you've changed something or are testing, how are the results trending?
Monthly: Run through your use cases again by putting email addresses into the automation. Review the results. Give someone the task of trying to break the automation by introducing records with erroneous or missing data.
Quarterly: Audit the content, If you change it regularly, plan out the next 3-6 months and assess existing plans for relevancy. Run the content through a rendering tool like Litmus or Email on Acid to make sure changes by email vendors haven't mangled what was previously a good looking email.
Six Months (or more frequently if possible): Look at your nearest competitors and peers are doing - have they made any changes you can learn from?
Home Improvement, Maintenance and Repair is a tricky business. Most of your new work is from word-of-mouth - I need something done so I ask my friends for recommendations. Most will have a recommendation or two that they're excited to share, or maybe a word of caution against an experience that that didn't go so well. People don't think about it until they need it, they don't appreciate how difficult it is to estimate before you've cut open a wall and they have really exacting standards and expectations.