1. If you can check the database and it is failed try giving the job some rights and have it issue a NET STOP OnTimeEmailService,  pause 10 seconds to close any open files, then issue a NET START OnTimeEmailService.

2. For the times where the service actually tells the system it has failed have your server admins set the service property tab named "recovery" to restart the service for the first failure after 5 minutes and second failure after 10 minutes and reset the fail count after a day. That works when the services are smart enough to tell the system they died. That gives the SMTP/SQL servers time to come back online.

3. We have one application that we stop and restart the services on every day. When stopped we roll off the log files using namedate.exe, purge log files older than 4 months using robocopy, and restart everything. Two services that used to fail every few weeks have never failed since. Turned out to be a memory leak in the services but the service restart and log file management was worth keeping in place. It only takes a minute to run.

Arash,

Did anything ever come of this?  It just happened to me again today.  Weird thing this time is that we have a second account now and it was still happily checking away, but our main account was not checking.  Nothing interesting in windows event log.

Chip 

Hi Chip,

I asked everyone to email me their exception log from ontime and event viewer log related to ontime. The couple that were sent to me detailed that the services couldn't connect to the sql server. Your situation seems a little different because only one of the two accounts stopped working. Are you checking them at the same interval? How often is this happening for you? Next time this happens before fixing the account (by doing a manual check) please look into your "mailaccounts" table at "lockdatetime" field. If the account is locked there will be a date and time value otherwise Null. If a date and time value is showing it means the service failed while checking that account. The next time it checks the service will skip that account because its locked but when you manually run the account it resets that value to Null if successful. That's why the account starts working after doing a manual check on the account.

I would note the value in the lockdatetime field (please let us know too) and see if there is a pattern and/or related errors?  

For our next release we are looking into notifying the administrator if the account is locked and the interval has exceeded ie 2 or 3 attempts. We are also adding some code that might fix the issue but we are not exactly sure what is causing this (besides not being able to connect to the sql server).

Thanks,
Arash

Arasha,

 It has happened again.  The account is shown as locked in the MailAccounts table.  The lock is showing 2008-04-27 01:01:48.280 and the last item in the Exceptions table is dated 2008-04-25 12:22:21.563.  My check interval is 5 minutes, so I doubt those are related.  Something did happen different this time, I got an error in Windows Event log on the server:

Event Type:    Error
Event Source:    OnTime
Event Category:    None
Event ID:    0
Date:        4/27/2008
Time:        1:02:11 AM
User:        N/A
Computer:    RVLOGIC-SVR01
Description:


1) Exception Information
*********************************************
Exception Type: System.Data.SqlClient.SqlException
Errors: System.Data.SqlClient.SqlErrorCollection
Class: 11
LineNumber: 0
Number: -2
Procedure:
Server: (local)
State: 0
Source: .Net SqlClient Data Provider
ErrorCode: -2146232060
Message: Timeout expired.  The timeout period elapsed prior to completion of the operation or the server is not responding.
Data: System.Collections.ListDictionaryInternal
TargetSite: Void OnError(System.Data.SqlClient.SqlException, Boolean)
HelpLink: NULL

I left out the Stack Trace to keep this message from being too long.  I can email it if you think it would help at all, but I'm guessing this is the exception you mentioned above. 

 Interestingly, the second account is working fine.  And to answer your other question, the check interval is different, it is 10 minutes.

Chip