Lost Entries

I’ve just finished the last of several conferences with our Microsoft support engineers this week, and despite everything that our staff and their staff have tried, they have concluded that it will be impossible to restore the data file that was corrupted last Sunday.

Unfortunately, this means that we do not have any more options for restoring the entries that were lost from our first 100 diaries last week. I know that this is no consolation to those of you who lost entries, but we spent a large amount of time, effort, and money trying to solve this – I really believe that we did everything humanly possible to fix this situation.

That doesn’t make breaking this news any easier. I certainly expect a great deal of anger over this, and nobody could blame you for that.

So, if you are one of the diarists with missing entries, what options do you have?

First, if you still have your original diary over in Open Diary you can transfer it over here again, and get those entries back. The import feature is online, so please feel free to do that, if you can.

Second, if you have saved any of your entries offline, you can also restore those to your diary.

There is a small shred of good news – the notes that were attached to these entries are still in our database. This means that once you have re-imported your OD diary or reloaded entries from offline, we should be able to restore the notes that you had on those entries. There is enough information in the notes file that we can connect the notes with the entries they belong to.

To do this, we will need to know if and when you have restored entries to your diary. So please, if you re-import or repost entries to your diary, let us know by e-mail to the problem@opendiaryplus.com address when you are finished. Then we will know that we can go ahead and restore your notes.

Finally, I know that there are some diarists who transferred their diaries here, and then deleted them on OD. If you are one of these diarists, please e-mail us the name of your old OD diary, and (if you know) the approximate date is was deleted. Depending on how long ago it was, we may be able to restore all or part of the diary from the OD backup tapes. This will be a very laborious task, but I am willing to try anything to bring back as many entries as possible.

Several diarists have asked about our backup facilities here on OD+, and why a backup didn’t exist for the file that was corrupted last Sunday. A backup did exist, but a couple of freak occurences caused it to become useless. Our SQL server performs an automatic daily backup starting around 2 a.m. every morning. When we discovered the problem with the data file last Sunday afternoon, I immediately shut down the site so that no more data would get written into the file. This was to ensure that no additional data would be corrupted.

We started work on trying to recover the file right away, and worked through Sunday afternoon and evening while the site remained offline. As you may or may not know, our office is on the East Coast – but our servers reside at another location. You may remember that there was a snowstorm on the East Coast last Sunday night. During this storm, as we were working on the file at about 11 p.m., we had a power outage at our office. Because of this, we lost all communications with our server location.

Power remained out until about 3 a.m. Monday morning – about four hours. When power was restored, we reconnected to our servers – and found that the 2 a.m. backup routine had run, and overwritten the previous day’s backup file with the corrupted data file. As a result, we didn’t have a backup file to recover from.

Since Sunday, I have taken several steps to ensure that this sort of problem does not happen again. First, we have moved OD+ to a larger and faster server, guaranteeing better performance. Second, we’ve changed from a single daily backup scheme to multiple daily backups – so that at any particular time we have several backup files ranging from one to seven days old. Third, we are also doing a physical backup of the entire server to separate tapes on a regular basis. All of these changes should ensure the safety of your diaries as we go forward.

I know it is small condolence for people who have lost writings here, but please know that nothing that could have happened here could have made me feel worse. I think of each of our diarists as more than just customers – and hurting any of you is the last thing that I would have wanted to do.

I am sorry for this problem, and we will work diligently to be sure that it never happens again.