What Happened?
When we deployed R106 on the morning of September 29th, we found that there was underlying database corruption. We worked diligently for the next week identifying the issues and attempting to recover the database. During this time we worked on a number of past iterations restored from our backups. After seven days of work we were still unable to achieve stability and made the very difficult choice to restore a database backup saved prior to releasing R105. The type of restore is known as a ‘Rollback’. This means that any and all progress gained from the point of the restored backup will have been lost. This also means that new characters created will no longer exist.
What’s my impact?
A rollback is like using a time machine. When the Live game server was restored and brought on-line, all progress, stats, etc. are as they were on August 24th at 8am CT. This impacts the following in addition to other aspects of your character’s status:
- New Characters: If you created a new character during this window it will be missing, please re-create the character and reach out to support for assistance in having any earned XP reallocated to your XP Pool.
- XP and Gold: We were able to extract information from the most current backup, which allowed us to restore player account XP levels. We are investigating what we can do to restore player account gold.
- Skills: XP has been placed on your account in the XP pool. We are unable to restore XP to specific skills, so any skill level changes to your character will need to be re-trained.
- COTOS/Website Purchases: Any COTOs or store items purchased from 8/24 – 9/29 and throughout the outage will be (re) delivered via the Rewards Chest in game.
- Houses/Lots: If you claimed a lot or placed a house during the period of time the rollback covers, you will need to reclaim.
- POTs: IF you upgraded or purchased a New POT during the time frame, we are working to re-apply the new or upgraded town to the game.
NOTE: We cannot restore decoration work done on lots or in POTs. - Taxes/House expiration timer: We have suspended the eviction timer and will keep that in place until at least November 18th. During this time we will be working to clear the back taxes on affected lots. If the deed was obtained prior to 8/24 @ 0800 CT then the deed should be where it was previously: in your bank, Inventory, or storage.
NOTE: Weare suspending the eviction timer until Nov 18th, not suspending taxes. The daily increment for lot taxes will continue as normal. The extension of the timer allows us to have time to clear the back taxes on lots without having lots fall. After your back taxes are cleared, you should start populating tax credit funds as normal. If you have paid off your back taxes instead of waiting for the scripted back tax purge, please reach out to support to have those funds refunded (we’ll verify the amount in the server logs).
Rewards?
Please note that Subscriber and Login rewards have been re-delivered as part of the R106 bundles. Due to the game’s structure we cannot re-deliver R105 and R106 rewards separately, so the Releases have been combined into one bundle. Some additional items have been added to the October Login Rewards.
What’s next?
We have a Livestream scheduled for Friday October 14th @ 4pm CT to go into more detail surrounding the issues and the recovery efforts, as well as the changes we are making to ensure this will never occur again. As noted above there are still aspects we are working on to make accounts as whole as possible. We will discuss all of this in detail on the livestream, along with other aspects that have impacted the players.
Did you buy items with COTOS in the Crown store during the timeframe we rolled back over? Though the COTOS will still be in your account, and any COTOS purchased in September will also automatically be delivered to you via the Rewards Chest. We understand that some of the items were purchased when they were on sale, or were available during a Vault or seasonal cycle making it near impossible to re-purchase them. We will be announcing our plans on how we will assist in allowing players to re-obtain these items during the Friday stream!
Thank you for being so patient during this calamity!