Source Code Repository down?

Apr 12, 2007 at 1:23 AM
Edited Apr 12, 2007 at 1:24 AM
Apr 12, 2007 at 2:35 AM
We are experiencing problems with the TFS03 server, all projects hosted on this server are currently experiencing problems with their source code and issue tracking.

System Outage Report
Apr 12, 2007 at 10:12 PM
Is there any update on this? The Report page is outdated at this point.
Apr 13, 2007 at 6:11 AM
I'm surprised that I still can't access TFS03 after more than 48 hours. While I can't complain too much because it is a free service being provided, just some information would be nice.
Apr 13, 2007 at 1:06 PM
Edited Apr 13, 2007 at 7:45 PM
Current status as of noon PDT on April 13:

At 3pm PDT on April 11th an operator error occurred that caused source control and issue tracker data on one of the Microsoft CodePlex servers to be accidentally overwritten. During the standard data recovery effort, a recovery backup configuration oversight was discovered in the routine backup process for this CodePlex server which is currently impacting immediate restoration of the data.

Note: The content in the Wiki as well as the release files have not been affected. This current outage only affects projects that were hosted on TFS03.

We are working on this situation as a matter of highest priority. All affected projects will have their source control and issue tracker functionality restored soon but the status of the source control and issue tracker data remains unknown at this time. We will report back with a status update within 96 hours.

We sincerely apologize for the inconvenience this may have caused. The necessary corrections are already in place to ensure this situation will not occur again.

The CodePlex Team
http://www.codeplex.com/ContactUs.aspx
Apr 16, 2007 at 12:00 AM
Hi Brad,

Do you guys have any estimation on when the server will become available again, and the data restored?

Cheers,

Wagner Silveira
Apr 16, 2007 at 12:37 AM
We will have another update on Monday around 11:00am PDT.
Apr 16, 2007 at 8:21 PM
Is this why my new project's source code and user managment cannot be changed? Or is it something else, like a grace period before I can start uploading source?
Apr 16, 2007 at 9:50 PM

jimnewkirk wrote:
We will have another update on Monday around 11:00am PDT.


Thanks Jim.
Apr 16, 2007 at 10:27 PM
I realize that it is after 11:00am and I apologize again. We are currently working on getting the functionality restored. We expect to have the functionality restored this evening. The status of the data is still unknown.

Apr 17, 2007 at 6:58 AM
At approximately 9:30pm PDT we were able to restore the source control and issue tracker functionality however the state of the source control and issue tracker data is still unknown. Work is continuing on being able to recover the data. Another note is that for projects that were hosted on TFS03 your project is now being hosted on TFS02. There is wiki page called SwitchTFS which is on the Codeplex FAQ which describes how to rebind your source control to the TFS. Once again, I cannot apologize enough for the problems in the past week.
Apr 17, 2007 at 7:46 PM
Thanks for the support, what about my post? We are anxious to get source control up to stop the passing of large Zip Files, but the Source page says "The source control server is currently unavailable. Source code cannot be accessed at this time." Likewise I cannot add users to the project.
Apr 17, 2007 at 9:24 PM

jsedlak wrote:
Thanks for the support, what about my post? We are anxious to get source control up to stop the passing of large Zip Files, but the Source page says "The source control server is currently unavailable. Source code cannot be accessed at this time." Likewise I cannot add users to the project.


JS, I think if you were on TFS 03 you will need to first rebind it to TFS 02 as per Jim's instructions above. Just my uneducated guess.
Apr 18, 2007 at 12:27 AM

jimnewkirk wrote:
Once again, I cannot apologize enough for the problems in the past week.


Thanks for the update Jim. I'm sure all of you guys are doing your best to get things back online. No worries.
Apr 18, 2007 at 12:41 AM

WagnerSilveira wrote:

JS, I think if you were on TFS 03 you will need to first rebind it to TFS 02 as per Jim's instructions above. Just my uneducated guess.



When following the instructions, I get the following error:
"error: TF50309: You do not have sufficient permissions to perform this operation. ---> Attempted to perform an unauthorized operation."

Part of the problem is I have no source information on the project as it is a new account. Could it be that some grace period has not yet ended? I get the above error when I attempt to checkout, and cannot follow the instructions as per the switch because I do not have a working directory.

I appreciate the help though.
Apr 18, 2007 at 5:32 AM
On another note - if you created your project after Wednesday April 11 please bear with us. We are working to correct this issue and get your projects up and running on a new TFS instance. We are in the process of getting it setup. I will have an another update tomorrow. We will also make sure that you get a full 30 days of setup period once we have the new TFS instance up and running.
Apr 18, 2007 at 12:50 PM

jimnewkirk wrote:
On another note - if you created your project after Wednesday April 11 please bear with us. We are working to correct this issue and get your projects up and running on a new TFS instance. We are in the process of getting it setup. I will have an another update tomorrow. We will also make sure that you get a full 30 days of setup period once we have the new TFS instance up and running.


Thank you for adressing my question! And thank you for working hard on it!
Apr 18, 2007 at 10:47 PM
We have some good news on the work item front, it looks like we will be able to re-create a lot of work item data. You should start seeing work items getting re-created within the next 24 hours. The status on the source control history is still unknown.
Apr 21, 2007 at 12:59 AM
Just wondering what the status of the restore is (if any). On one of my projects (Tree Surgeon) I seem to have lost all changesets and issues. Not sure if the source code repository is gone or not. Just hoping these will come back as I don't have a backup of my issues.

Thanks.
Apr 24, 2007 at 12:36 AM
bump

It now appears several projects that were on tfs03 have lost work items. I'm fine to recreate them but just want to be sure that they're not going to be restored at this point. Also not sure of the state of source control as the changesets are gone. Will I need to re-upload the initial tree again?
Apr 25, 2007 at 4:11 AM
The work item import is still happening. We're working as quickly as possible to verify the import process to make sure there are no problems (remember, there's no "delete work item" in TFS ;) ).
Apr 26, 2007 at 12:12 AM
We were able to recreate most of the work items for the affected projects this afternoon. The status of the source control data remains unknown.
Apr 28, 2007 at 9:01 AM
We are still missing our source in the project "cropperplugins"... Any chance that this will be recovered?
May 9, 2007 at 8:35 AM
The source code for PageMethods is still missing almost one month later. Are you people still working on this problem???
May 9, 2007 at 7:39 PM
My FlickrMetadataSynchr project is also still in some state of limbo. The work items are restored, but all sources and their history are still missing. It would be a shame if the source control history and the links to the work items were to be lost forever, considering how much work I put into them.
May 9, 2007 at 9:25 PM

Erwyn wrote:
My FlickrMetadataSynchr project is also still in some state of limbo. The work items are restored, but all sources and their history are still missing. It would be a shame if the source control history and the links to the work items were to be lost forever, considering how much work I put into them.


Same here - my CacheMyWork got its work items back, but no source or source history. This is really frustrating - please at least indicate status of the efforts to recover/not this data (so that at least we know whether to wait, or just start recreating the code).
May 17, 2007 at 11:19 PM
The action taken to address the operator error reported last month that caused source control and issue tracker data on one of the Microsoft CodePlex servers to be accidentally overwritten has now reached its conclusion.

During the standard data recovery effort, a recovery backup configuration oversight was discovered in the routine backup process for the affected CodePlex server which impacted the restoration of some data. The recovery efforts concluded with no Wiki or release file data affected and over 90% of work items were successfully recreated. However, some CodePlex projects did have change histories that were not able to be recovered.

As of April 18th 2007 the CodePlex site was fully operational. The necessary corrections are in place to ensure this situation will not occur again.

We apologize for any inconvenience this may have caused. Your patience during this recovery period has been appreciated.

Please contact the CodePlex team if you have any questions.
Jun 14, 2007 at 3:39 AM
I haven't worked on one of my projects for a while and just checked back today and guess what... no code anymore :(. I'll ping you guys an email

the project in question
http://www.codeplex.com/TestableWebRef
Jul 14, 2009 at 11:09 AM

I started a new project (joomNET) yesterday. At the Source Code Tab i see only "The source control server is currently unavailable. Source code cannot be accessed at this time"

Any idea ?

Jul 14, 2009 at 6:41 PM

wuetrs, your issue has been resolved.

Apr 24, 2010 at 9:54 AM

I have the same issue as wuetrs did. My project name is charactergenerator.

Apr 26, 2010 at 4:27 PM

Hi davreetom,

Your issue has been resolved. Sorry for the troubles.

Matt

Apr 27, 2010 at 4:38 AM

TFS08 TFS server is down right now.  I can't connect through visual studio (e.g. connect to team foundation server)  I can view all the source & work items online.  This seems to happen sporadically, is it an issue on my end or doe the server experience outages regularly?  Either way, thanks for the great service you provide (really!)

 

-T.J.