Webbynode System Status

Level3 Connectivity

cartab — Wed, 03 Mar 2010 04:42:00 +0000

[11:40 PM EST] We are working with our network provider level 3 to solve a routing issue that has caused some packet loss for traffic going through this route.

Update: 11:59 PM EST Issue has been solved, we’re back to optimal level3 performance.

[Completed] miami-b12 Maintenance

cartab — Wed, 17 Feb 2010 18:30:20 +0000

We had a raid failure on miami-b12. The server is up and running, however the raid is running a full rebuild, so the disk access might be slower than usual.

We will keep monitoring the progress, and update this post.

Note: All Webbies are up.

Update 11PM EST: The rebuild has fully finished with no problems.

RESOLVED – GitHub Integration Unavailability

fcourywebbynode — Wed, 03 Feb 2010 14:20:28 +0000

We are suffering from an issue where GitHub’s API is timing out. This is affecting any deployments that uses GitHub integration.

We’re actively working on this problem however we don’t have an ETA at this time.

Thanks for your patience!

Update 9:30AM EST: It seems it was a hiccup on GitHub’s end. Integration is available again.

[Completed] miami-b14 Xen bug

cartab — Mon, 25 Jan 2010 17:10:00 +0000

[12:05 PM EST] We hit a Xen bug with miami-b14, we will update this post as soon as its finished.

Update 12:08 PM We had to reboot miami-b14 and Webbies are coming back online.

Please contact support if you have any issues.

[Completed] miami-b04 Xen bug

cartab — Mon, 25 Jan 2010 14:17:49 +0000

[9:17PM EST] We hit a Xen bug with miami-b04, we will update this post as soon as its finished.

Update 9:22AM We had to reboot miami-b04 and Webbies are coming back online.

Please contact support if you have any issues.

[Completed] miami-b11 Maintenance

fcourywebbynode — Sat, 23 Jan 2010 23:45:07 +0000

We had a raid failure on miami-b11. We had to turn off the server for a hardware maintenance, and it is now complete.

The server is back up and Webbies are coming back online.

If you have any issues please contact support.

UPDATE 8:20PM :

We continue to have issues with miami-b11 we are now working on migrating everyone off miami-b11. We’ll update this shortly.

UPDATE 01/24 0:20AM :

All Webbies are up but we have disabled rebooting, stopping, starting and redeploying Webbies in the meantime. Once you’re migrated to another node, you’ll be ready to manage your Webby as usual.

Update 01/24 8:00PM :

We have finished migrating all Webbies to a new node. We managed to rescue all data, no data loss was suffered.

[completed] miami-b04 hardware maintenance

cartab — Fri, 08 Jan 2010 20:48:56 +0000

[3:15PM EST] We are working on an issue with miami-b04, we will update this post as soon as its finished.

[4:15PM EST] Miami-b04 is now back. We had to do an emergency hardware maintenance and no data was lost.

We apologize for the long delay, however we had to do this carefully to avoid any possible data loss and more downtime.

All Webbies are back now, if your Webby isn’t up, please contact support.

[completed] miami-b11 & miami-b15 reboots

cartab — Sun, 13 Dec 2009 05:34:00 +0000

[11:59PM EST] We ran into a Xen bug on miami-b11 and miami-b15 and the nodes had to be rebooted. All Webbies are coming back up, and should be up shortly.

Please contact us if you have any issues.

Your Webbynode Team

[completed] Multiple node reboots

cartab — Tue, 08 Dec 2009 06:39:25 +0000

1:35AM EST. After applying some kernel patches to a group of nodes, we had to perform reboots on them. All nodes are back up and Webbies are coming back online.

Please contact support if you experience any issues.

[Completed] miami-b10 Maintenance

fcourywebbynode — Sun, 29 Nov 2009 18:19:54 +0000

1:19 PM EST: We are investigating Xen problems on miami-b10.