Posted by Blue Box Group about 11 months ago...
Our shared mail server system is experiencing high load issues which we are currently investigating. This is causing intermittent problems with both sending and receiving mail. (However, we do not as yet believe any mail is being lost as a result of these issues.) We will update this status page entry as we learn more.
UPDATE 3:40pm PDT (22:40 UTC): Our initial investigation seem to point to this being caused by very large amount of extraordinarily large e-mail being received early this afternoon (followed by the users of the system then downloading these large messages). In any case, this problem appears to have been mitigated for now. We will update this post with a more permanent solution as we learn more about the cause of today’s e-mail outage.
Posted by Blue Box Group about 11 months ago...
At 10:32pm PDT (Sep 4, 05:00 UTC) our secondary facility lost power for approximately 10 minutes. We are still investigating the matter with the facility NOC staff and will update this status page if we expect another outage. Our secondary facility contains only backup servers and redundant production servers. As such no customer production services should have been affected by this outage.
Minor update (10:50pm / 05:50 UTC): All affected machines appear to be back up and running at this time.
Web only post
Posted by Blue Box Group about 11 months ago...
The Ruby on Rails team has announced the latest update of RoR today, version 2.3.4. Aside from a few new features and bugfixes, there are two security issues that have been fixed in this release, so it’s strongly recommended that any user currently using the 2.3 series upgrade as soon as possible. As always, the Blue Box Group staff would be more than happy to offer any assistance you may need.
You can find the RoR team’s writeup here:
<a href=http://weblog.rubyonrails.org/2009/9/4/ruby-on-rails-2-3-4>http://weblog.rubyonrails.org/2009/9/4/ruby-on-rails-2-3-4
Posted by Blue Box Group about 10 months ago...
EDIT 09/10/2009 - 3:15 PM : We received reports from a number of our customers this afternoon that they were having trouble connecting to sites on our servers. This was affecting a very small subset of our customer base (less than 2%), but we were able to trace the problem down to a lingering effect of our upstream provider’s issues last night. This should now be resoved – if you have any issues, please contact us.
Tonight, starting at approx 02:35 AM (Thu Sep 10 09:35:01 UTC 2009), one of our redundant upstream providers experienced a problem with the router we are directly connected to. Our network is designed in a fully redundant configuration, so our traffic is now presently going out our other providers without a problem. We are leaving our connection down with this particular provider until we’ve received a report on the status of the router in question.
This issue did not create an outage, but customers may have noticed suboptimal routing while our routes reconverged through our other providers.
We will update this ticket when we’re connected back with this particular provider.
Posted by Blue Box Group about 10 months ago...
vsh14 appears to have locked up as of 4:08 PM Pacific (Sat Sep 12 23:08:00 UTC 2009). Technicians are responding and diagnosing the machine now. All virtual servers housed on that machine should be available momentarily. We will track updates in this system status post.
Update 4:17pm (Sat Sep 12 23:14:40 UTC 2009) - vsh14 has fully restarted and all VPS accounts on the server are in the process of starting. Technicians will confirm that all known and monitored services have properly started on boot. We will update this ticket with more information as it becomes available.
Update 4:44pm (Sat Sep 12 23:44:25 UTC 2009) - All virtual servers came back online as of 4:28pm, and as of 4:44pm, BBG technicians have confirmed that all known services are operating as expected. We will now consider this issue resolved.
Thanks
(Web only post)
Posted by Blue Box Group about 10 months ago...
At 11:28 AM Pacific (Sun Sep 13 18:28:11 UTC 2009), customers running on the legacy Rails Box Car servers (those not yet migrated to the BBG network), experienced a brief network outage. Service was restored within 5 minutes. We are awaiting details from the legacy facility as to the cause of the outage and will update this post with that information as we get it.
We recommend that all legacy customers prioritize a time to move service within the BBG network, so we can ensure a priority level of service.
Update September 14th, 9:10 am (Mon Sep 14 16:10:32 UTC 2009) - No additional information has been provided by the legacy hosting provider. We continue to encourage legacy box car customers to migrate their services to our network as soon as possible.
Thank you
(Web only post)
Posted by Blue Box Group about 10 months ago...
As of 9:40PM (Fri Sep 25 04:38:13 UTC 2009), vsh31 is presently undergoing a reboot to clear an issue with the disk subsystem. We will update this post when the machine is back online and functional.
Update 10:19PM (Fri Sep 25 04:40:44 UTC 2009) All nodes are back online and in operation. We will continue to monitor this box for further issues.
(Web only post)
Posted by Blue Box Group about 10 months ago...
sh01 became overloaded and was rebooted to stop the process that was causing the problem.
As of 4:12am (Wed Sep 30 11:12:08 UTC 2009) the system was operational and BBG technicians have confirmed that all known services are operating as expected.
-Blue Box Tech Support
(Web only post)
Posted by Blue Box Group about 10 months ago...
Between 3:00pm and 5:00pm today (10:00pm and midnight UTC) we will be doing a company-wide training session. During this time no technicians will be answering phone calls or trouble tickets. However, should a service-affecting emergency arise during this window, please open an urgent support ticket, and a technician will be with you shortly.
