Recent feed issues, upcoming scheduled maintenance/downtime, and how you can help - Anime-Planet forum

Go Back   Anime-Planet forum > Site Information > Site Announcements

Notices

Reply
 
Thread Tools Display Modes
Old 07-13-2012, 09:09 PM   #1
sothis
Overlord
 
sothis's Avatar
 
Join Date: May 2002
Location: Seattle, Washington
Age: 33
Posts: 15,680
Default Recent feed issues, upcoming scheduled maintenance/downtime, and how you can help

Today, there was an issue where the feed showed an error on the profiles. Just wanted to give you a heads up about what happened, and tell you about some upcoming downtime tomorrow to maintain the servers.

Basically, the hard drive the feed's database resides on had some issues with bad data (as it's a year old drive, is heavily used, and has started to fail), and the drive needs to be replaced. It took awhile, but we were able to fix the bad sectors and stabilize the drive. But, it does need to be replaced. This issue ONLY AFFECTED THE SITE FEED, which is stored on a separate type of database on a separate server. Your list data, all site content (reviews, screenshots, everything) are on the main server, and are not affected by this.

Meanwhile, there's some things we need to do on the main server as well - one of the drives of the RAID array needs to be replaced, and a few other things tinkered with to keep them going. With such a heavily trafficed server, that comes with the territory.

We've decided we'll do these all at once, since each thing would require the server to be down. We're planning it for tomorrow at some point, and I'll post updates warning people ahead of time. Updating the feed server may take extra time to get up and running after the hard drive switch, but we'll update the site code to not output errors as a temporary measure, once the server is back up, if needed.

HUGE thanks to holms and dmitrijus who assisted for hours today with troubleshooting what could be wrong with the server. They deserve a lot of kudos.

Finally, while I really, really dislike bringing up donations, several people in irc today asked about it, so I'll mention it. If you feel like supporting the site, here's the link to donate

Right now times are pretty tough - the site keeps growing, traffic-wise, which means more resources (bigger and better servers, more support, etc) are needed, but the revenue coming in from ads is not growing, and it's low to begin with. The site is only pulling in a little more money than the servers and other related bills cost (I don't pay myself a dime, never have, all money goes straight into the site).

Meanwhile, because of this crash, I just had to pay 10gen $2,500 USD for the next year, for support on the activity feed database - it's NoSQL, which is still a newish technology and there aren't really any reputable companies out there (or highly skilled server admins with that specialty). There wasn't really another choice for the feed, as all site activity is logged and to a huge degree (it's a 20 gig size for the last 8 months). The main developer I had for years is gone now, and I'm not an expert on the subject. We need ongoing support to update safely, optimize so those cursortimeout errors stop, etc. So $200/month sucks, but is pretty necessary to keep the site going.

And because of that extra $200/month, we're now basically breaking even. That's really scary, given there's no money to do ANYTHING - nothing for emergencies that pop up, I can't hire anyone to help with some of the really complicated aspects of the site, and I can't move forward with some of the bigger projects I've wanted to do (like getting a professional redesign for cetain pages, etc). Unlike some of our competitors, we aren't corporate owned and don't have huge sponsors that can pay for multiple salaries - it's just me, and i work for free.

So, again, if you love Anime-Planet and want to see it able to continue, please consider donating

**note: I realize I still haven't inputted badges for a few months for donations - I apologize, it's due to a lack of time. ;_; with no developers now except me, I've found it increasingly difficult to get everything done, by myself, that's needed to support a site of 400,000 people a month. Donations help, among other things, to ensure i can actually get more people onboard to help out.

Thanks everyone for your patience during this time, and I'll update more once it gets closer to the scheduled downtime. And thanks for using A-P
__________________

Anime-Planet.com - anime | manga | reviews

Adding Anime - Synopsis Templates | Twitter | MySpace

[sixth sense voice] I SEE BRITISH PEOPLE - Control my signature or avatar!
sothis is offline   Reply With Quote
Old 07-14-2012, 12:40 AM   #2
r18
Ojii-san
MY DNA IS MADE UP OF ANIME
 
r18's Avatar
 
Join Date: Apr 2008
Location: Ittan desuka? Ee, zutto
Age: 61
Posts: 46,139
Default Re: Recent feed issues, upcoming scheduled maintenance/downtime, and how you can help

well i can't speak for others ...but i very much appreciate the time and effort that you put into this site....and i don't feel any pressing need to have a badge dealt with when there is actual site related emergencies to be dealt with...so take your time and don't worry i'm not going anywhere....
__________________

Anime-Planet.com - anime | manga | reviews



look into my face children ....it is i that you shall become...

r18 is offline   Reply With Quote
Old 07-14-2012, 07:04 AM   #3
Kari5
MY DNA IS MADE UP OF ANIME
 
Kari5's Avatar
 
Join Date: Mar 2008
Location: Canada
Age: 24
Posts: 11,318
Default Re: Recent feed issues, upcoming scheduled maintenance/downtime, and how you can help

Thanks for your hard work as always, Sothis!
__________________

Kari5 is offline   Reply With Quote
Old 07-14-2012, 07:33 AM   #4
NarkyOtic
Narkylepsy
Otaku
 
NarkyOtic's Avatar
 
Join Date: Feb 2012
Location: Roinne
Age: 24
Posts: 2,181
Default Re: Recent feed issues, upcoming scheduled maintenance/downtime, and how you can help

I have a buttload of awestruck respect for your efforts on this site, sothis. I don't pretend to know anything about its inner workings, but a blind man in space could see that A-P is a GARGANTUAN undertaking all of the time, especially for someone working practically alone.

I plan on donating a decent dime when my little end of financial year bonus comes through, which will likely be early next week. I'll do more if I can.

All the best for the shift ahead!
__________________
~Judo is a language: you are speaking it when you practise it, you are reading it when you watch it.
These lines are its alphabet, which is written in the air~




Lost for words? Look no further: SVM - Professional Editing

NarkyOtic is offline   Reply With Quote
Old 07-14-2012, 07:56 PM   #5
sothis
Overlord
 
sothis's Avatar
 
Join Date: May 2002
Location: Seattle, Washington
Age: 33
Posts: 15,680
Default Re: Recent feed issues, upcoming scheduled maintenance/downtime, and how you can help

The site is now back up after many long hours and is working perfectly. The delay was due to the RAID array on the main server - not only was a drive not working, but the configuration was wacky and needed to be fixed. This led to needing 2 separate array rebuilds.

Thanks to everyone for your patience during the maintenance - and if you're glad to see the site back up, please stop by the profiles of holms and dmitrijus and drop them a thanks. They spent all day today, again, helping out with making sure the servers are working during the maintenance, and I couldn't have done it without them.

And if you want to help AP continue running, as unfortunately now we'll be spending a few more hundred dollars a month on server support, here's the link to donate to the cause - anything is appreciated!

I'm off to a much, much needed dinner and adult beverage
__________________

Anime-Planet.com - anime | manga | reviews

Adding Anime - Synopsis Templates | Twitter | MySpace

[sixth sense voice] I SEE BRITISH PEOPLE - Control my signature or avatar!
sothis is offline   Reply With Quote
Old 07-14-2012, 10:56 PM   #6
Drahken
Cross-eyed Cyclops
 
Drahken's Avatar
 
Join Date: Nov 2007
Location: 1985
Posts: 2,864
Default Re: Recent feed issues, upcoming scheduled maintenance/downtime, and how you can help

Quote:
Originally Posted by sothis
...and I'll post updates warning people ahead of time.
^You neglected this, but oh well.


Decidedly strange that holms goes from some random member that you were ticked off at because of his attitude & his threat to build an app, to someone who helps fix the site. Does this mean he gets his request for an api or whatever it was filled? :p
__________________
*Warning: The above post may be highly opinionated, read it at your own risk.
Wham! Bam! It's the dino-jam!
Drahken is offline   Reply With Quote
Old 07-15-2012, 08:49 AM   #7
sothis
Overlord
 
sothis's Avatar
 
Join Date: May 2002
Location: Seattle, Washington
Age: 33
Posts: 15,680
Default Re: Recent feed issues, upcoming scheduled maintenance/downtime, and how you can help

i posted one update (on facebook/twitter), but due to the need to get it started i started it shortly after i got up in the morning (not really possible to send warnings from bed). better that than it being down another night. i then posted judicious updates on facebook throughout the day, so i suggest people check that out anytiume theres a weird issue

and no, it's not that simple. but yes, holms did help extremely
__________________

Anime-Planet.com - anime | manga | reviews

Adding Anime - Synopsis Templates | Twitter | MySpace

[sixth sense voice] I SEE BRITISH PEOPLE - Control my signature or avatar!
sothis is offline   Reply With Quote
Old 07-19-2012, 11:04 AM   #8
Anathemus
Anime Fan in Training
 
Anathemus's Avatar
 
Join Date: Oct 2009
Location: Portugal
Age: 30
Posts: 345
Default Re: Recent feed issues, upcoming scheduled maintenance/downtime, and how you can help

I might donate another small amount when i can, have you consider searching for sponsor solutions?
Anathemus is offline   Reply With Quote
Old 07-20-2012, 06:22 AM   #9
About7Narwhal
Narumon Z
 
About7Narwhal's Avatar
 
Join Date: Sep 2011
Location: Tennessee USA
Age: 25
Posts: 28
Default Re: Recent feed issues, upcoming scheduled maintenance/downtime, and how you can help

I threw a ton of money at my screen. Did it help?
About7Narwhal is offline   Reply With Quote
Old 07-26-2012, 11:33 AM   #10
GodzillaGus
Narumon Z
 
Join Date: Sep 2011
Age: 28
Posts: 86
Thumbs up Re: Recent feed issues, upcoming scheduled maintenance/downtime, and how you can help

Quote:
Originally Posted by Anathemus View Post
I might donate another small amount when i can, have you consider searching for sponsor solutions?
this!
__________________

Anime-Planet.com - anime | manga | reviews
GodzillaGus is offline   Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 02:36 PM.

Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.