FileFactory Server Consistently Overloaded

[Lycanthrokeith]

I'm getting this message consistently when trying to download something:

Server Load Too High

The server hosting this file is temporarily overloaded. You can download this file immediately by upgrading to FileFactory Premium. Alternatively, you can try again shortly.

Please contact support if you require further assistance.

I'm following the rules for magazine downloads for the number of posts I've done (I'm still under 25, so I'm sticking to one a day like I'm supposed to). But it's for naught if the external service isn't functioning properly, or worse, if FileFactory is doing this intentionally to hustle paid subscriptions out of people. I understand the benefits, but when I'm being directed there for a free download, it comes off as a hard sell tactic.

Any ideas as to what's going on with FileFactory?

Thanks.

[Phillyman]

I am right on the edge of putting us on a dedicated server, where we would get 1TB of storage for our magazines and get 5TB of bandwidth for you guys to download directly from us. The only hold up is it would cost us $1400 per year ($119 per month)

Do we have 100 members out of our 4000 that would donate $14 per year to Retromags?

[KiwiArcader]

That's why over at OGM all new downloads are being hosted on both FileFactory and Mediafire. However, there's only so much one can do when a website is being run on a budget where most everybody is simply there to leech and contribute nothing so in the end one person carries the can/costs for the whole site. I always find it interesting and annoying at the same time when these people moan and groan about access but don't want to do anything about it other than moan.

[Phillyman]

That's why over at OGM all new downloads are being hosted on both FileFactory and Mediafire. However, there's only so much one can do when a website is being run on a budget where most everybody is simply there to leech and contribute nothing so in the end one person carries the can/costs for the whole site. I always find it interesting and annoying at the same time when these people moan and groan about access but don't want to do anything about it other than moan.

Well if I bring everything back in house, we can use the download manager software that we used to use.....and just paste those links over into the Magazine DB. Guests would still be using FileFactory, and I could restrict down direct downloads to members only. I spoke to my hosting company and they have assured me, that as long as I dont go over 1TB of storage and 5TB of downloads, that they will not limit us.

Guests - FileFactory

Members - 10 files per day

Team Members - 20 files per day

etc..

or I can set it up to use the Credits system.....

Post a new Topic - 25 credits

Repy to topic - 10 credits

Upload Missing Cover - 10 credits

Upload Magazine - 100 credits

Download Magazine - 10 credits

[RetroDefense]

Well if I bring everything back in house, we can use the download manager software that we used to use.....and just paste those links over into the Magazine DB. Guests would still be using FileFactory, and I could restrict down direct downloads to members only. I spoke to my hosting company and they have assured me, that as long as I dont go over 1TB of storage and 5TB of downloads, that they will not limit us.

Guests - FileFactory

Members - 10 files per day

Team Members - 20 files per day

etc..

or I can set it up to use the Credits system.....

Post a new Topic - 25 credits

Repy to topic - 10 credits

Upload Missing Cover - 10 credits

Upload Magazine - 100 credits

Download Magazine - 10 credits

I really like this idea. I think maintaining the status quo is a good option for non-members (though, some file-sharing alternatives to FileFactory would be nice) while site participants could be "rewarded" with the direct option.

That said, I'd have a hard cap set on that 5TB so as not to incur crazy bandwidth charges, just in case.

If you decide on the credit option, you might want to keep an eye on forum credits - I've seen other forums abused with "Me too!" posts and the like just to run up numbers.

Would it be possible to tie monetary donations to credits or would that be too difficult/too much of a grey area?

I know it's a work in progress but I really think the planned collection tracking functionality will make a difference in participation rates. I imagine it'll attract more enthusiasts who'll be more inclined to help out, or perhaps even convert some lurkers into participants.

[Phillyman]

I really like this idea. I think maintaining the status quo is a good option for non-members (though, some file-sharing alternatives to FileFactory would be nice) while site participants could be "rewarded" with the direct option.

That said, I'd have a hard cap set on that 5TB so as not to incur crazy bandwidth charges, just in case.

If you decide on the credit option, you might want to keep an eye on forum credits - I've seen other forums abused with "Me too!" posts and the like just to run up numbers.

Would it be possible to tie monetary donations to credits or would that be too difficult/too much of a grey area?

I know it's a work in progress but I really think the planned collection tracking functionality will make a difference in participation rates. I imagine it'll attract more enthusiasts who'll be more inclined to help out, or perhaps even convert some lurkers into participants.

Well it is a double edge blade whenever you have a "tit for tat" system. If we do a credits system we will surely have the users who post a reply with 2-3 words. The same will happen if we do it by member group though, it will just be people creating 10 clones to bypass the limits. I would be really surprised if we even came close to 5TB per month in bandwidth. When we stopped hosting the magazines locally, we were doing about 1TB of transfers per month, and while we do have more magazines now, we also have less members due to the current state of affairs. Here are the options as I see it.....

1) Go to a dedicated server, reward users who add value to Retromags.....with easy and quick downloads of their favorite magazines.

2) Keep on depending on an outside file sharing service like Rapidshare/FileFactory. Assume the position and wait for them to either be shut down, or change their terms of service.

3) Find a bigger site to host our magazines, keeping it cheap to run Retromags, but in the long run harder to keep everything synchronized.

If everything comes back in house, E-Day and myself will have an easy way to sync our collections to the Retromags collection......and users will only need to upload to us.....and everyone can download from here. I can tie credits to donations, but at that point I might as well just tie donations to member groups like we used to do.

[Sean697]

I don't know what you have in the works. Or if you want to keep the magazines downloads exclusive to this site. But having downloads on archive.org kind of like what John smith is doing would solve all of these problems. As long as the retromags splash page is there, I think that would be the ideal solution.

[KiwiArcader]

It seems like an awful lot of effort just to keep the moaning leechers happy though. Team members who have contributed mags to the site, I can understand wanting to reward them, as the mags wouldn't even exist without those guys but everyone else essentially comes along for the ride when it comes to providing magazines. I know people have contributed to the databases etc so they fall into the middle ground but it's not really the same as providing mags so a tiered structure makes sense there but having Joe Leecher who hasn't done anything other than create a login so he doesn't have to use FileFactory seems like just rewarding mediocrity.

[Phillyman]

I don't know what you have in the works. Or if you want to keep the magazines downloads exclusive to this site. But having downloads on archive.org kind of like what John smith is doing would solve all of these problems. As long as the retromags splash page is there, I think that would be the ideal solution.

That is something I have been really thinking about. While Archive.org could help, how much overhead would it create to keep it all up to date. If a member comes to Retromags and scans a magazine, they would have to upload it somewhere like FileFactory for E-day or myself to retrieve. Then we would need to pass it along to Archive.org, wait for someone over there to upload it and provide us a direct download link, then we could insert that link into the Magazine Database for the correct record. While we only have a few people contributing new magazines, it would be managable. At our height back in 2010, we were having anywhere from 8-14 new releases per week. Plus it seems as though we curse anywhere we put our scans...lol. We started with Mininova and PirateBay, both are gone! We tried UnderGroundGamer, gone! MegaUpload? Gone! Rapidshare? Pretty much gone! If Filefactory keeps up the shady buisness of fake download buttons and forcing people into paid plans.....they are pretty much next. I am just thinking KISS (Keep It Simple Stupid). We go back to allowing people to upload directly to Retromags, and download directly from Retromags. E-Day and myself can then easily fire up a FTP every few weeks and pull down any new files that we may have missed. No worrying about users upload to their own FileFactory accounts, or somewhere else.

It seems like an awful lot of effort just to keep the moaning leechers happy though. Team members who have contributed mags to the site, I can understand wanting to reward them, as the mags wouldn't even exist without those guys but everyone else essentially comes along for the ride when it comes to providing magazines. I know people have contributed to the databases etc so they fall into the middle ground but it's not really the same as providing mags so a tiered structure makes sense there but having Joe Leecher who hasn't done anything other than create a login so he doesn't have to use FileFactory seems like just rewarding mediocrity.

Well of course we would reward the Team members on the site, that goes without saying. It would almost certainly be a tiered system that rewards contributions at every level. Keep in mind that donations can come in handy when rare magazines pop up. Those two Numb Thumb issues set me back $45, if I want the missing three issues of Odyssey Adventure they are on eBay at $50 each......don't even get me started on that first issue of Team Sega Newsletter. I think the last one went for a few hundred.

[KiwiArcader]

Don't get me wrong mate. I understand where you are coming from but having been a member for years I can well remember the local downloads causing all manner of jerks signing up, making stupid posts to get a download and still moaning about how unfair things were. LOL

[Phillyman]

Don't get me wrong mate. I understand where you are coming from but having been a member for years I can well remember the local downloads causing all manner of jerks signing up, making stupid posts to get a download and still moaning about how unfair things were. LOL

Oh for sure, but it is the lesser of the evils out there for us to resort to. I am just getting tired of putting the Retromags collection in someone elses hands only to have them screw us over later. I am looking at this from the point of, how can I make Retromags run better?

1) Direct downloads - Retromags would have complete control over. No worrying about another site changing things or disappearing

2) Restore All Missing downloads - We are still missing items from the big shut down

3) Make it easier for members and staff - Upload to Retromags, Download from Retromags. Nothing gets lost or forgotten!

With bringing the collection back in house, we can stop screwing around with trying to make other solutions work, and work on preserving magazines. That is what we are supposed to be doing

[Lycanthrokeith]

I am right on the edge of putting us on a dedicated server, where we would get 1TB of storage for our magazines and get 5TB of bandwidth for you guys to download directly from us. The only hold up is it would cost us $1400 per year ($119 per month)

Do we have 100 members out of our 4000 that would donate $14 per year to Retromags?

I'd be agreeable to this.

[Phillyman]

I'd be agreeable to this.

Yeah, and I am pretty sure that at least 99 other members registered on this site would be to

As an extra added benefit, the site would be faster because Retromags would be the only site on the server. Plus the next tier of dedicated server allows for 10 Terabytes of bandwidth per month, which should take us into 2020! Right now a mere mention of Retromags on Digg, Kotaku or Reddit could bring down the site in just minutes. On a dedicated server we would not be subject to CPU/Memory limits and would be able to stay up during such an event.

[Lycanthrokeith]

Yeah, and I am pretty sure that at least 99 other members registered on this site would be too

As an extra added benefit, the site would be faster because only Retromags would be the only site on the server. Plus the next tier of dedicated server allows for 10 Terabytes of bandwidth per month, which should take us into 2020! Right now a mere mention of Retromags on Digg, Kotaku or Reddit could bring down the site in just minutes. On a dedicated server we would not be subject to CPU/Memory limits and would be able to stay up during such an event.

It's certainly a safer way to go than the risk of identity compromising and potential litigation, too. That always makes me gunshy about subscribing to a download service.

$14 a year is a whole lot better than $8 a month, too.

Guess the next questions are, would there be any special bonuses for the people who do the donation toward the server cost, and how do we go about donating?

[Phillyman]

It's certainly a safer way to go than the risk of identity compromising and potential litigation, too. That always makes me gunshy about subscribing to a download service.

$14 a year is a whole lot better than $8 a month, too.

Guess the next questions are, would there be any special bonuses for the people who do the donation toward the server cost, and how do we go about donating?

There is an email going out to Retromags members, posted a copy to the front page for those members who have elected not to receive emails from us.

[Lycanthrokeith]

$15 has been donated.

[Phillyman]

$15 has been donated.

Thank you! Much appreciated. Once the funds have been raised, I just need to hit the order button and wait for them to migrate us to the new server. Then its a simple mass upload of everything I have back to the site, and I will open it up to everyone for missing submissions, so we can get back up that much faster.

[Sean697]

That is something I have been really thinking about. While Archive.org could help, how much overhead would it create to keep it all up to date. If a member comes to Retromags and scans a magazine, they would have to upload it somewhere like FileFactory for E-day or myself to retrieve. Then we would need to pass it along to Archive.org, wait for someone over there to upload it and provide us a direct download link, then we could insert that link into the Magazine Database for the correct record. While we only have a few people contributing new magazines, it would be managable. At our height back in 2010, we were having anywhere from 8-14 new releases per week. Plus it seems as though we curse anywhere we put our scans...lol. We started with Mininova and PirateBay, both are gone! We tried UnderGroundGamer, gone! MegaUpload? Gone! Rapidshare? Pretty much gone! If Filefactory keeps up the shady buisness of fake download buttons and forcing people into paid plans.....they are pretty much next. I am just thinking KISS (Keep It Simple Stupid). We go back to allowing people to upload directly to Retromags, and download directly from Retromags.

Actually it would be quite easy. Dropbox really is an ideal tool for uploading magazines to you guys. The free account generate public or private links for files. In fact I just uploaded the EGM 0 I scanned and PMed the private link to Eday to edit it. Archive.org just requires uploading the magazine to the site and making an entry. In fact it is kind of becoming the default place to look for old game mags. But having the highest quality stuff from here would make it the best. Our database would just need the archive page link and could download straight from there, and preserving old game magazines falls well within the scope of that sites purpose. For all those who haven't checked here is a download link already down by John smith.

https://archive.org/details/Next-Generation-1996-09

Of interest is the preview pane where you can flip through the magazine without having to DL it. A direct download of the CBR file. And different version like ePub etc. And if you use the preview pane our website splash page is included. We could rule archive .org with your magazines. Everyone could check out the magazines at little to no effort. Users interface is nice. No shady connection problems or anti virus warnings. Ask John Smith the process of adding mags to the site since he is actively doing it. It should be pretty simple.mi know a lot of gaming vids like Chrontendo and Generation 16 who also post their videos there in 1080p 60fps with no issues.mimwould seriously consider this option.

[Phillyman]

Actually it would be quite easy. Dropbox really is an ideal tool for uploading magazines to you guys. The free account generate public or private links for files. In fact I just uploaded the EGM 0 I scanned and PMed the private link to Eday to edit it. Archive.org just requires uploading the magazine to the site and making an entry. In fact it is kind of becoming the default place to look for old game mags. But having the highest quality stuff from here would make it the best. Our database would just need the archive page link and could download straight from there, and preserving old game magazines falls well within the scope of that sites purpose. For all those who haven't checked here is a download link already down by John smith.

https://archive.org/details/Next-Generation-1996-09

Of interest is the preview pane where you can flip through the magazine without having to DL it. A direct download of the CBR file. And different version like ePub etc. And if you use the preview pane our website splash page is included. We could rule archive .org with your magazines. Everyone could check out the magazines at little to no effort. Users interface is nice. No shady connection problems or anti virus warnings. Ask John Smith the process of adding mags to the site since he is actively doing it. It should be pretty simple.mi know a lot of gaming vids like Chrontendo and Generation 16 who also post their videos there in 1080p 60fps with no issues.mimwould seriously consider this option.

With housing them on Retromags, there is nothing preventing the other solutions from being utilized as well. There are a ton of additional things that become much easier with the files being housed here though. I can write a script that automatically calculates the MD5 checksum for any files uploaded to us, and puts that value in our database so that our newly created collection tool can always provide the most current results. We would have the ability to import in filesizes to the database, we would also be able to quickly fix mistakes in house. A dedicated server gives us a faster website, gives us redundancy and backups. We can also do some bigger picture things that I have planned for Retromags. Again nothing against Archive.org, I would definitely welcome them as a secondary option for our members. Between having direct downloads from Retromags and a backup download from Archive.org, we could pretty much remove all Rapidshare, 4Shared and FileFactory links from our releases. Win win for everyone.

[RetroDefense]

If everything comes back in house, E-Day and myself will have an easy way to sync our collections to the Retromags collection......and users will only need to upload to us.....and everyone can download from here. I can tie credits to donations, but at that point I might as well just tie donations to member groups like we used to do.

So this has been done before? Must've been before my time here at RetroMags.

While I've never been fond of Archive.org's interface (ugh), they do seem to be far more stable and reliable than file-sharing services like FileFactory.

[Phillyman]

So this has been done before? Must've been before my time here at RetroMags.

While I've never been fond of Archive.org's interface (ugh), they do seem to be far more stable and reliable than file-sharing services like FileFactory.

When Retromags started back in 2005, we released everything via Torrents .....example from those days here....

community.retromags.com/topic/883-sega-visions-releases/

I forget what date we went to direct downloads, but Direct Downloads go down on this date....

http://community.retromags.com/topic/5418-why-the-magazines-are-gone/

Then it was a scramble to get everything back up on MegaUpload, then once we were getting close to 100% completion, BOOM....

http://community.retromags.com/topic/7018-megauploads-shut-down-99-of-retromags-collection-unavailable/

Then we start getting things back together again, and Rapidshare changes their terms of service and start becoming an unreliable source for stable downloads. I think it is time to stop acting like this project can run efficiently on a $10-$20 shared webhosting plan.

Torrents? - No one likes to seed

Newsgroups? - Ah too complicated, I have to pay $10 per month for unlimited downloads!

Rapidshare/FileFactory? - Why are you forcing us to use this horrible service?

I know $1400 sounds like big deal of money, but we have 4000 members on Retromags. I have no doubt that we have 100 members that would rather donate $1.17 per month for to avoid using any of the services above. If we ever start getting near the 5TB limit, we can double it for $500 more which if we divided that by 200 members, would come out to $0.79 per month. My end goal here is to stop wasting time on moving Retromags and the magazines, and get back to focusing on preserving the next magazine. I wish I could have done this back in 2009 when the collection first went offline. However dedicated solutions were quite a bit more money and I thought we could survive on MegaUpload.

[Phillyman]

Below are the download stats from the year 2008. This is a year before we had to resort to MegaUpload. As you can see, we were well under 5000GB each month.

and for measure, Decembers 2014 stats

[KiwiArcader]

My web hosting company says in theory I have an unlimited size/bandwidth plan on a shared server although I suspect if I started uploading GB's of mags to my account and making direct downloads available they would soon be asking questions.

[KiwiArcader]

Even though I run my own site I believe that helping preservation sites like this one benefits the preservation scene as a whole so I have donated.

[Phillyman]

My web hosting company says in theory I have an unlimited size/bandwidth plan on a shared server although I suspect if I started uploading GB's of mags to my account and making direct downloads available they would soon be asking questions.

Oh it is an absolute marketing scam. A dedicated server has a 1TB storage limit and 5TB bandwidth limit for $149 per month, while a shared server has unlimited storage and bandwidth for $5.95 per month? They put it into their terms of service wording that will make the "unlimited" claim void and null!

http://www.hostgator.com/tos

1. CPU, Bandwidth and Disk Usage
   1. Permitted CPU and Disk Usage.

      All use of hosting space provided by HostGator is subject to the terms of this Agreement and the Acceptable Use Policy.

      1. Shared hosting space may only be used for web files, active email and content of User Websites. Shared hosting space may not be used for storage (whether of media, emails, or other data), including, as offsite storage of electronic files, email or FTP hosts. HostGator expressly reserves the right to review every shared account for excessive usage of CPU, disk space and other resources that may be caused by a violation of this Agreement or the Acceptable Use Policy. HostGator may, in our sole discretion, terminate access to the Services, apply additional fees, or remove or delete User Content for those accounts that are found to be in violation of HostGator’s terms and conditions.
      2. Dedicated and VPS usage is limited by the resources allocated to the specific plan that you have purchased.
   2. Bandwidth Usage.

      Shared servers are not limited in their bandwidth allowance. Unlimited bandwidth usage is not available for resellers, dedicated or VPS servers, which are subject to the terms of the plan you purchased and can be viewed in your control panel.

You have unlimited storage and bandwidth as long as you are within the standards. They look at their clients and say "On average, our customers use X space and X bandwidth per month. Once you start going over that number, you are preventing them from adding more clients to the server you are on. They know that on each server they should be able to fit X amount of sites, if you are using 100GB of storage.....and they know on average each website should be 2GB, they are going to start looking at you.

Even though I run my own site I believe that helping preservation sites like this one benefits the preservation scene as a whole so I have donated.

Even though you would have already been exempt from any restrictions for your contributions as a Team Member......That means a lot