Retromags Collection Moving to Newsgroups

[E-Day]

One of the biggest problems we have run into over the years, is how to provide these magazines to our users. The first venue we tried were torrents on Mininova and ThePirateBay. Now this was all fine, because the Retromags collection was only around 25-40 magazines. We also benefited from having our name/links on two of the biggest trackers on the internet. People would see our work, and then come to this website to see what else we had. Once our collection of magazines grew, it became quite a chore to keep it seeded 24/7/365.

We then tried housing the Retromags collection locally on this website. The entire Retromags collection was uploaded here and we put quota's on how much could be downloaded. It was a tiered system that allowed donators/contributors to download more than their guest/member counterparts. Then we were hit with an email from our webhost, stating that we were using more than our fair share of bandwidth/storage.

So we went out and signed up for a Lifetime Premium MegaUpload membership. Again we re-uploaded the entire collection of magazines. This time everyone was on the same level, the links were available to everyone. The only restrictions were those that MegaUpload put on users. People who wanted to download the magazines faster, could pay $10 for one month of Premium Membership from MegaUpload. This allowed us to provide the magazines 24/7 and not violate our web hosting terms of service.

Then as we all know, the FBI shutdown MegaUpload and instantly ALL our download links were useless. I started uploading files to the local server almost immediately since we were with a different host, but realistically this is not a viable solution. So we needed a new solution.

Phillyman came up with the idea of putting everything onto Newsgroups. Unlike torrents there is no need to seed. Once a file is uploaded to one newsgroup provider, it becomes available on others. Also the files can remain available for years. Your IP address is not exposed like P2P clients (Limewire, Bearshare, etc). There are no wait times or captcha's to enter like premium download services (MegaUpload, HotFile, Rapidshare).

It is true that Newsgroups are not free, but unlimited access to the Retromags collection was never free. And right now there is no unlimited access. But in order to provide access to the scans without having to re-enter everything in the Download Manager every couple of years, this is probably the best solution.

A lot of people will not like this change, and that is understandable. But it is really the only viable and long term solution that exists apart from not making scans available anymore.

I don't know when this will take full effect. More details to follow soon.

[zdarfan]

Thank you and everyone who works on this site for your hard work. I'm very happy to hear that there will be a long-term solution, and I personally don't care how much it costs.

The downloads from this web site take me back to the days before the internet, where I would spend weeks with a single issue of Game Player's or Nintendo Power, combing through every single article, even about games on systems I didn't own. These magazines are little time capsules into the years when gaming subculture was just developing. Cheers, Retromags staff!

[KiwiArcader]

The only issue with newsgroups is retention. A lot of hosters seem to have variable retention times which can lead to files disappearing, or worse, parts of files disappear. Arghhh!!!

So I'm guessing you had an email from the hosting company again? Usage?

[E-Day]

No, no email yet. But why wait until they shut down the site and send an email? The newsgroup Phillyman uses seems to have a retention time of almost 4 years. We haven't been able to keep files in one place for that long since the site came into existence.

[Phillyman]

I believe Newsgroups will be a good option for providing our members with the Retromags collection. Now I am not saying this should be our "only" option, but this would provide a rock solid way to always have 24/7 access to the scans. Once I have all the magazines in our collection on my workstation, I can upload that entire collection to a newsgroup provider in 4-6 weeks. I only get 250GB of bandwidth per month, so if the entire collection comes out to 200GB, I will need to do it over a two month period.

We can continue to look into other ways to provide the scans, but our options are very limited.

Locally

We are currently on a shared server, which puts E-day back roughly $10 a month. When you are on a shared server, they have rules as to how much of the servers resources you are allowed to use. If E-day puts back all 900+ magazines, they will see that Retromags went from using 1GB of storage space, to 200GB of space over the course of a few weeks. This will set off their alarms and we will get another letter. Plus when the files are hosting locally, we have to limit our members on how much they can download each day, week, month. Right now Retromags probably uses a few gigs of bandwidth each month, put the files back and that is going to be 100X more. It will alert the hosting company, and they will probably just disable the entire site and send E-day a nasty letter.

Now if Retromags had its own DEDICATED server, then all the disk space is ours, and all the bandwidth. But you are looking at around $174 per month for a dedicated server plan.

http://www.hostgator.com/dedicated.shtml

So unless Retromags can pull $2088 per year in donations.....

Torrents

Torrents were the other option, but that requires someone to seed each file/collection of files. We tried this option back in the day, and NO ONE, other than myself and two other people were seeding. I watched people hit 100% for a file, and IMMEDIATELY disconnect. All this does is aggravate the people who are seeding. They are getting magazines for nothing, I am spending my bandwidth to seed, but then they can't return the favor.

There is also private trackers like UG, but that requires each of our members to get accepted into their website to have access to the files. Then our members have to obey the ratio rules and such that are in place over there.

File Share Service

We could always go with another MegaUpload clone. There is Hotfile, Rapidshare and a ton of others. Considering how fast MegaUpload got their plug pulled, why expect different if we move to another? It would again require time to upload each file, and money to buy a Premium Membership. Would it provide people a way to download, yes. But they would be subject to HotFile/Rapidshare's download quota.

NewsGroups

Now lets look at Newsgroups.....

• Upload Once, Available 24/7
  
• Upload Once, Available for Years
  
• Downloads, As fast as your ISP will allow
  
• Downloads, As many as you want
  
• Requires No Seeding
  
• No Captcha To Enter
  
• No Wait Times
  
• No Ratio to maintain
  
• Does Not violate or TOS for this webhost
  

The only issue is that each member that wants to use this method, will need to subscribe to a premium newsgroup provider. But! For as little as $3 a member could sign up for a 3 day membership with Unlimited Downloads!

UseNetServer $3 for 3 Days

Unlimited Downloads = Unlimited Magazines!

Click Here!

So, if a member REALLY wants to download a magazine....And there are no other avenue's to do so. They can always pay $3 and download as much/fast as they want.

Its not a perfect plan, but its 1 STABLE option for providing magazines to our members. We will always continue to look for others, but its nice to know that the entire Retromags collection is available to our members.

PS - Look at this...Back in 2009 I uploaded some of the Nintendo Power's to a few different newsgroup categories. As of last night, I was still able to download those files! They have been up and available for the past 1061 days!

http://binsearch.info/?q=Retromags&max=100&adv_age=1100&server=

[Phillyman]

Locally

We are currently on a shared server, which puts E-day back roughly $10 a month. When you are on a shared server, they have rules as to how much of the servers resources you are allowed to use. If E-day puts back all 900+ magazines, they will see that Retromags went from using 1GB of storage space, to 200GB of space over the course of a few weeks. This will set off their alarms and we will get another letter. Plus when the files are hosting locally, we have to limit our members on how much they can download each day, week, month. Right now Retromags probably uses a few gigs of bandwidth each month, put the files back and that is going to be 100X more. It will alert the hosting company, and they will probably just disable the entire site and send E-day a nasty letter.

Now if Retromags had its own DEDICATED server, then all the disk space is ours, and all the bandwidth. But you are looking at around $174 per month for a dedicated server plan.

http://anonym.to?htt...dedicated.shtml

So unless Retromags can pull $2088 per year in donations.....

What about making this a pay site for anyone who wants to download?

I would be more than happy to pay a one time charge for the privilege of being able to download all of the mags and I would think you would make more than enough to pay for a dedicated server AND make a little bit of profit for your hard work at the same time.

Maybe you could set it up so that non paying members can only view the mags without the ability to download and people who pay a one time fee can download as much as they want or whatever daily download limits you want to enforce.

I'm not sure how practical this would be, but wanted to throw the idea out there...

The only issue is that we should not be turning a profit from these scans. If a few people want to kick us a couple dollars so that we can get a magazine, that's fine. But we would be no better than the people on Ebay selling these scans if we made it a pay "US" system.

[terpsfan101]

Every scan that E-Day and I have can be found here:

http://www.retromags.com/forums/topic/7043-external-links-to-magazine-scans/

Isn't this a more viable option than downloading from newsgroups? As long as mediafire doesn't tank, the links will be working for years to come.

[room88]

I think newsgroups is a great idea.. and a way I've been thinking it should have been done for a long while. You just upload everything at once and bam, they're there for a good 3 years on most good providers.

And the great thing is, it doesn't have to preclude other means of distribution. If an alternative means (such as the web-based mediafire suggestion above) etc. is found to be viable, it can live side by side. Again, the upload to Usenet is a one shot deal, and it's there for a good long time with zero maintenance. Other methods of web access can come up and be used with no problems.

If anything it's an excellent win-win with no drawbacks whatsoever.

[E-Day]

I'm not too sure about that. A one time fee of ten to twenty bucks for the rights to access and download the entire library is a lot different than charging per mag or for a small collection of mags. I can only speak for myself, but I would have no problem with paying that fee and no issues if you guys were to turn a decent profit at the same time.

I think it's awesome that you guys take that stand, but it may become impractical due to circumstances out of your control.

However it goes down, I appreciate you guys fighting the good fight and keep up the great work!

P.S. - If you guys ever get to a point that you need donations to keep this site going, don't be bashful about saying so. I think you would be amazed at the immediate and extremely positive response you would get!

I do not feel comfortable charging people money to access content that we do not own or even have a right to scan and make available, even if it's a one-time fee. That is a sure-fire way to get content owners - who have ignored us and let us be up to this point - coming after us and asking us not to host their stuff or worse. They know we exist, but they leave us alone. And I think they do so for two reasons:

1. The magazines we have are many years older than anything some of them might be offering as back issues.

2. This site has no means of revenue generation: no fees, no ads, not even a donation button (though I am not opposed to the last item).

This site was never about turning any kind of profit, nor should it be. It hasn't even been about covering the costs of running the site, though donations did help with that in the past.

Every scan that E-Day and I have can be found here:

http://www.retromags.com/forums/topic/7043-external-links-to-magazine-scans/

Isn't this a more viable option than downloading from newsgroups? As long as mediafire doesn't tank, the links will be working for years to come.

Not really. Think of how much time and work it took to upload everything to MediaFire, and how long it took to post all the links in that thread. There is no promise that MediaFire won't close down tomorrow, nor is there a promise that they won't delete files on their own if they realize that they are copyrighted works. If either of those things happen, then you have to start all over again. Files on Newsgroups may not last forever, but they will last longer than anything we've attempted in the past. I don't think any methods used by this site before have lasted more than 1061 days. Newsgroups won't be the only method we use, but it will be the main one. Hopefully it will let me get rid of the Download Manager

[terpsfan101]

Do you remember which magazines you clicked on? I just tried various links from Edge, EGM, GamePro, and Nintendo Power (not logged into my mediafire account) and they worked fine.

[Ferneu]

Phillyman came up with the idea of putting everything onto Newsgroups. Unlike torrents there is no need to seed. Once a file is uploaded to one newsgroup provider, it becomes available on others. Also the files can remain available for years.

That depends solely of the server. On most I ever used, "years" is not a good word to describe how long the file stay up.

Your IP address is not exposed like P2P clients (Limewire, Bearshare, etc).

You guys do realize that is very, very easy trace anything back to you, doesn't matter if you are using torrents, direct HTTP, FTP, etc. It does not even matter if you are behind a transparent anonymous proxy, one that do not tell it is a proxy to the server and do not keep nor forward your IP address.

Do you remember that old Hackers movie, the one with Angelina Jolie? When they said something like "what? You did it from your house? Dude the FBI/NSA/WhatEver3LetterAgency will get you". Many things on movies are not true, but that is not one of them. If you think you are safe because your IP is not showing on the "peerlist" or whatever, think again. Your false sense of security is based on the other peers not knowing you IP. But it is not the other peers you should be afraid of. The powers that be, the ones who will take you down and to the court, are not among the other peers. They do not care if your IP is being public displayed or not. If they wish to take you down they will simply trace it all back to you. Unless you have a cable connected directly to the server, which I highly doubt you do, your connection is passing through an ISP, and an ISP knows to what MAC address a certain IP was lent, they know what requests came from there. Of course they know. They have to know that, otherwise how would they fulfill it. That is simply how things work. Nothing new about it. And that is all what it take to get to your house.

1- The target server is 1.1.1.1 and this IP belongs to ISP A

2- Alright ISP A, give me a list of IP that have connected to 1.1.1.1.

3- 2012/03.31 20:00:00 server 1.1.1.1 received a request from IP 2.2.2.2

4- IP 2.2.2.2 belongs to ISP B

5- Alright ISP B, give me the MAC to which IP 2.2.2.2 was lent. OK, now give me the street address of that MAC.

6- Ding Dong. "Hi my name is Mr Smith, I work for the Interpol and you are under arrest"

Anyway, that is just my 2 cents about the subject. Please do not consider this just a another whining "buahhh, now I am going to have to pay" post. After, as I said before, I already have almost every old mag I wanted. I just wanted to give you guys a heads up about how weak is the shield you are using. Anyway, you guys should do whatever you think is best for the site. And whatever you decide, I wish you good luck and may everything workout for the best.

[E-Day]

No matter what we do, there is no fail-safe way of giving access to the magazines while making everyone anonymous. This is more about having some stability as far as the files staying in one place as opposed to having to redo things every couple of years.

[KiwiArcader]

My question is this .....

If you go with newsgroups as the method of providing the mags what is the point of even having the retromags website at all? You can just as easily annouce a new mag via Twitter etc so what's the point of even having a forum in this situation?

The whole point of running a site like this I would have thought is to provide a platform for making content available locally which encourages visitors and by that definition continued interaction and involvement with them. If you no longer have download links, eg, just a disclaimer saying they are hosted in alt.binaries.x.x then people who don't have newsgroup access will simply stop coming here and those with newsgroup access won't either as they don't need to anymore. And with newsgroups you have no control of what content goes into the directory so you may as well dump the year restrictions in which case removing the retromags website is beneficial as it removes that avenue of having publishers attacking you anyway.

I think this site will become fairly defunct as a result of this decision if you don't find/finance a method for linking to direct downloads

[Phillyman]

I think this site will become fairly defunct as a result of this decision if you don't find/finance a method for linking to direct downloads

You know, I could go back and forth with you. But Retromags is no longer my site, so I am just going to leave it alone.

[KiwiArcader]

So have your say!! This is a democracy and everyone who participates on the site is entitled to voice their concerns. I am very interested to hear how you think the site won't be affected by the decision.

When downloads were freely hosted off Megaupload the volume of people signing up to the site was extremely low because they could simply leech and not have to contribute to the site in any way. That abruptly changed when MU was taken offline and E-Day started hosting them on the site host & put restrictions on again. We suddenly see people spamming the forums to get the necessary posts to enable them to leech again. Do you seriously think that freely hosting them on newsgroups won't result in people not visiting the site? Why should they anymore when files are no longer hosted here or are no longer only obtainable from this site via a download link? Sure, the dozen off us who frequent this site continually won't change our habits but that'll be about it if past indications are anything to go by.

So again ... I am interested in your thoughts and E-Day's of course.

[Phillyman]

So have your say!! This is a democracy and everyone who participates on the site is entitled to voice their concerns. I am very interested to hear how you think the site won't be affected by the decision.

When downloads were freely hosted off Megaupload the volume of people signing up to the site was extremely low because they could simply leech and not have to contribute to the site in any way. That abruptly changed when MU was taken offline and E-Day started hosting them on the site host & put restrictions on again. We suddenly see people spamming the forums to get the necessary posts to enable them to leech again. Do you seriously think that freely hosting them on newsgroups won't result in people not visiting the site? Why should they anymore when files are no longer hosted here or are no longer only obtainable from this site via a download link? Sure, the dozen off us who frequent this site continually won't change our habits but that'll be about it if past indications are anything to go by.

So again ... I am interested in your thoughts and E-Day's of course.

Forced participation never works out (my thoughts). I think E-day wants people to help because they enjoy preserving these magazines. If E-day wants to force participation, he could always do so in a mixed method. Release the existing 900+ magazines on Newsgroups, then keep the current 6-12 months worth of releases on Rapidshare and make it so that only members/contributors have access. Somewhat like how the Angry Video Game Nerd only puts new videos up on his website, and then on Youtube a year later.

[KiwiArcader]

Forced participation never works out (my thoughts). I think E-day wants people to help because they enjoy preserving these magazines. If E-day wants to force participation, he could always do so in a mixed method. Release the existing 900+ magazines on Newsgroups, then keep the current 6-12 months worth of releases on Rapidshare and make it so that only members/contributors have access. Somewhat like how the Angry Video Game Nerd only puts new videos up on his website, and then on Youtube a year later.

That may indeed work. The big question is whether E-Day wants to fork out money on a hosting company like Rapidshare in the current climate? He stated he wasn't keen on going back there but the implications to doing that rather are less than having the hosting company take exception to having locally hosted files and loading files as a free user doesn't work well as after a period of inactivity, or worse in some cases, the files are taken down.

The other issue is with newsgroups there is no way to stop other content being loaded into the directory. Say you create alt.binaries.magazines.retromags. What's to stop someone loading the latest issue of XBox mag over there? Nothing ... so RM still has the potential of being implicated in pirating current content.

[E-Day]

If they don't visit the site, then they don't visit the site. I am not that concerned about it because the number of visits doesn't affect me. I don't run ads on here, so whether the site gets 10 visitors a day or 1000 makes no difference except in the amount of work it takes to moderate things.

The site didn't become defunct in the early stages when things were hosted through torrents. People still came, people still contributed. I don't want people posting simply to gain access to stuff and not contributing quality posts to the community. I have seen sites that do that (frihost.com for example), and some of the posts are ridiculous. People who truly have an interest in gaming magazines of days gone by will participate, and will want to contribute. That is what happened to me. I could have easily just paid for my lifetime membership when I signed up in 2007, and downloaded what I wanted and that's that. But I was so thrilled about what was being offered and that people were helping out, that I wanted to help to. I wanted to become a regular member of the site instead of a lurker who just paid some money and downloaded stuff. These are the kind of people that would make the site better.

By no means is Newsgroups the ideal way of doing things. But how many times is this site expected to shuffle 900+ scans around, and then changing all the links in the Download Manager? Torrents will not work, nor are they really feasible for the size of our collection. Local hosting is like playing Russian Roulette; eventually the web host will stop that. Premium file hosting sites are a gamble; you either pay for a membership and hope that they don't get shut down too, or your files get deleted for copyright violation or because they've been inactive for too long.

The ONLY way to ensure that we wouldn't have to do this file and link shuffle every couple of years is to go to each publisher/content owner, and get their permission to host their stuff, and submit that permission to the web host. But realistically, how likely is it that we would actually get permission from them? Chances are better that we would actually lose everything. THEN the site becomes defunct. The way Phillyman explained it to me, Newsgroups can offer far more stability than what we have used so far. And you can download as many magazines as you can manage through Newsgroups. Every other method that I know of throttles you in some manner. There are issues with Newsgroups for sure, but are they worse than the issues that come with other methods of hosting these files?

Phillyman's idea of a mixed method with new releases going on something like Rapidshare for the first few months isn't a bad idea. I won't be paying for a membership though since they could close down tomorrow and take my money with them

This site is here to preserve magazines. It's not here to become popular, or to make money, or to make downloading super easy and free. It's not about this site; it's about keeping the magazines preserved. If they are a bit harder for some to get to because they are on Newsgroups, or if this site gets less traffic because of that, that is fine. The goal is to have those magazines floating out there somewhere long after this site is gone.

[KiwiArcader]

Then maybe it's a good time indeed to look at making the site easier for you to manage by changing it to more of a informational site on what is in a particular mag" rather than a full blown forum with all the attendent issues with maintaining it?

You could turn the site into a Wordpress type site .... very easy to write page data into ... and comments are available against every page anyway ... and people will indeed submit if they want to. That way you get the benefit of reduced workload and you can still give access to content uploaders as required. It seemed like originally you wanted to hand the site off to someone else due to the workload and now you're introducing more. LOL

[E-Day]

I've been thinking about doing that (while keeping the forums), but that requires a lot of work and a lot of time on my part, which I don't have right now.

[KiwiArcader]

I've been thinking about doing that (while keeping the forums), but that requires a lot of work and a lot of time on my part, which I don't have right now.

Well ...with something like Wordpress you just set a template up and copy/paste into a new page for the mag and populate with the information. Additionally, you can add users you trust, just like here, to create new pages/update existing etc. But you are right ... there is certainly some work involved although I'd wager less than maintaining forums.

[Schloss Ritter]

Just wanted to note that the UG Torrents for 420's Nintendo Power sans are still up after 1850-2200 days. Many have only a couple seeders (I agree there's a bit of a problem there), but the two oldest (in PDF, ugh) still have over a dozen seeders.

[Anchordude]

One of the biggest problems we have run into over the years, is how to provide these magazines to our users.

<SNIP>

Phillyman came up with the idea of putting everything onto Newsgroups. Unlike torrents there is no need to seed. Once a file is uploaded to one newsgroup provider, it becomes available on others. Also the files can remain available for years. Your IP address is not exposed like P2P clients (Limewire, Bearshare, etc). There are no wait times or captcha's to enter like premium download services (MegaUpload, HotFile, Rapidshare).

It is true that Newsgroups are not free, but unlimited access to the Retromags collection was never free. And right now there is no unlimited access. But in order to provide access to the scans without having to re-enter everything in the Download Manager every couple of years, this is probably the best solution.

A lot of people will not like this change, and that is understandable. But it is really the only viable and long term solution that exists apart from not making scans available anymore.

I don't know when this will take full effect. More details to follow soon.

As a long time Usenet user, I think you've come up with a very good solution. A few thoughts to add:

Come up with a mutually agreeable naming convention - the subject line for the posts doesn't have to be the same as the file names. Many groups that are using Usenet for storage obscure the name of the contents to remain "under the radar" - however this is considered poor netiquite in the Usenet community.

For those doing serious posting, you'll want to get a copy of PowerPost - it's the best Usenet posting tool for large groups of files. If you are just posting onesies and twosies, then posting abilities in Newsbin Pro or one of the other newreader programs will probably be just fine.

As soon as a collection is posted, use your newsreader program to create an NZB file of all the article numbers for the parts that were uploaded. (NZB file is similar to a torrent file).

This forum then becomes the place where you can post the NZB files along with a more descriptive list of the contents.

There's a secondary reason for posting the NZB file here in the forum. Whether you partition for a new usenet group or just commandeer an abandoned one, the newsgroup itself will become littered with spam in no time flat. Finding the articles you want will become a nightmare. But if posters add the NZB file to the forum here - no-one needs to even search the newsgroup headers for entries - and everyone is saved the headache of dealing with all the spam.

Note that almost all the reputable usenet providers follow DCMA guidelines and will remove any set of articles that a publisher says is theirs - so just as you already do here, you need to be ready to accept that some postings may be removed. Often times boards have a semi-formal way where forum members can report that they have found that a file has been DCMA removed so others do not waste time trying to access it. There's also a line of reasoning that you should NOT obscure the subject names in usenet posts if you will be keeping a cross reference between names and actual contents in your forum. Doing so could put you at risk of being accused of knowingly posting stuff that is not free and clear of copyright concerns. This community already seems to already take the high road on the topic, so I see no reason to try to hide names when posting. If an owner makes a fuss and has the usenet providers remove the articles, it's not as if you were purposely trying to get away with something illegal.

And yes - I have a vested interest in all this. I've been using Usenet for well over 10 years, and running a Usenet recommendation and FAQ site for nearly as long. I've built some comparison tables for providers, different newsreaders, insights into who does what in the industry, who's feeding which sites, etc. www.newsgroupservers.net

And for those concerned with ISP's tracking your IP, you can get real anonymity by doing everything through a VPN account. I have a website about that too: vpncompanyreviews.net

I'm not here to shill them, so I won't say anything more about it. Just a lurker who wanted to throw in his two cents worth.

Keep this stuff alive guys!

Bruce of the Anchodudes

[Digimon]

I use the binaries, since mor then 10years.

and to my mind it's the best solution.

[flacnvinyl]

Is it ludicrous to use more than one method of distribution? Personally I love torrents and use several trackers to trade concert recordings. I personally would volunteer to host the big torrent indefinitely. If there were serious implications and legal complications with doing so, this site would have received a C&D and the hammer would come down.

I am currently seeding the big torrent on every public tracker. I was only able to get about 49% of the torrent and no one is seeding. If just one person were to seed the whole thing for a week, I'd have it complete. I intend to seed it indefinitely with the hopes that others will join me.