Retromags Roadmap for 2016

[Phillyman]

Due to the length of this post, best to read it in the forums and not on the front page.

Click!

As we go into 2016, I wanted to take a moment and share with you what to expect this upcoming year. I sometimes forget that I may have an idea in my head, but you guys have not yet been brought into the loop. There seems to be a bit of resentment and hostility towards us preventing bandwidth abuse at Retromags. Lets forget about the restrictions for a second and concentrate on the functionality of this site. Retromags was created to be a centralized location for members to come together and preserve video game magazines. Back in the very beginning, Retromags had ZERO files on our site. We simply existed as a way to put our minds together and get these magazines preserved before all physical copies were lost to time.

The Retromags download manager, which is that Downloads tab up under our logo....exists to let people download magazines. This system CAN allow users to download our entire collection, but it CAN'T allow all our users to download the entire collection. If you go into our Magazine Database (Databases tab > Magazines), you will see that we have programmed additional download fields into all the magazine records. This was done so that Retromags could allow its members to create redundant mirrors of any files we provide. These redundant user created mirrors of our files would allow guests and members the ability to download more magazines in a day.

Lets step through plan for this year....


Finish Restoring the Download Manager - When MegaUpload got shut down, we got screwed over! Our Team Members were uploading their scans to MegaUpload and sometimes E-Day and I did not get to pull a copy of their contributions to our workstations. I fear that some of these users have moved on and that we may never get those releases back up on Retromags. I believe I have somewhere around 100-200 more files to restore to our Download Manager before I can check this off my list. Some of these items I need to restore, I am unsure of where to stick them, as I don't have much information on where the magazine is even from.


Error Checking Restored Files - For the past 10 years members have uploaded items to Retromags. Sometimes mistakes find their way into our system. Maybe a file was too new to be uploaded, maybe some pages were missing or in the wrong order. Whatever the reason is, I am hand checking items that get put back into the download manager, page by page. I am making sure that all the cover shots and the same size, the names of the files are correct, everything gets checked before I restore it to Retromags.

Move Everything to Newsgroups - Once I have error checked and reuploaded everything to this site....its time to put the collection on Newsgroups. Files only need to be uploaded once to Newsgroups, and they can be downloaded by millions of users for years afterwards. Once they are on Newsgroups, i will create .NBZ files to add into the databases, and this will give everyone a SECOND method to download our releases.

Move Everything to FileFactory - While most of what we have in the Download Manager is available on FileFactory already, sometimes there are minor differences between those files. I want to make sure that the version of a release that is in the download manager, is the same as the version on Newsgroups and FileFactory. So that is why I will be reuploading all the files to FileFactory and reimporting all the download links into the databases. This will give everyone a THIRD method to download our releases. This method would be best for users who want to catch up and get all our entire collection.

Find Users to Torrent - I will not torrent the Retromags collection. But I will see if I can find a few members here at Retromags that are willing to torrent our releases. Again once I have the Retromags collection gathered together and error checked, I can easily throw it on a spare 200GB SATA drive and mail it to a member that is willing to seed our releases. I am not much into the torrent scene, but I would probably expect our releases to be better seeded on a private tracker where seeding is enforced. But in the end I leave it up to the members willing to seed to figure out where best to distribute the collection. If I can find people to torrent, this would be a FOURTH method to download our releases. This method would be best for users who want to catch up and get all our entire collection.

Find Members with Extra Bandwidth - From time to time I get PM's from members that are willing to mirror some or all of our releases on their servers. This circles back to those additional download methods in our databases. Once I have the collection error checked and back in the Download Manager, I can again take a hard drive and fill it up and mail it off to these members who are willing to assist us. This would be a FIFTH Method to download our releases.


So that is my plan for the Retromags Collection. Now the question pops up from time to time...

Why dont you just put all your files on X, and fall back to a $20 hosting plan again?

Sure, I could upload everything to Newsgroups or FileFactory and not do Direct Downloads at all. However this is EXACTLY what we have done for the past 5 years! Since 2009 Retromags has hosted none of its collection. How did that work out for us?

• MegaUpload - Gone in a night!
• Rapidshare - Gone
• FileFactory - Getting pretty crappy these days!

When we hand over the Retromags files to other sites, Retromags is at the mercy of those sites. If that site disappears tomorrow, we are screwed. If we put all our eggs in someone elses basket, we have to hope they keep their doors open, we have to hope they dont run malicous popup ads, we have to hope they are not tricking our members into clicking false download buttons. We have to hope their copyright values align with our own. Do we want our preserved magazines showing up on a Emulation website that offers ROM downloads? This is why even if a member offers to mirror the Retromags collection, it needs to be the secondary method.

Also with haivng the Retromags collection stored on OUR SERVER, Both E-day and I can keep better track of the Retromags project. If we just let users upload elsewhere, maybe USER1 stores all their releases on 4Shared, maybe USER2 puts all their stuff on Mega. When we require members to upload directly to Retromags, those files are able to be backed up, our community can be notified immediately of a new release, and it makes running this project a lot easier.

With running the project within the Download Manager, we can control the project. We get tons of users that mean well but try to upload current magazines or magazines that were found on the Internet. If they are just allowed to upload to external sources and put a link in the database, we have no way to prevent that. Running this project, we want to make sure that magazines submitted to Retromags meet a certain level of quality.

Now on the subject of keeping track of the Retromags collection, that brings me to the Retromags Collection Tool I am creating.....

Retromags Collection Tool - One of the issues with running a project such as Retromags is keeping track of everything. Our newly created databases allow for our members to go and change statuses on magazines. This ensures that two members aren't scanning the same item. But how do we know that the digital copy of Nintendo Power Issue 1 on our hard drives is the most current version? Maybe you downloaded that issue, but a few months later a member pointed out that a page was in the wrong spot, or that the poster was never scanned. How do you know if the version of Nintendo Power Issue 1 that you downloaded from 4Shared, is the same version that we have on this site? Perhaps someone downgraded the quality of our release before sharing it on the web.

Simply put, the Retromags Collection Tool works like Cowerings GoodTools does for ROMS. When a file has been digitally preserved for Retromags, the Team Member who created the file runs his finalized .CBR/.CBZ/.PDF through any MD5 calculator to get its digital fingerprint or MD5 Checksum. If any changes are made to that file after the MD5 Checksum has been created (remove pages, add pages, downgrade quality) the MD5 checksum will no longer match. The Retromags Collection Tool leverages the power of our databases to allow Team Members to add their checksums into the database, and then our tool pulls down that MD5 Checksum with a few key values....

For example, here is the information for Nintendo Power Issue 50....

81FE3747C75375CBC5EB88FE27AF370B, nintendo_power_050_-_1993_jul, nintendo_power, usa

This tells our collection tool, that the MD5 value for Nintendo Power Issue 50 is 81FE3747C75375CBC5EB88FE27AF370B, and that if our member wants it to be officially renamed, that its name is nintendo_power_050_-_1993_jul.cbr and if our member wants it officially sorted, to put it in the folder...

usa
|_nintendo_power

The program will also generate a list of what you HAVE and are MISSING from our collection. Making it easy to know what files you still need. Again this all comes back to making things easier for the STAFF and USERS at Retromags. If a file has a more updated version of itself, the Team Member can simply add the new MD5 value, and everyone will know that a more current version is avaiable.

One of my other pet projects is the Video Game Advertisements....

Video Game Adverts - I really love old school video game advertisements. We have a Gallery section devoted to them, but it has been stagnant for quite some time. I want to start going through our magazines and pulling out any adverts that are not yet posted in this section. I also want to take those adverts and start linking them to their coresponding Video Game Datbase pages.

The Bridging Database - Right now the Magazine and Video Game databases are seperate. I have a crazy idea to connect them via a third database. This new database won't be terribly hard to create, but needs to be thought out in detail. Basically this new database would allow you to fill out fields like.....

• Magazine Title
• Video Game Title
• Page Type
• Sub Type
• Review Scores
• Notes
• Pages

Which then you could put...

• Electronic Gaming Monthly Issue 39
• Mario Paint
• Review
• None
• Steve 8/10, Martin 9/10, ED 8/10, Sushi-X 8/10
• Martin says "While this card is not what you would call a video game, I found it to be a blast!
• Page 24

You could then go into this new database and request to be shown data based on criteria you put forth. Examples being....

• Select all reviews from EGM in the year 1995
• Select all prototype mentions in GamePro
• Select all interviews in Next Generation
• Select all mentions of Donkey Kong Country in Nintendo Power

This new database would be very easy to add a new piece of information into, less than a few seconds...But it could potentially be a game changer in how we look for data in these magazines. Just imagine being able to call forth any mentions of Project 64, or the Super Nintendo CD addon. This new database would create permanent bonds between the games and the magazines. Right now this exists in my head, and I have to thank Nintendo Power for putting it there. If you remember back in the day...Nintendo Power sent out thier Nintendo Power Indexes, which told you EXACTLY which issue they mentioned a game in, what page it was on and what type of mention it was. I see this being recreated on Retromags, to allow us to find information very quickly.


Another idea that is in my head and I would like to see come to life....


Magazine Collection Fields - I would like to leverage our existing magazine databases to give our members the ability to keep track of their own collections. I see many video game sites that do this, where a user can go to a video game page and click (Have, Want, Selling, For Trade). I see something similar coming to Retromags, where users can visit magazine pages around this site and build a list of what magazines they have and which they are missing. This could potentially match up users who want to trade magazine, or buyers with sellers of magazines.

Magazine Covers - Yet another side project of mine is to start organizing my own physical collection of video game magazines, and get any missing covers that I can provide into the Gallery/Database. Most of my collection is boxed up in a utility closet and I always seem to throw my back out when I lift boxes of magazines.


And finally, that brings us to a Retromags Store!

Retromags Store - Over the years of doing this, I have too much stuff. There are some magazines that I have 3 and 4 copies of. I have posters, trading cards, pogs, stickers...you name it. I really hate to throw anything away, especially if another person can use it. If I were to list everything on eBay, I would have to continously pay listing fee's to do so. If I list these items on Retromags, I only need to pay Paypal if an item actually is bought. I have no time to run a store over at eBay and quite frankly they are expensive. eBay would mean higher prices as I would have to pay them 3 different fee's (Listing Fee , Selling Fee, Recieving Money Fee). Much easier to just list them over here. Look for more information about this in a week or so.

[Anime-niac]

Move Everything to Newsgroups - Once I have error checked and reuploaded everything to this site....its time to put the collection on Newsgroups. Files only need to be uploaded once to Newsgroups, and they can be downloaded by millions of users for years afterwards. Once they are on Newsgroups, i will create .NBZ files to add into the databases, and this will give everyone a SECOND method to download our releases.

When you do this, please make sure you create and post 20% par2 (parity) files, either per issue (requires fewer parity files) or publication (requires more parity files). When I tried to download the current set on usenet at the beginning of the year, approximately 1/3 of the files were either missing parts or came through damaged. If at least 20% parity files had been posted, I might have been able to complete everything.

I can also help with posting to Usenet and/or creating par2 files.

Find Users to Torrent - I will not torrent the Retromags collection. But I will see if I can find a few members here at Retromags that are willing to torrent our releases. Again once I have the Retromags collection gathered together and error checked, I can easily throw it on a spare 200GB SATA drive and mail it to a member that is willing to seed our releases. I am not much into the torrent scene, but I would probably expect our releases to be better seeded on a private tracker where seeding is enforced. But in the end I leave it up to the members willing to seed to figure out where best to distribute the collection. If I can find people to torrent, this would be a FOURTH method to download our releases. This method would be best for users who want to catch up and get all our entire collection.

You can count me in for this. I usually have at least one computer running 24/7 (unless I'm on vacation), so keeping one or more torrents going isn't a problem.

Two more download options to consider are ED2K and DC hubs. I'm not sure how popular these two methods are anymore as it's been a couple of years since I've actively used them, though I still use the programs every now and then.

[Phillyman]

When you do this, please make sure you create and post 20% par2 (parity) files, either per issue (requires fewer parity files) or publication (requires more parity files). When I tried to download the current set on usenet at the beginning of the year, approximately 1/3 of the files were either missing parts or came through damaged. If at least 20% parity files had been posted, I might have been able to complete everything.

I can also help with posting to Usenet and/or creating par2 files.

You can count me in for this. I usually have at least one computer running 24/7 (unless I'm on vacation), so keeping one or more torrents going isn't a problem.

Two more download options to consider are ED2K and DC hubs. I'm not sure how popular these two methods are anymore as it's been a couple of years since I've actively used them, though I still use the programs every now and then.

If you could write me up a small tutorial on how you would go about pushing them out to Newsgroups. It would be very helpful, programs you use, methods, etc. I believe when I last pushed everything to Newsgroups, I broke down each individual magazine into 10 or 15MB parts, and then made 10 or 15% parity files for each set. I am pretty sure I did a few things wrong, but this time I would like to get them up there with no issues.

[Areala]

Chiming in here to say I have been culling ads from the magazines posted here for the past year, and I've got WAY more than what the gallery currently holds. The bottleneck to posting them here is re-sizing everything. Even with the batch function, it takes forever to do it, and even longer to upload since I have a very slow DSL connection (about 15k/sec upload speed).

Along the way as I've been collecting the ads, I've also been making sure to find the best-looking copy of each ad, so it's not uncommon for me to have gone through 3-4 copies of the same ad looking for the best resolution, smallest amount of bleed through, sharpest lines, etc...

When it comes time, I almost wish we could wipe out the current ad database and start over with me using the batch uploader here at Retromags to dump everything on the server while I'm at work, or not watching lesbian porn sleeping. Seems a shame to wreck all that hard work though, so maybe a file-by-file basis would be easier.

*huggles*
Areala

[Phillyman]

Chiming in here to say I have been culling ads from the magazines posted here for the past year, and I've got WAY more than what the gallery currently holds. The bottleneck to posting them here is re-sizing everything. Even with the batch function, it takes forever to do it, and even longer to upload since I have a very slow DSL connection (about 15k/sec upload speed).

Along the way as I've been collecting the ads, I've also been making sure to find the best-looking copy of each ad, so it's not uncommon for me to have gone through 3-4 copies of the same ad looking for the best resolution, smallest amount of bleed through, sharpest lines, etc...

When it comes time, I almost wish we could wipe out the current ad database and start over with me using the batch uploader here at Retromags to dump everything on the server while I'm at work, or not watching lesbian porn sleeping. Seems a shame to wreck all that hard work though, so maybe a file-by-file basis would be easier.

*huggles*

Areala

Well what would get lost by starting over again? If you need me to send you a 32GB flash drive or SD card and you want to send them back to me for importing under your name, just let me know. I was just going to start by pulling the single page adverts out of all our magazines and making sure the the file name is the name of the game, so that when they get uploaded, they are searchable.

[TOLUST]

if you would tell me the missing magazines before MegaUpload went belly up I can try to find it from my collection on what I downloaded from Day one here from Retromags

[badinsults]

I have actually been musing about a video game magazine bibliography database recently, nice to see you have plans for one. I actually drew up some notes about my thoughts on it last week, but I will copy them here.

The main reason I want a bibliography is for my research on beta and unreleased game stuff, but obviously such a system could be useful for any video game researcher. As of right now, most information in magazines can only be acquired by directly looking at the magazines. This requires that you either have physical copies of the magazines, or a huge digital collection of scans. There is no way to just hit ctrl-f to find information on a particular game.

There are many types of information that can be gleamed from magazines. Most notably:

• Previews
• Reviews
• Advertisements
• Cheats
• Strategy guides
• Upcoming games lists
• Contests
• Rumours lists
• Trade show overviews
• Posters/ Swag
• News
• Letters/ Editorials
• Sales/ "Power" rankings
• Interviews
• Table of contents
• Q&A

There are two potential ways of doing a bibliography:

1) All encompassing

This would be the simplest approach, and maybe wisest. In this approach, you have the following pieces of information:

• Game Name
• System name
• Magazine Name
• Issue Number
• Publication date/time
• Page Number(s)
• Information type (see first list)
• Screenshot count
• Notes

In the notes section, you could further describe the type of info (e.g. review scores, authors, type of cheat). This is sort of what I was doing on SNES Central already, but it is a bit unwieldy to update and search at the moment.

If such a bibliography system were used for other systems, you would also have to add that to the list.

2) Split things up

This would probably be more effort, but has its own rewards. in it, you have several subsections of bibliography. In common would be the following information:

• Game Name
• System name
• Magazine Name
• Issue Number
• Publication date/time
• Page Number
• Screenshot count
• Notes

However, you could split off reviews, to add the following information:

• Author / pseudonym
• scores

The weakness, of course, is that no two magazines had the same system for setting up scores, and some magazines changed how they did it over time (for instance, Nintendo Power and Gamepro. Having an author/pseudonym would be nice in order to identify who did reviews. It is known that some pseudonyms covered different people (such as Sushi-X in EGM), though.

I sort of have an implementation of review scores on SNES Central, but I wonder how much it is worth splitting it off in a bibliography. It would probably be trivial to spend a few hours to compile scores from the magazines for a separate database (EGM actually did this for their year-end buyers' guides).

You could also split off the cheats, so that it actually contains the cheat in question. That could be covered by the earlier.

I think when it comes to a bibliography, it would be good to ask around the video game research community to see what they think, so that it is in a format that is most useful to them (since they are likely to be the primary users). I really think that making the database game/system oriented rather than publication oriented would be the wisest approach. When I do research, it is usually focused on that.

[Areala]

Well what would get lost by starting over again? If you need me to send you a 32GB flash drive or SD card and you want to send them back to me for importing under your name, just let me know. I was just going to start by pulling the single page adverts out of all our magazines and making sure the the file name is the name of the game, so that when they get uploaded, they are searchable.

I may take you up on that...

*huggles*

Areala

[Sean697]

I'm hopefully within the next week going to audit all the EGM's the site has. And note descrepencies or stuff that needs to be fixed. Was going to make a list. So would it be better if I correct some of the easy ones, like a page in the wrong order and upload it myself as a revision, or just post the list and an you or eday fix it and remove the old file and remove from the list?

[E-Day]

I would say make a list first. If it ends up being a lot of files, we can divvy them up or something.

[aloram]

Phillyman, I read your whole post. Those are some excellent ideas. Becoming a lifetime patron was a bargain and a half. Thanks a lot.

[Anime-niac]

If you could write me up a small tutorial on how you would go about pushing them out to Newsgroups. It would be very helpful, programs you use, methods, etc. I believe when I last pushed everything to Newsgroups, I broke down each individual magazine into 10 or 15MB parts, and then made 10 or 15% parity files for each set. I am pretty sure I did a few things wrong, but this time I would like to get them up there with no issues.

Will do, though it'll be after the holidays before I have it ready.

[Leathco]

I may begin mirroring some stuff on my website too down the road. I gotta start small, as I am limited to a DSL connection to upload to my server, but I could FTP a few issues at a time.

[kossan123]

I may begin mirroring some stuff on my website too down the road. I gotta start small, as I am limited to a DSL connection to upload to my server, but I could FTP a few issues at a time.

Once I have it up on my servers I could transfer it from there to yours, if you'd like.

[Leathco]

Once I have it up on my servers I could transfer it from there to yours, if you'd like.

Yea, I'd be fine with that. I could set up an FTP account for you whenever you are ready.

[MajorLag]

You can count me in for seeding a torrent. I can set it up on my NAS, let it run indefinitely. PM when you want to get that set up.

[aloram]

What's the general plan for acquiring the missing issues of, say, Game Players, EGM, and VG&CE? Is there an eBay budget? Or is it more of a "hope a Retromags user offers to scan what they have" kind of thing?

[Phillyman]

What's the general plan for acquiring the missing issues of, say, Game Players, GamePro, and EGM (and everything else)? Is there an eBay budget? Or is it more of a "hope a Retromags user offers to scan what they have" kind of thing?

On the forum index (community.retromags.com) I have included a secondary donation button for eBay Slush Fund.

This is for users who want to donate towards purchasing rare and missing magazines. Retromags also accepts magazine donations from users who wish to go that route. However the vast majority of users that contact me with magazines to donate, wish for Retromags to pay for the shipping of those magazines. So again, that is where the slush fund would come in. I would love to buy these 3 magazines, to complete our digital preservation of Odyssey Adventure magazine, but I can't justify the $150 price tag to do so. Sure these are staple bound issues, so I don't technically need to disassemble them. However I know that I would maybe get $25 per issue on resell (if I am lucky).

Odyssey Adventure Issue 2

http://www.ebay.com/itm/Odyssey-2-Adventure-Fan-Club-Magazine-2-Video-Game-System-Console-/301244893032?hash=item4623984768:g:y4gAAOSw9NxTxfCc

Odyssey Adventure Issue 4

http://www.ebay.com/itm/Odyssey-2-Adventure-Fan-Club-Magazine-4-Video-Game-System-Console-/291203209894?hash=item43cd105aa6:g:fmAAAOSwd4tT1wl~

Odyssey Adventure Issue 6

http://www.ebay.com/itm/Odyssey-2-Adventure-Fan-Club-Magazine-6-Video-Game-System-Console-/291203213469?hash=item43cd10689d:g:2QgAAOSw7NNT1wpe

[PhoenixVanguard]

Hi.

Just wanted to say I'm totally into seeding these suckers once I download them. Might be best to split them into torrents for each publication, like "EGM 1989-1999", although I have noticed a few issues missing (suppose not much can be done about that).