Phillyman's Work In Progress...

[Phillyman]

And another dedupe......this one is for Ultra Game Players Issue 93, but got imported a second time as UGP Issue 74? They are both the same file....

The 74 labeled version is getting nixed.

 

[Phillyman]

And again, another duplicate this time for UGP 104, which got pulled in as 85 also. Again blowing away the 85 issue, the underlying files are identical.

 

 

[Phillyman]

Again, nixing the dupe with the wrong name

 

[Phillyman]

and I believe this is the last one, 109 stays, 90 gets nixed!

 

[Phillyman]

Ooops one more, this one is Video Gaming Illustrated Issue 3, the copy VGBounceHouse uploaded is much better quality. Deleting the one I uploaded.

 

 

[Phillyman]

I cleaned up the Magazine Database and Download Manager over the last few days. Using the matching MD5 records between them I was able to do the following....

 

In the Download Manager now anything that is available via torrent now states what Torrent package it is in

 

Also on the Download Manager page I have dropped in the corresponding Database Record links to 99% of the files in that section. I just need to manually do the PDF versions and Strategy Guides at this point. Now you can get to the Magazine Database from the Download Manager, and vice versa!

 

On the Magazine Database side, I fixed up the Official Download links, many of them were still pointing to the community.retromags.com address.....and while it will auto resolve, I figured I might as well clean them up in mass. You will also notice that I dropped in torrent links for everything that has been released via torrent.

The torrent links will download the torrent file that contains the magazine you are looking for. So if you go to Nintendo Power Issue 2 in the Magazine Database, the torrent link on that page will download the 2017 Base Torrent Pack 9 which contains that issue.

My next item is to get rid of the "Retromags User Download" links above (done). With me rolling up the entire Retromags collection from our Download Manager and dumping it onto torrents and Usenet, the User Provided links to FileFactory, 4Shared and other third party file hosting services will surely result in out of date versions and broken links. We will be deleting the existing data for those 3 fields (after a backup) and then changing them to the following.....

I will be reaching out to the admins of these sites and a few others to gather some spreadsheets of what they have already preserved. We can then mass import the links to their download pages and mark those records as "Preserved" so that users can get an overall view of what has been preserved digitally across all of these sites.

 

[kitsunebi]

5 hours ago, Phillyman said:

My next item is to get rid of the "Retromags User Download" links above (done). With me rolling up the entire Retromags collection from our Download Manager and dumping it onto torrents and Usenet, the User Provided links to FileFactory, 4Shared and other third party file hosting services will surely result in out of date versions and broken links. We will be deleting the existing data for those 3 fields (after a backup) and then changing them to the following.....

I will be reaching out to the admins of these sites and a few others to gather some spreadsheets of what they have already preserved. We can then mass import the links to their download pages and mark those records as "Preserved" so that users can get an overall view of what has been preserved digitally across all of these sites.

 

Old Game Mags doesn't exist anymore.  You're going to need to change that button to "Kiwi's World"

[Phillyman]

37 minutes ago, kitsunebi77 said:

Old Game Mags doesn't exist anymore.  You're going to need to change that button to "Kiwi's World"

Good catch! I fixed it

[KiwiArcader]

The only problem I see with this is most everyone out there like RetroCDN and Archive.org is simply pinching everyone else's scans so I'm not sure what value there is pointing to other sites? If it's to allow scanning members to determine which mags don't need to be scanned because someone else has already done it that is fraught with peril in several ways...

• Existing scan may be missing pages/ centre posters etc. 
• Quality of the other sites scans might be inconsistent with your requirements
• Other sites may have just ripped your scans off anyway but also may have edited them ( to remove Retromags advertising page and/or dropped quality )

I should think it would be far better to concentrate on the scans from members of this site? The only sites you can verify that most files originated from are OoPA, mine and Retromags. Pointing to Archive.org for most gaming magazine files hosted there is like advertising/popularizing a thief in my opinion.

[kitsunebi]

I agree that asking archive.org which scans originated there is a waste of time, since they don't regulate such things.  It's just a depository that anyone can upload anything to.  RetroCDN was described to me by one of their main "contributors" as a Wiki that anyone can add to.  Which isn't exactly true since most Wikis still require the content you input to be original and not just copy/pasted from elsewhere, but again, almost none of their scans originated there.

That said, I'm not above downloading from archive.org.  I know that whatever it is probably originated somewhere else, but sometimes finding that original website is difficult, so it's more convenient to just get it wherever I find it first, and I know that the scans are usually unchanged from the originals (when they pilfer our stuff, they usually don't even remove the Retromags credit page).  I try to avoid RetroCDN as anything other than a last resort since most of their files have been shrunken and compressed, so anything available there is probably available in better quality from whatever website they took it from originally.

I feel that most English video game/computer game magazine scans originated from either RM, OOPA, or KW.  Most of the stuff available elsewhere is either non-English language, or focused more on hardware than games (mags about older computer systems, for example), and most of the time is not of the same quality.  It's pretty rare for an American or British video game mag scan to have originated from anyplace other than these three sites.  So even if we just point to OOPA and KW scans, we'll have the bulk of the mags pertaining to RM's primary focus covered, I feel.

[Phillyman]

Fair enough, at the moment I will just hide the User Download links from being populated. I only included Archive because that was where Marktrade was placing his items and he was linking to Archive for them. I will still move forward on linking to active publishers that we can still subscribe to with either physical or digital versions. What is still active, maybe 15 different magazines between the USA and UK at the moment....if that?

 

[KiwiArcader]

marktrade acquired a lot of mags from you Phillyman? If so, those items should be available here and not just over at archive.org anyway.

Are you after the actual links to the magazine issues themselves or the individual pages the magazines are listed on over at KW? 

[Phillyman]

7 hours ago, KiwiArcader said:

marktrade acquired a lot of mags from you Phillyman? If so, those items should be available here and not just over at archive.org anyway.

Are you after the actual links to the magazine issues themselves or the individual pages the magazines are listed on over at KW? 

This is everything I sent Marktrade, not sure what he scanned so far. A few months after sending these off Mark stopped uploading items to Retromags and started to just dump his scans onto Archive.

 

[kitsunebi]

Well if you consider that a burn, I wonder how his patrons on Patreon feel?  I guess it's technically their fault for not canceling their monthly payments, but marktrade's collected over $1000 from them in the 6 months since he stopped scanning and dropped out of sight on the internet.  A profitable business, retirement

[Phillyman]

Just now, kitsunebi77 said:

Well if you consider that a burn, I wonder how his patrons on Patreon feel?  I guess it's technically their fault for not canceling their monthly payments, but marktrade's collected over $1000 from them in the 6 months since he stopped scanning and dropped out of sight on the internet.  A profitable business, retirement

Actually I may still have the Dreamcast magazines on hand, its been so long, but I did send him two boxes of magazines with the remainder of the images above. I don't think Marktrade ran off with the magazines, and I still hold out hope for a flash drive full of scanned pages to show up in my mailbox one afternoon

 

[kitsunebi]

I don't know, man.  It seems a bit suspect to be collecting $1000 in donations and not even have the courtesy to post a status update in 6 months.  I wouldn't hold your breath.

Btw, AFAIK, the only stuff in those pics that was scanned were the SuperPlays, CDi mags, and NextGens.

[Phillyman]

3 minutes ago, kitsunebi77 said:

I don't know, man.  It seems a bit suspect to be collecting $1000 in donations and not even have the courtesy to post a status update in 6 months.  I wouldn't hold your breath.

Btw, AFAIK, the only stuff in those pics that was scanned were the SuperPlays, CDi mags, and NextGens.

He did comment to Jason of Archive back in June, so its only been 16 weeks since we last heard from him. Who knows, he could pop up tomorrow and drop out 100 magazines as a surprise.

 

[kitsunebi]

6 minutes ago, Phillyman said:

He did comment to Jason of Archive back in June, so its only been 16 weeks since we last heard from him. Who knows, he could pop up tomorrow and drop out 100 magazines as a surprise.

 

If he had just gone away like Sean697 (and any number of scanners from before I was a member here), it wouldn't bother me.  It happens.  This isn't a lifelong gig for most people.  I guess I'm just hung up on the money thing.  It really feels like he's taking advantage of people.  If people were giving me money to do something and I decided to take an extended break, I'd have the decency to let them know.  I'd also stop taking their money, because I'd feel like an asshole otherwise.  But that's just me, I guess.

[Phillyman]

Well look what showed up today in the mail! My wife is not going to be pleased

 

 

Just went thru them, looks like

EGM 91-98, 100-236, and 238-261

 

[Phillyman]

I got thinking the other night on how I could use the Magazine Database to help structure the project files. It always slows me down a bit to create each folder one at a time, so I extracted the information from the database and wrote a script to mirror the file structure locally on my computer/NAS. I can technically take it one step further and have this script pull down a small thumbnail of each magazine from our Gallery for reference. But this helps when I am scanning tons of magazine covers, I will have a place to drop each of the files. Once I clean up this tiny script I will release it for other Team Members to use as well. I just need to have it clean up characters that can't exist in Windows folder names like slashes and periods.

 

 

[kitsunebi]

I'm not sure if I know what you're talking about regarding "project files."  Are you referring to your raw scans?  If so, by releasing the script to members, are we encouraging more people to contribute unedited scans?  'Cause I've got several of those sitting around already which can be added to the dozen or so that have already been sitting there since last year.

Man, imagine how our output would increase if we all just agreed that editing is just too much of a pain in the ass to bother with and uploaded raw scans as fast as possible.

Scan 'em all and let god sort 'em out.  And edit them if he's not too busy.  'Cause I sure as hell am.

 

[Phillyman]

On 10/31/2018 at 4:59 AM, kitsunebi77 said:

I'm not sure if I know what you're talking about regarding "project files."  Are you referring to your raw scans?  If so, by releasing the script to members, are we encouraging more people to contribute unedited scans?  'Cause I've got several of those sitting around already which can be added to the dozen or so that have already been sitting there since last year.

Man, imagine how our output would increase if we all just agreed that editing is just too much of a pain in the ass to bother with and uploaded raw scans as fast as possible.

Scan 'em all and let god sort 'em out.  And edit them if he's not too busy.  'Cause I sure as hell am.

 

So yes Project Files would refer to the raw unedited scans fresh from a scanner. I want things to mesh up between the Magazine Database and the QNAP NAS. Right now you can see that we have different naming schemes, just want to bring things into one way.

I am not saying that we are letting go of editing magazines. What I am saying is that my 10 year dream of having 20 users who can scan, edit and release 1 magazine per month will probably never happen. That would equate out to 240 new releases per year or a new release every 36 hours. Rather than try to find energetic users who own old magazines, willing to possibly destroy them, willing to scan them, have a decent photo editing software, willing to edit each image, willing to compile them into a release, willing to upload them here. Well I am saying....

• I am energetic
• I have old magazines
• I am willing to destroy them
• I am willing to scan them
• When I have time or need a break from scanning, I am willing to edit them.
• The last parts of compiling and uploading are very light tasks, so if someone is willing to edit, they are probably fine creating a zip, renaming it, and uploading it.

I am tight on space at the moment, so if I can mass create 5000+ USA folders for my raw scans, and chop apart magazines and just come home each night and feed a few thru the scanner and drop them in place. Well I am hoping that we can find a few users that own a Photoshop subscription and can assist with cleaning up the scans. Which it should be much easier to find users with photo editing software/skills, than someone with stacks of old gaming magazines and a willingness to scan them.

Plus pipes break, fires start all the time, I would rather have the magazines scanned into a digital format, rather than waiting for a disaster to strike.

 

[kitsunebi]

4 hours ago, Phillyman said:

. Which it should be much easier to find users with photo editing software/skills, than someone with stacks of old gaming magazines and a willingness to scan them.

I would posit that this has already been proven untrue.  There are already more people scanning and releasing unedited or lightly edited scans at archive.org than we have editing files for Retromags.  It only takes one time to realize how much faster and easier it it to scan a magazine than it is to edit a magazine.  Additionally, there are people willing to scan their magazines so long as they don't have to damage them, and those scans usually end up on archive.org as well.  And the truth of the matter is that most people downloading the mags are content so long as the scan is decent and readable, perfectly edited or not.

I don't disagree that you would be better off scanning all of your mags to get them out of the house, regardless of if they ever get edited (I suspect most of them never will.)  And I may one day do the same with all of the hundreds of Japanese mags I have lying around.  I've already decided that once I've finished releasing the mags I have which were donated to me, I'm done with editing scans (unfortunately, I still have quite a few donations to work through).   But if there's no problem uploading terrabytes of raw scans, I might scan the rest someday rather than just toss them in the garbage.  Although to be honest, I may go the archive.org route as well.  Not much point in uploading them here if no one is editing them - better to release them somewhere with more exposure so people can at least read them.

If only it weren't ethically dubious to hire cheap foreign labor to edit scans...

[E-Day]

8 hours ago, Phillyman said:

I am tight on space at the moment, so if I can mass create 5000+ USA folders for my raw scans, and chop apart magazines and just come home each night and feed a few thru the scanner and drop them in place. Well I am hoping that we can find a few users that own a Photoshop subscription and can assist with cleaning up the scans. Which it should be much easier to find users with photo editing software/skills, than someone with stacks of old gaming magazines and a willingness to scan them.

 

Just be sure that as part of the scanning process you review all the scans and make sure none of the pages are missing, extremely crooked or went in funny causing digital smearing on the page before you toss the scanned pages into the recycling. That way in three years if someone is editing a scan they don't have to stop because one of the pages needs to be rescanned and no one has the issue to rescan that one page.

[kitsunebi]

6 hours ago, E-Day said:

Just be sure that as part of the scanning process you review all the scans and make sure none of the pages are missing, extremely crooked or went in funny causing digital smearing on the page before you toss the scanned pages into the recycling. That way in three years if someone is editing a scan they don't have to stop because one of the pages needs to be rescanned and no one has the issue to rescan that one page.

Unless I'm remembering incorrectly, Phillyman doesn't throw anything away, he just puts loose pages into storage.  Still, better to get it right the first time.  And on that note, I want to point out that those of us using ADF scanners should be constantly vigilant against dust particles causing scan lines.  Dropping 50 pages into the ADF and letting it go without checking them carefully for scan lines is a recipe for disaster.  I've NEVER managed to scan a complete magazine without having to stop several times, clean the glass, and rescan pages that were affected thus.  I guess old magazines (or my room) are just too dusty to begin with.