Greatest source of gaming info ever!

[kitsunebi]

Hey guys, I was surfing the internet and I found this INCREDIBLE website called Wikipedia that has all kinds of info on not just games, but gaming mags as well.  I was thinking we should copy everything we can from their site and put it here (if anyone knows a good way to just copy all of their source code and route it through our site, that would be awesome.)  And it turns out that the internet has lots of other websites, too.  Maybe we could mirror those as well?  I realize that people could just visit those other sites, but I think it would be better if our site had a copy of everything from all the other sites all in one place so they never have a need to navigate away from our site for any reason whatsoever.   You can never have too many mirrors of this stuff - who knows when it might disappear - better to squirrel it away just in case.  

[MigJmz]

??? what is this referencing to? I'm lost.

[AJIMS64]

2 hours ago, MigJmz said:

??? what is this referencing to? I'm lost.

Me too... :/

[kitsunebi]

LOL.  If my comment seems a bit sketchy to you, don't have a cow.  It's totally understandable that you'd be left in the black about my comments about mirrors and squirrels, since it wasn't aimed at you guys.  Most people are probably in your shoes - if you haven't archived 100 or more scans either here or some other retro site, I doubt it would make much sense.

I know @KiwiArcader knows what I'm talkin' bout.  Everyone else just go back to reading about video games and pay us no mind.

[MigJmz]

if were talking about retrocdn and the like then I understand.. what can you do..

[ml0045]

You joke but you can actually mirror wikipedia if you want. It's about 17 GB for English.

https://meta.wikimedia.org/wiki/Data_dump_torrents#English_Wikipedia

[kitsunebi]

I think we should do it.  There's no reason our members should have to visit both our site AND another site to get information if we have the ability to make it all available here.  We'd be performing an invaluable service.

[MigJmz]

Hmm i thought he was complaining about people ripping RM content.. guess i was wrong.

[kitsunebi]

We should make a list of all the websites we like to visit, and then mirror them here so that there's never a reason for any of our members to ever go anywhere else.  We'll probably have to stop producing any new content at Retromags in order to have time for such an endeavor, but I think backing up the Internet should take precedence over creating new scans, don't you?

[MigJmz]

[kitsunebi]

Not everyone is cut out for doing the Important Stuff.

Once we're finished backing up the Internet by mirroring all of it at Retromags, we should create a new site to mirror the new Complete-Internet-Retromags.  Just in case.

Migjmz is out, but I know at least one member who is up for the task and has the vision to see how vitally important this is.  Once we secure government funding we can hire people to help, so lack of volunteers shouldn't be a problem.

[KiwiArcader]

@Black Squirrel ....where are you? ..... I'm pretty sure this is right up your alley. You can do it .... we know you can. Your reputation as a backer upper is second to none.

Hey ... here's a thought. He could backup Retromags to RetroCDN (he likely already has I believe) then backup RetroCDN to Retromags for one big incestuous pool of Retro then if he added Playboy we could call the site RetroCans ... Retro for gaming mags and Cans for well, boobies. 

Man oh man!!! .... I'm salivating at the prospect!! 

[kitsunebi]

You're still thinking too small.  First, we back up the entire Internet here.  Then we wait a day to allow for our backup to be backed up at the Retro sites, and then we backup their backup of our backup and call it RetroCaboodle.  Then wait a day for RetroCaboodle to get backed up at the Retro sites, then backup their backup of our backup of their backup of our backup as RetroEnchilada.  Wait a day for the Retro sites to work their magic.  Then backup their backup of our backup of their backup of our backup of their backup of our backup and call it RetroShebang.

It's gonna be glorious.

[KiwiArcader]

The ONLY problem with that will be when I search for Cans. I'll get a bazillion images of the same pair of Cans. Hopefully they will a nice set or else ... 

[kitsunebi]

We can keep the cans separate on a team members only portion of the site for your perusal.  As you know, that section is only visible to people who have actually contributed something to this site/hobby, so there will be no danger of those being backed up by anyone but us.

[KiwiArcader]

I dunno. Jason the Cow is pretty good. Probably even better than Black Squirrel. He might be able to get past our team member defenses with the might of the archive.org machine!!

[kitsunebi]

We could make the team member's section open to anyone willing to contribute a few bucks to keeping the site running.  That seems to work pretty well for you at keeping away any and all leechers.  The freeloading kind, at any rate.

[KiwiArcader]

If I had a super set of Can pics on there I'd likely charge more that's for sure.

[kitsunebi]

No one works harder than me at making sure Retromags isn't completely lacking in cans.

[KiwiArcader]

I know. You are my Can idol. I only hope you never decide to Can idle ...... although if you did maybe Black Squirrel would step in and take up the "Can"tle

Edited May 5, 2020 by KiwiArcader

[JHD]

I realise that kitsunebi is trying to make a point here, but there is something to be said for maintaining personal copies of interesting content.

 

For example, I am very interested in Canadian history. At least two longstanding, publisher-affiliated sites hosting Canadian academic journals have either gone offline or decided to restrict access in the past year or so. Another scholarly journal made the decision to go "open access" for just a few months, at which point the content would then be restricted again. If I had not grabbed personal copies of this content, I would not now have access to it! In a similar fashion, I have downloaded copies of several hundred YouTube videos that are no longer available on the site.
 

Now, I have no plans on sharing my collection in any fashion -- indeed it lives in an external hard drive that is disconnected from the PC when not required -- but I can still see the argument for making backup copies of online content. 

[JonnyCGood]

If the complaints are about other sites mirroring content, I don't get the problem. I want this content as freely available as possible, and it doesn't belong to any one person, it belongs to all of us. I want to build easily searchable databases that people can have access to so they can search about whatever they want to. I in fact want these in as many hands as possible simply to prevent any of this material ever being lost.

My big complaint is that they are absolutely enamored with the pdf format, presumably because that's what their site software uses. They will take books I know were in cbz/cbr format and convert them to pdfs! Ugh! When you host something leave the files alone and change nothing!

 

[E-Day]

On 12/13/2020 at 3:09 PM, JonnyCGood said:

If the complaints are about other sites mirroring content, I don't get the problem. I want this content as freely available as possible, and it doesn't belong to any one person, it belongs to all of us. I want to build easily searchable databases that people can have access to so they can search about whatever they want to. I in fact want these in as many hands as possible simply to prevent any of this material ever being lost.

My big complaint is that they are absolutely enamored with the pdf format, presumably because that's what their site software uses. They will take books I know were in cbz/cbr format and convert them to pdfs! Ugh! When you host something leave the files alone and change nothing!

 

I would love to move to PDF, or have PDF in addition to CBZ files so that all the scans have searchable text. That would make all our stuff that much more useful.

I've tried JPG to PDF conversion programs to get OCR text, as well as scanning directly to PDF with OCR, and while they work, you don't get a clean straight scan like you get with JPGs that we edit. Both the programs and the scanner will skew each page  so you have this uneven edge of white along both sides.

I would LOVE a program that could take out edited JPG files and add OCR to them without adjusting or rotating the page at all so that we could have a PDF scan with searchable text that looks exactly like our finalized CBZ files. 

If anyone knows anyone who can create such a program, let me know. If someone could create something to do that, it would be worth paying for in my eyes. I'd like people researching retro gaming to come to us and search for stuff rather than the Internet Archive.

[JonnyCGood]

3 hours ago, E-Day said:

I would love to move to PDF, or have PDF in addition to CBZ files so that all the scans have searchable text. That would make all our stuff that much more useful.

In principle I think the cbz format could be expanded upon to allow for OCR without anything major changed. Something like an additional text file that a CBZ reader knows what to do with. That way you can have a nice clean cbz with searchable text, which i think is the only advantage pdf has over cbz. Archive.org added OCR a while back and allows for text searching and that's what I've been using for text searching of cbz/cbr files. Originally it was for latin characters only but they've expanded it to Japanese characters too. For example, it's neat that you can search for "Faxanadu" in both English and Japanese. Whatever they do could probably be done on standalone cbz readers as well.

PDFs are a bit of a chore to work with since they only can be edited by a few programs like photoshop. And it's frequent people will lazily extract images from pdfs via just exporting .jpgs which either means a quality decrease or file size bloat.

[CaliA]

On 5/5/2020 at 3:20 AM, ml0045 said:

You joke but you can actually mirror Wikipedia if you want. It's about 17 GB for English.

https://meta.wikimedia.org/wiki/employee monitoring/Data_dump_torrents#English_Wikipedia

Okay, this idea initially sounded insane. But is someone REALLY doing this? How is it going?