IremDb - Internet Retro Magazine DB is now open

[retroprime]

https://iremdb.com

I finally managed to get something out ... This has become a much harder effort that I could have ever imagined.

Having to both index magazines (currently at 17k pages) and build that horrible website, has become a very hard job, Hope it somehow people enjoy it.

I got the basics out, list platforms, publications, magazine, search feature, and a basic viewer.

Some things you can find there:

All pages that contains Super Mario Bros 3 : https://iremdb.com/platforms/nintendo-entertainment-system-nes/super-mario-bros-3

 

 

All famous Sega ads "Genesis does what nintendon't": https://iremdb.com/search?mode=ads&tags=123

 

All John Madden Football ads: https://iremdb.com/search?mode=content&games=38633&contentTypes=Ads

 

All content published by "Andromeda" editor from Gamepro: https://iremdb.com/search?mode=editors&editors=43

 

The entire "Legend of Zelda" comic strips from nintendo power published over 12 issues https://iremdb.com/search?mode=content&tags=228&contentTypes=Comic

 

I need now to integrate a user management into the site, so people can start contributing if they wish.

 

Some progress I made in automating the whole thing:

 

I had managed to do some neat tricks such as automatic ads detection, given that 50% of those magazines are ads, I used a perceptual hash algorithm to finger print each ad page, and for each new issue, it simply looks in the database and usually get 80-90% of all ads indexed (it does miss about 10%, and there's always some new ads on each issue).

 

I have also played with using custom object detection, and it works fine, for instance I automated all detection of gamepro reviews by looking for those silly faces, problem is that data labeling to train a model is as expensive in time as actually, you know labeling the page

 

I had set a goal to release this to public as soon as I reached March 1993 (exact 30 years ago) in magazines indexed.

 

I'll keep pushing new issues, and adding more features, need to get user integration or this is just a waste of my time.

 

Once again, if you want to participate ping me again, I had to shut the other site down.

 

I may also have to one day keep the viewer behind a login screen, the site has never been shown to anyone else but somehow hundreds of bots from some countries are just crawling it nonstop, if I want to keep CDN costs at bay, I may need to just let the thumb images to be public, other pages, and future features such as like, collection, custom searches, to be only for registered users.

 

Constructive (does not need to be good, just constructive) Feedback is welcome  

[hardcorehubz]

This has the potential to be incredibly useful in so many ways and huge kudos to you for working on this. Absolutely if you can add user integration. This will allow the project to really take off imo, it's not a waste of time at all.

[E-Day]

This is absolutely insane. We've dreamed of something like this for over a decade for this site but never managed to have the time or know-how on how to do it. This is a crazy amount of work and am glad to see it actually exist somewhere! Hopefully some people here can make time to try and help (myself included); hopefully some members from the Gaming Alexandria community is also willing to pitch in. They seem to be an active bunch!

If you didn't send this to Frank at the Video Game History Foundation (aka @TheRedEye on here). Maybe do so. They have a pretty solid support base as well and some from there may be able to help out also.

[retroprime]

Thanks @E-DayI have not reached out to him yet, more than happy to chat. I still stand for the first message of making the data public, I just need to get user login under control, public means registered user, as I found out that even with no users, I'm still getting hit pretty bad by bots hitting the endpoints.

I have also thought of maybe one day make it work with this site, integrate the backend of iremdb into retromags, not sure if this would be possible through this platform but it is a thought nevertheless.

 

[E-Day]

1 hour ago, retroprime said:

I have also thought of maybe one day make it work with this site, integrate the backend of iremdb into retromags, not sure if this would be possible through this platform but it is a thought nevertheless.

 

That's a Phillyman question for sure. Even if it can't be integrated, just the fact that this will exist for the public to use is amazing.

[miketheratguy]

Just browsing the review scores for various games is SUPER interesting. Also very easy and very handy, just go to the platform, pick the game, and there's the average score across multiple magazines PLUS the individualized scores with just a couple clicks. Awesome, thank you for this!

[sCZther]

Yeah, I must join in with the congratulations. If this continues to be enhanced and used by the community, it is an invaluable source. Godspeed and I will be following the progress closely!

[AvantMessiah]

Holy shit, this is amazing.

[RetroDefense]

Wow - you need to talk to Phillyman about a website API team-up or something.  This is fantastic!