Version 0.3.0 - The biggest update

I’ve worked hard this weekend to further improve the bot in terms of success rates, how it works and even introduced a new bot for tracking the Playstation 5!

Changelog

Before going into detail let’s quickly take a glance at the overview of new or improved things:

• New bot: @iloveps_5 which tracks PlayStation 5 availability.

  • With that the ilovertx3000 bot no longer tracks PlayStation 5 availability (was already turned off on thursday)

• Added shops:

  • newegg.com

• Disabled shops:

  • nvidia shops (see below)
• Introduced an entire landing page with better FAQ and more information about the bot.
• Refactored the bot to be configuration-centered, which makes it lot easier to start multiple instances to track different products.
• Added statistics. They’re not public (yet) but will give me a better idea of what’s happening and where things need to be improved.
• Improved crawling success rates.
• Added parallelization.

New bot: @iloveps_5

With the bot using dedicated configuration files it was fairly easy to add new products - without interfering with the @ilovertx3000 bot. The @iloveps_5 will track Sony Direct, Amazon.de and Amazon.com and announce changes on its own Twitter acccount.

Landing page

Since this project has grown a lot within the last week I needed a better page to give an overview about what this is all about, why and how this project tries to help in an ethical and friendly way, how to support the project, etc. Hope you like it.

Big shoutout to @amritnagi for creating the Tailwindcss Landing Page which was used as the template for this landing page.

Improved crawling success rates

During the last days multiple shops changed their code so that the bot often times didn’t know how to find stock updates. All of these issues should be resolved by now. As a reminder a quick list of supported shops and products:

• @ilovertx3000

  • alternate.de (RTX 3080 and RTX 3090)
  • caseking.de (RTX 3080 and RTX 3090)
  • evga.com (RTX 3080 and RTX 3090)
  • nvidia shop US (RTX 3080 and RTX 3090)
  • nvidia shop EU (RTX 3080 and RTX 3090)
  • amazon.de (RTX 3080 and RTX 3090)
  • amazon.com (RTX 3080 and RTX 3090)
  • newegg.com (RTX 3080 and RTX 3090)

• @iloveps_5

  • Sony Direct Shop (PS 5 and PS 5 Digital)
  • amazon.de (PS 5 and PS 5 Digital)
  • amazon.com (PS 5 and PS 5 Digital)

Nvidia shops are currently disabled - see below.

Support

Since this project has begun to cost some real money (for servers, etc.) and required a decent amount of work so far I’ve setup a dedicated support section. Feel free to support the project by buying me a Spritzer - thanks and Prost!

Upcoming shops

After an insane amount of 1163 votes the next shops I’m going to implement (if possible) are:

• nvidia Denmark (I’m still confused)
• best buy

Newegg is already up and running. nVidia requires some more work (see below) and BestBuy.com is also pretty tricky but should be done next week.

nVidia Shops are disabled for now

The nVidia shop is a weird thing. There are essentially two different possibilities on how to get stock availability:

• try to “read” it directly from the page
• try to get it from an API

But even if nVidia uses an API which is preferably used for getting this information it can’t be used. Either the API returns an error or the information provided by the API is bullshit:

{
    "inventoryStatus": {
      "uri": "https://dispatch-nvidia.digitalriver.com/v1/shoppers/me/products/5438793500/inventory-status",
      "availableQuantityIsEstimated": "false",
      "productIsInStock": "true",
      "productIsAllowsBackorders": "false",
      "productIsTracked": "true",
      "requestedQuantityAvailable": "true",
      "status": "PRODUCT_INVENTORY_IN_STOCK",
      "statusIsEstimated": "false"
    }
}

This is returned by the API for the RTX 3090 on the nVidia denmark shop. Notice how productIsInStock is true and status equals to PRODUCT_INVENTORY_IN_STOCK - but if you take a look at the page itself it clearly states the the card is “Udsolgt”, which - according to my translator - means “out of stock”.

“Reading” the page is also not possible since nVidia shows an “ADD TO CARD” button (which would indicate the card is available) until getting an error from their own API which changes the button to “Out of Stock”.

As long as I haven’t found out what the hell is happening here I have to disable nVidia shop crawling in order to prevent false positives.

Hiccups and missed stock changes

The bot had a severe case of getting nuts on thursday some time after midnight (it’s always in the night, because why not). Fun reason behind that:

The german Amazon page lists two GPUs which have an identical name, “PNY GeForce RTX™ 3080 10GB XLR8 Gaming Epic-X RGB™ Triple Fan Grafikkarte”. One of them was out of stock while the other one was available somewhen in October.

The issue here is that the bot differentiates based on the name of a product - and with two products having the same name but different stock status every minute the bot tweeted a change for that product. This was fixed shortly after occurring for the first time - but I just wanted to shortly talk about what happened and why the bot caused some skipped heartbeats in the middle of the night.

Another thing that has been mentioned: missed stock changes. Some stock changes might have been missed due to the reasons mentioned above - which should be fixed by now. But always keep in mind that this bot is a one-man show, developed by a “crawling-bot virgin” who has a regular job and life. I’m not saying it’s bad - I just want to remind you that there is and always will be room for improvements and some stock changes are going to be missed. I’m trying my best - and I’m sad about every missed update since I’d like a new GPU too.

Please use the Tweet about this post for comments!