scraper-en

Scrape your Roms with ARRM

Before embarking on your first scrape with ARRM, I invite you to read the previous paragraphs (Application Setup, and Interface Overview) to fully understand the ARRM mechanism.

The different ARRM options must be set before proceeding.

Video Tutorial


View video


BatoceraNation made an english tutorial on how to use ARRM Scraper : https://www.youtube.com/watch?v=fzMPB--STsw&ab_channel=BatoceraNation


View BatoceraNation video tutorial

Since version 1.9.0.3 beta 11 you can only scrape the missing media (artworks) by checking Without Artwork only in the Scrape Options tab. Only the missing media (artworks), checked in the Images options tab will be recovered.
In the scrape engines, only these missing media (artworks) will be presented and selectable.
See : Only scrape missing Artworks

Scrape Multi-Sytems

To activate this mode, simply select more than 1 system from the list of systems on your Recalbox / Batocera

When multiple systems are selected, the 'Show' button changes to a 'Multi Systems' button. Since version 1.9.1.3 Beta 9, this mode works with all scrape engines available in ARRM.

Automatic assignment of a template

If you want ARRM to apply a different template depending on the systems, do not forget to tick the box 'Template auto association'. ARRM relies on the file %appdata%\Nexouille Soft\arrm\database\template_association.txt (which you can modify via the template selection interface) to determine which template to apply to which system.

To choose your template, see this section : Choose template

Start the scrape

When you press the Multi Systems button, a pop-up will appear, asking you to choose the scrape engine you want:

ARRM will treat each of the selected systems and will:

  • Load the existing gamelist.xml, or create a new one with the found roms
  • Will search the selected database for game information
  • Apply the different options chosen in the settings
  • Will save the gamelist.xml once the searches are done for a system
  • Will switch to the next system

At any time you can stop the scrape by pressing the ' Stop ' button.

When all systems have been processed, a summary appears showing the number of games found per system and the time spent.

Apply a global filter to treat only certain Roms

You can only process certain Roms, for example those for which you do not have a description or no boxart. To apply this global filter, before clicking on the “ Screenscraper ” button, you must enter the filter / query in the “Global Filter” textbox.

The searched strings must be surrounded by single quote (') and not by double quotes (“) If a searched string contains a single quote ('), you must put two single quotes in the query

Examples:

description_rom is null or boxart_rom is null
favoris_rom = true and (image_rom is null and manual_rom <> '')

The fields available for the conditions are:

adult_rom
annee_rom
arcadesystemname_rom
boxart_rom
boxback_rom
cache_rom
cartridge_rom
core_rom
description_rom
developer_rom
emulator_rom
fanart_rom
favoris_rom
fichier_rom
gameid_rom
gametime_rom
genre_rom
genreid_rom
hash_rom
image_rom
kid_rom
lang_rom
lastplayed_rom
manual_rom
map_rom
marquee_rom
md5_rom
mix_rom
nomjeu_rom
numauto_rom
playcount_rom
players_rom
publisher_rom
rating_rom
region_rom
screenshot_rom
temporary_rom
thumbnail_rom
titleshot_rom
video_rom
wheel_rom



Scrape Mono-Sytem and scrape of games

If you want ARRM to apply a different template depending on the systems, do not forget to tick the box 'Template auto association'. ARRM relies on the file %appdata%\Nexouille Soft\arrm\database\template_association.txt (which you can modify via the template selection interface) to determine which template to apply to which system.

To choose your template, see this section : Choose template

ARRM allows you to scrape an entire system, or only certain games of a system.

For this you need to select only 1 system in the system screen:

Click on the Display button to display the found Roms. ARRM loads the existing gamelist.xml and verifies the existence of the Roms files. If the file does not exist, the ROM will not appear in the database. And conversely if a game is present on the disc but not present in the existing gamelist.xml, it will be added to the Database.

Once the Database is filled with the Roms found, you can

  • Apply filters on the Database to process only certain roms

If you only want to scrape the Roms that do not have an image or description, you just have to check, in the Filters tab: Image , Description , without , or then click on the Filter button.
Roms without image or description will be displayed in the database.
You will be able to select them and start a scrape on these Roms.
At the end of the scrape, click on No Filter to remove the filter and find all your roms, then save your gamelist via the button Create gamelist.xml

  • Select all or part of the elements of the Database
  • Execute actions on selected items

  • Start a scrape using one of the scrapers proposed. You can stop at any time a scrape by pressing the button 'Stop'

See on the wiki the specifics for

When you are done with the Database manipulations, remember to save your content in the gamelist.xml by clicking on the button 'Create gamelist.xml'

ARRM has a data recovery system when an application crash occurs. When loading a system, if ARRM detects that a crash has occurred, it proposes to restore the recovered information until the moment of the crash.

You can do a recovery with the cache system : How to use ARRM cache

You can also consult after each operation a log by clicking on the 'View Log' button

Scrapes not found are listed in log files located in %appdata%\Nexouille Soft\Arrm\temp, in the format:


_missing_arrm_%SYSTEM%_%SCRAPE_DATABASE%.txt

example: _missing_arrm_ZXSPECTRUM_screenscraper.txt



Recover data following a crash during scraping

ARRM has a data recovery system when an application crash occurs. When loading a system, if ARRM detects that a crash has occurred, it offers to restore the information recovered up to the moment of the crash.

See wiki: Recover gamelist.xml after ARRM crash

You can also retrieve data by using the cache: How to use the ARRM cache



Remove image file suffixes

See the dedicated section: Remove image file suffixes



Multi-threaded scraper with ARRM on Screenscraper

Since version 1920 Beta 11, it is possible to use multi-threading when scraping on Screenscraper.

This feature is reserved for ARRM donors

The number of usable threads depends on your status on Screenscraper.

To have additional threads on Screenscraper, you must participate in the database, or make a donation on Screenscraper. For more information, see the FAQ Screenscraper

Memory/cpu usage increases with the number of threads.

ARRM's multi-threading only makes sense if you're scraping multiple ROMs. A unitary scrape will not use multi-threading.

In order to be able to use the ARRM Multi-Thread on Screenscraper, you must first fill in the Screenscraper Threads Limit parameter available in the Scrape Option tab.

It indicates the maximum threads you want to use.
This number must be less than or equal to the maximum number of threads you have on Screenscraper.
You can check this information via the Check Screenscraper button at the top right of the ARRM page.

If your Screenscraper account is filled in (see: Log in to Screenscraper, ARRM will recover the maximum value that Screenscraper authorizes you. However, you can decrease it by changing the value.

If you want to find your maximum value, type a large number like 99, ARRM will automatically adjust to the maximum value that Screenscraper allows you.


The choice of images follows the usual scraping process on Screenscraper.



You can follow the progress of the scrapes (number of media recovered) by doing an “undock” of the database before launching the scrape. A text zone above the log window indicates the different elements retrieved.





LIMITATIONS :

Some ARRM features are not available in multi-threaded mode to optimize speed:

  • No local image search.
  • No creation of temporary gamelist (in case of crash). But you can enable caching. See:How to use the ARRM cache
Scrapes not found with multiu-threads are listed in log files located in %appdata%\Nexouille Soft\Arrm\temp, in the format:


_missing_arrm_%SYSTEM%_screenscraper.txt

example: _missing_arrm_ZXSPECTRUM_screenscraper.txt



scraper-en.txt · Dernière modification : 2022/11/27 10:26 de nexusone13