Both spider arguments and settings can be used to configure your spider. There is no strict RULE that mandates to use one or the other, but settings are more suited for parameters that, once set, don’t change much, while spider arguments are meant to change more often, even on each spider run and sometimes are required for the spider to run at all (for example, to set the START url of a spider).

To illustrate with an example, assuming you have a spider that needs to log into a site to SCRAPE data, and you only want to scrape data from a certain SECTION of the site (which varies each time). In that case, the CREDENTIALS to log in would be settings, while the url of the section to scrape would be a spider argument.

Both spider arguments and settings can be used to configure your spider. There is no strict rule that mandates to use one or the other, but settings are more suited for parameters that, once set, don’t change much, while spider arguments are meant to change more often, even on each spider run and sometimes are required for the spider to run at all (for example, to set the start url of a spider).

To illustrate with an example, assuming you have a spider that needs to log into a site to scrape data, and you only want to scrape data from a certain section of the site (which varies each time). In that case, the credentials to log in would be settings, while the url of the section to scrape would be a spider argument.