Steps to run the crawler locally:

1. Dependencies:

To be able to run crawler locally you have to meet following dependencies:

• python3, pip3 and scrapy framework

1. Install App Requirements:

• Switch to the project root directory.
• run $ pip3 install -r requirements.txt

4. Run the crawler

• Switch to project root directory.
• run command to run the script python3 run_scraper.py -u <category url>
• example run
  • python3 run_scraper.py -u https://apps.apple.com/us/genre/ios-books/id6018