Run product scraper manual
-
create virtual environment python3 -m venv env_api
-
enter virtual environment source ./env_api/bin/activate
-
intall necessary modules pip3 install -r requirements.txt
-
Change to product scraping main directory
- Same local directory as 'TCG Sole Trader Copy.xlsx' workbook cd product_scraping
-
Call controller main method python3 single_run/product_scraper_controller.py
-
exit virtual environment deactivate
Run product scraper continuous
""" Call
EMAIL_SENDER=bot@partsltd.co.uk
EMAIL_PASSWORD=@sJt1b5Cliu6
EMAIL_RECIPIENT=teddy@shuffleandskirmish.co.uk
python3 product_scraper.py
One-time setup (edit credentials first):
1. Fill in your email details in the service file
nano product_scraping/tcg-scraper.service
2. Copy to systemd and enable
sudo cp product_scraping/tcg-scraper.service /etc/systemd/system/ sudo systemctl daemon-reload sudo systemctl enable tcg-scraper # auto-start on boot sudo systemctl start tcg-scraper Day-to-day controls:
sudo systemctl stop tcg-scraper # stop immediately sudo systemctl start tcg-scraper # start again sudo systemctl restart tcg-scraper # restart (e.g. after editing the .py) sudo systemctl status tcg-scraper # is it running? last exit code? journalctl -fu tcg-scraper # live log tail (Ctrl+C to exit) journalctl -u tcg-scraper --since today # today's log """