Feat: Some new source websites not fully integrated.
This commit is contained in:
51
README.md
Normal file
51
README.md
Normal file
@@ -0,0 +1,51 @@
|
||||
## Run product scraper manual
|
||||
1. create virtual environment
|
||||
python3 -m venv env_api
|
||||
|
||||
2. enter virtual environment
|
||||
source ./env_api/bin/activate
|
||||
|
||||
3. intall necessary modules
|
||||
pip3 install -r requirements.txt
|
||||
|
||||
4. Change to product scraping main directory
|
||||
- Same local directory as 'TCG Sole Trader Copy.xlsx' workbook
|
||||
cd product_scraping
|
||||
|
||||
5. Call controller main method
|
||||
python3 single_run/product_scraper_controller.py
|
||||
|
||||
6. exit virtual environment
|
||||
deactivate
|
||||
|
||||
|
||||
## Run product scraper continuous
|
||||
""" Call
|
||||
EMAIL_SENDER=bot@partsltd.co.uk \
|
||||
EMAIL_PASSWORD=@sJt1b5Cliu6 \
|
||||
EMAIL_RECIPIENT=teddy@shuffleandskirmish.co.uk \
|
||||
python3 product_scraper.py
|
||||
|
||||
|
||||
|
||||
One-time setup (edit credentials first):
|
||||
|
||||
|
||||
# 1. Fill in your email details in the service file
|
||||
nano product_scraping/tcg-scraper.service
|
||||
|
||||
# 2. Copy to systemd and enable
|
||||
sudo cp product_scraping/tcg-scraper.service /etc/systemd/system/
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable tcg-scraper # auto-start on boot
|
||||
sudo systemctl start tcg-scraper
|
||||
Day-to-day controls:
|
||||
|
||||
|
||||
sudo systemctl stop tcg-scraper # stop immediately
|
||||
sudo systemctl start tcg-scraper # start again
|
||||
sudo systemctl restart tcg-scraper # restart (e.g. after editing the .py)
|
||||
sudo systemctl status tcg-scraper # is it running? last exit code?
|
||||
journalctl -fu tcg-scraper # live log tail (Ctrl+C to exit)
|
||||
journalctl -u tcg-scraper --since today # today's log
|
||||
"""
|
||||
Reference in New Issue
Block a user