moshferatu/database-automation
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Backfill script for news events

Browse Source
This commit is contained in:
moshferatu 2023-10-25 06:31:23 -07:00
parent aad0cc59eb
commit 7035fe69dc
2 changed files with 14 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
news/backfill_news.py Normal file
View File

@ -0,0 +1,9 @@
from database.news import insert
from datetime import datetime
from news import scrape_news_in_range
start_date = datetime(2023, 10, 25)
end_date = datetime(2023, 10, 25)
scrape_news_in_range(start_date, end_date)
for news_data in scrape_news_in_range(start_date, end_date):
insert(news_data)

5
news/requirements.txt Normal file
View File

@ -0,0 +1,5 @@
pandas
psycopg2
python-dotenv
schedule
sqlalchemy