import only day specified from url list

Question

I'm new to python.
using Python 2.7 what I'm trying to achieve is I got list of url in script each url need to run on specific day in cronjob. i need to import datetime and import time get server day ("%A") and match with url to run on that day this way will save me for each url doing same script

import urllib2,urllib
import re,os
import datetime
import time

mylist={
monday:'http://www.trt.tv/baba-candir/bolumler/80212',
thusday:'http://www.trt.tv/filinta/bolumler/20198',
wensday:'http://www.trt.tv/dirilis-ertugrul/bolumler/20196',
}
for days in mylist:
    print days
    req = urllib2.Request(url)
    req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; az-AZ; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3')
    response = urllib2.urlopen(req)
    link=response.read()
    response.close()

Dictionary keys must be quoted (i.e. `monday` should be `'monday'`). But apart from this, what is your question? — Andrea Corbellini, Jan 11 '16 at 11:10
i need to get server day in a format (Monday) and look in to url list match the Monday url and start scraping only Monday url. hope this explain better — Murat Yilmaz, Jan 11 '16 at 11:15

pposca · Answer 1 · 2016-01-11T11:45:16.890

from datetime import date

today = date.today().strftime('%A')

If today is Monday, for example, this will return 'Monday'. If you want it in lowercase, you can do: today.lower()

All together:

import urllib2,urllib
import re,os
import time
from datetime import date

mylist = {
    'Monday':'http://www.trt.tv/baba-candir/bolumler/80212',
    'Thursday':'http://www.trt.tv/filinta/bolumler/20198',
    'Wednesday':'http://www.trt.tv/dirilis-ertugrul/bolumler/20196',
}


today = date.today().strftime('%A')
if today in mylist:
    url = mylist[today]

    req = urllib2.Request(url)
    req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; az-AZ; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3')
    response = urllib2.urlopen(req)
    link=response.read()
    response.close()

score 0 · Accepted Answer · edited May 23 '17 at 12:15

0

As is stated in this answer, the day can be taken from

import datetime
print(datetime.datetime.today().weekday())
# prints '0' because it's Monday

This prints a number between 0 (Monday) and 6 (Sunday). So, in your code (if I understand correctly):

import urllib2,urllib
import re,os
import datetime
import time

mylist={
0:'http://www.trt.tv/baba-candir/bolumler/80212',
1:'http://www.trt.tv/filinta/bolumler/20198',
2:'http://www.trt.tv/dirilis-ertugrul/bolumler/20196',
}

today = datetime.datetime.today().weekday()
if today in mylist:
    # Forgotten?
    url = mylist[today]
    # Forgotten?     

    req = urllib2.Request(url)
    req.add_header('User-Agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; az-AZ; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3')
    response = urllib2.urlopen(req)
    link=response.read()
    response.close()
else:
    print('error: today (day %s) was not found in mylist.' % today)

edited May 23 '17 at 12:15

Community

1
1

answered Jan 11 '16 at 11:35

Nander Speerstra

1,496
6
24
29

great answer thank you last question if I run this on cronjob if day not in mylist=[] how can I avoid getting error and close() – Murat Yilmaz Jan 11 '16 at 11:43
`for day in mylist: if day == datetime.datetime.today().weekday():` Is a very long way around. Why not just `if datetime.datetime.today().weekday() in mylist:` – Holloway Jan 11 '16 at 11:49
@Holloway: .weekday() is always in mylist (each weekday is present). But thanks for the remark, because I can speed up my by removing the superfluous for-loop! – Nander Speerstra Jan 11 '16 at 11:59
@MuratYilmaz: The if checks whether the weekday is in the list. I now added an extra 'else' that gives an error message (better than just exiting the script). – Nander Speerstra Jan 11 '16 at 12:04

import only day specified from url list

2 Answers2