• Jason Nickola

Iterating Dates in a URL with Python

I was recently working with a site that included a date in the URL for certain resources and wanted to quickly generate requests across a wide range of dates. There are lots of ways to do this, but this was a quick and easy way using dateutil's rrule and requests in python.

import requests
from datetime import date
from dateutil.rrule import rrule, DAILY

for date in rrule(DAILY, dtstart=date(2021,1,1),until=date(2021,1,31)):
    url = "https://example.com/resources/?object={}.xml".format(date.strftime("%Y-%m-%d"))
    if requests.get(url).status_code == 200:

The rrule will generate dates according to the frequency (DAILY in this case), beginning at the dtstart date and ending with the date provided to the until parameter. With the ability to generate the desired date range, you can craft URLs by using Python's string format to inject each date into the URL string. The response content, status_code (200 in this example), or other properties can be used to easily filter for results you are interested in.

23 views0 comments