3

I would like to scrape data from one web page. My code looks like this:

grad = s.get('https://www.njuskalo.hr/prodaja-kuca/zagreb',headers=header, proxies=proxyDict)
city_soup = BeautifulSoup(grad.text, "lxml")
kvarts = city_soup.find_all(id="locationId_level_1")
print kvarts[0]
print "++++++++++++++++++++++="

for kvart in kvarts[0]:
    print kvart

As result I get:

<option data-url-alias="/brezovica" value="1247">Brezovica</option>
<option data-url-alias="/crnomerec" value="1248">Črnomerec</option>
<option data-url-alias="/donja-dubrava" value="1249">Donja Dubrava</option>

From there I need to extract data-url-alias and value. How to do that?

Galen
  • 1,315
  • 7
  • 14
jack_oneal
  • 79
  • 7

1 Answers1

6

bs4 stores tag attributes in a dictionary so you can select them by name. 

for kvart in kvarts[0].find_all('option'):
    print kvart['data-url-alias'], kvart['value']

As mentioned by Evyatar Meged in the comments this will raise a KeyError if a key doesn't exist, so if you're not sure about it use the .get method. 

for kvart in kvarts[0].find_all('option'):
    print kvart.get('data-url-alias'), kvart.get('value')

dict.get returns None if a key doesn't exist (or you can set a default value)

t.m.adam
  • 14,756
  • 3
  • 29
  • 50