Python Language Web scraping with Python Scraping with curl

Help us to keep this website almost Ad Free! It takes only 10 seconds of your time:
> Step 1: Go view our video on YouTube: EF Core Bulk Insert
> Step 2: And Like the video. BONUS: You can also share it!



from subprocess import Popen, PIPE
from lxml import etree
from io import StringIO


user_agent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36'
url = ''
get = Popen(['curl', '-s', '-A', user_agent, url], stdout=PIPE)
result ='utf8')

-s: silent download

-A: user agent flag


tree = etree.parse(StringIO(result), etree.HTMLParser())
divs = tree.xpath('//div')

Got any Python Language Question?