python


How to read and map the description of the google search result using python request ?


I used python requests to find the google search result. I'm able to find the links and descriptions but how can I map the links with its respective description.
Below is the code and its output:
req=urllib2.Request('http://www.google.com/search?hl=en&safe=off&q='+'cat')
sock=urllib2.urlopen(req)
data = sock.read()
soup = BeautifulSoup(data, 'lxml')
if(soup):
links = soup.findAll('cite')
spans = soup.find_all("span", {"class": "st"})
for link, span in ((l,s) for l in links for s in spans):
print link.text, span.get_text()
Output:
https://en.wikipedia.org/wiki/Cat, The domestic cat (Latin: Felis catus) is a small, typically furry, carnivorous mammal. They are often called house cats when kept as indoor pets or simply cats
https://en.wikipedia.org/wiki/Cat, A non-profit organisation, run entirely by volunteers who want to improve the welfare of stray cats in Singapore.
https://en.wikipedia.org/wiki/Cat, Manufacturer of construction and mining equipment, diesel and natural gas engines, industrial gas turbines, and a wide offering of related services.
www.catwelfare.org/,The domestic cat (Latin: Felis catus) is a small, typically furry, carnivorous mammal. They are often called house cats when kept as indoor pets or simply cats
www.catwelfare.org/, A non-profit organisation, run entirely by volunteers who want to improve the welfare of stray cats in Singapore.
www.catwelfare.org/, Manufacturer of construction and mining equipment, diesel and natural gas engines, industrial gas turbines, and a wide offering of related services.
etc..
The same link is mapped with all the description of the google search result.
The required output is,
https://en.wikipedia.org/wiki/Cat, The domestic cat (Latin: Felis catus) is a small, typically furry, carnivorous mammal. They are often called house cats when kept as indoor pets or simply cats
www.catwelfare.org/, A non-profit organisation, run entirely by volunteers who want to improve the welfare of stray cats in Singapore.
www.cat.com/,Manufacturer of construction and mining equipment, diesel and natural gas engines, industrial gas turbines, and a wide offering of related services.
Kindly help me to resolve this.


Related Links

SQLAlchemy's or_ in remodel
Code won't loop through the correct while loop. Python
Python - What's wrong with my code?
Adding an Amazon RDS Database Instance to your Elastic Beanstalk Environment django
Make a dot-plot in python
How to scrape geographic coordinates from dynamic map?
How to pip install tensorflow on El Capitan?
How can made multiple user login at same time in same browser in django project
python using self in decorators in class with kwarg
Workflow to work on a github fork of a python library?
numba NotImplementedError: Unknown operator 'in'
Django, store function in database
count the number of dictionaries in a string python
Odoo : Value in one2many doen't change in api.depends
Django text in SmallIntegerField results in a KeyError
regex returns None on parsing address

Categories

HOME
caching
security
docker-swarm
vlc
orientation
value
codeblocks
sudo
goutte
pca
xamarin-studio
jacoco
datagrip
sql-tuning
scaling
zope
yeoman-generator
ng-show
ms-access-2007
movies
flexlm
distance
cvs2svn
url.action
avplayeritem
explode
spring-test
hanami
jenkins-job-dsl
codelite
nesc
autofill
ioc-container
skygear
npm-shrinkwrap
node-horseman
shibboleth
meta-raspberrypi
freetts
s3cmd
cross-entropy
rule
wcf-security
ddms
exiftool
productivity
service-fabric-stateful
median
lftp
mbaas
etherpad
react-native-fbsdk
launch
jszip
unity3d-editor
packagemaker
cron-task
children
visible
fps
post-processing
usb-drive
portfolio
void
code-readability
p-np
oai
retro-computing
adodb
collabnet
funq
colt
picat
lumx
slick-2.0
broadcasting
internal
mimosa
healthvault
windows-mobile-6
convex-polygon
robospice
digital-design
htmltextwriter
process-monitor
lync-server-2010
crocodoc
deploying
z-machine
throttling
querypath
sitemappath
windows-controls
opcodes
configurable
dotproject





Mobile Apps Dev
Database Users
javascript
java
csharp
php
android


MS Developer
developer works
python
ios
c
html
jquery


RDBMS discuss