python


Python: To find the website of a list of companies


I tried using the below python code to find the websites of the companies. But after trying few times, I face a Service Unavailable error.
I have finished the first level of finding the possible domains of the companies. For example :
CompanyExample [u'http://www.examples.com/', u'https://www.example.com/quote/CGL:SP', u'http://example2.sgx.com/FileOpen/China%20Great%20Land.ashx?App=Prospectus&FileID=3813', u'https://www.example3.com/php/company-profile/SG/en_2036109.html']
from google import search
for link in links:
parsed_uri = urlparse(link)
domain = '{uri.scheme}://{uri.netloc}/'.format(uri=parsed_uri)
for url in search(domain,stop = 4):
print url
Kindly help me on:
Why do I find urllib2.HTTPError: HTTP Error 503: Service Unavailable error suddenly.
Is there any other method(Python requests) to find websites of the list of companies ?
Google APIs usually are rate limited for non-paying users. Going over your limit is probably what causes the 503 responses. According to the API documentation you get 100 free searches per day, after that it's $5 per $1000 queries up to 10000 queries:
Custom Search Engine (free)
For CSE users, the API provides 100 search queries per day for free.
If you need more, you may sign up for billing in the API Console.
Additional requests cost $5 per 1000 queries, up to 10k queries per
day.

Related Links

Parse XML time series data to dat files
Google App Engine DB Query Memory Usage
Taking two lists and printing specific characters based on their postition
How to call a C++ function which returns a vector of doubles from Python?
Force datetime with hour and minutes to null pandas
Difference between _sql_constraints and _constraints on OpenERP/Odoo?
Is there a Python JSON encoder which JUST works?
Parse large split XML file(s) with Python
Tried everything but still cannot serve static files of Django project via nginx+gunicorn
Bin values based on ranges with pandas
Read specific lines from text file as numpy array
Automatically remove referencing objects on deletion by mongoengine in django
How to get values from split function?
Developing Geoprocessing Tool in ArcPy
Constructing efficient functions in Python
Numpy combine all nonzero elements of one array in to another

Categories

HOME
ajax
cluster-computing
ssas-2012
puzzle
checkbox
textwatcher
mainframe
rfc
adsense
algorithmic-trading
schemacrawler
gimp
pca
ndis
xlsx
data-synchronization
qt-installer
simple-injector
squarespace
aspell
pyephem
flexlm
pycrypto
multiple-columns
flexboxgrid
excel-2010
epicor
profile
searchbar
iis-10
rhandsontable
xilinx-ise
aurelia-binding
vertex-buffer
encase
backup-strategies
hotmail
body-parser
salesforce-chatter
large-data
rider
npm-publish
vmd
equivalence
polyfills
mediawiki-extensions
facebook-chatbot
encapsulation
dojox.grid.datagrid
data-integration
sapui
rhel6
spring-data-hadoop
slot
nuget-server
gce
cdk
python-hypothesis
barcode-printing
sql-server-administration
coremidi
json-schema-validator
try-finally
amf
boost-hana
gmsmapview
faraday
mongo-c-driver
test-class
self-hosting
evo
kendo-menu
ipojo
shift-jis
hg-git
tilestache
ultrawingrid
dmp
facebook-wall
yahoo-boss-api
operations
dynamic-proxy
layered
driver-signing
goinstant
justgage
nuspec
magickwand
code-conversion
moq-3
pstree
subtract
throttling
scraperwiki
noir
smooth
recordset
google-instant
openvg
openwysiwyg
web-statistics
economics

Resources

Mobile Apps Dev
Database Users
javascript
java
csharp
php
android
MS Developer
developer works
python
ios
c
html
jquery
RDBMS discuss
Cloud Virtualization
Database Dev&Adm
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App