Blog Technology Uncategorized

web scraping using python

admin

webbrowser Comes with Python and opens a browser to a specific page.

Requests Downloads files and web pages from the Internet.

Beautiful Soup Parses HTML, the format that web pages are written in.

Selenium Launches and controls a web browser. Selenium is able to fill in forms and simulate mouse clicks in this browser.

webbrowser Module

The webbrowser module’s open() function can launch a new browser to a specified URL.

>> import webbrowser
>>> webbrowser.open('https://www.flyhiee.com/')
True

A web browser tab will open to the URL https://www.flyhiee.com/.

Program to launch a map in the browser using an address from the command line or clipboard.

#Python3
#mapIt.py - Launches a map in the browser usng as address from the command line
import webbrowser, sys,pyperclip
if len(sys.argv) > 1:
    #Get address from command line.
    address = ' '.join(sys.argv[1:])
    #sys.argv is a list of string.
    #You don't want the program name in this string, so instead of sys.argv, you should pass sys.argv[1:] to chop off the first element of the array.
else:
    #Get address from clipboard.
    address = pyperclip.paste()
webbrowser.open('https://www.google.com/maps/place/' + address)

If you are getting an error ModuleNotFoundError: No module named ‘pyperclip’

You can fix this by installing pip3 install pyperclip

You may Encounter another Error like this after installing pyperclip is Pyperclip could not find a copy/paste mechanism for your system.

You can fix this by installing one of the copy/paste mechanisms:

  • sudo apt-get install xsel to install the xsel utility.
  • sudo apt-get install xclip to install the xclip utility.
  • pip install PyGTK to install the gtk Python module.
  • pip install PyQt5 to install the PyQt5 Python module.
Pyperclip

The purpose of Pyperclip is to provide a cross-platform Python module for copying and pasting text to the clipboard.

To copy text to the clipboard, pass a string to pyperclip.copy(). To paste the text from the clipboard, call pyperclip.paste() and the text will be returned as a string value. (source Pypi)

>>> import pyperclip
>>> pyperclip.copy('Hello world!')
>>> pyperclip.paste()
'Hello world!'

Leave a Reply