web scraping using python
webbrowser Comes with Python and opens a browser to a specific page.
Requests Downloads files and web pages from the Internet.
Beautiful Soup Parses HTML, the format that web pages are written in.
Selenium Launches and controls a web browser. Selenium is able to fill in forms and simulate mouse clicks in this browser.
webbrowser Module
The webbrowser module’s open() function can launch a new browser to a specified URL.
>> import webbrowser >>> webbrowser.open('https://www.flyhiee.com/') True
A web browser tab will open to the URL https://www.flyhiee.com/.
Program to launch a map in the browser using an address from the command line or clipboard.
#Python3 #mapIt.py - Launches a map in the browser usng as address from the command line import webbrowser, sys,pyperclip if len(sys.argv) > 1: #Get address from command line. address = ' '.join(sys.argv[1:]) #sys.argv is a list of string. #You don't want the program name in this string, so instead of sys.argv, you should pass sys.argv[1:] to chop off the first element of the array. else: #Get address from clipboard. address = pyperclip.paste() webbrowser.open('https://www.google.com/maps/place/' + address)
If you are getting an error ModuleNotFoundError: No module named ‘pyperclip’
You can fix this by installing pip3 install pyperclip
You may Encounter another Error like this after installing pyperclip is Pyperclip could not find a copy/paste mechanism for your system.
You can fix this by installing one of the copy/paste mechanisms:
sudo apt-get install xsel
to install the xsel utility.sudo apt-get install xclip
to install the xclip utility.pip install PyGTK
to install the gtk Python module.pip install PyQt5
to install the PyQt5 Python module.
Pyperclip
The purpose of Pyperclip is to provide a cross-platform Python module for copying and pasting text to the clipboard.
To copy text to the clipboard, pass a string to pyperclip.copy()
. To paste the text from the clipboard, call pyperclip.paste()
and the text will be returned as a string value. (source Pypi)
>>> import pyperclip >>> pyperclip.copy('Hello world!') >>> pyperclip.paste() 'Hello world!'