Hi everyone, Today I want to show you how to write a web-scraper to get a title of a website in Python. This will include some python libraries.

Step 1: Meet the requirements

You will need :-

  • Python installed on your computer
  • A text editor
  • Python’s pip installed on your computer (after version 3.5 it comes installed by default)
  • Internet

Step 2: Install the packages

Now, we will install the required packages using pip. Install usin the following commands

pip install requests
pip install bs4
pip install lxml

OR

pip3 install requests
pip3 install bs4
pip3 install lxml

Step 3: Writing the code

Now, we will write the code. Make sure you followed step 2 or it will not work. Make sure to include http:// before you put a url or it will return a error.

The code is :-

import requests
import bs4

chosen_url = "https://virajkhanna.in/"
result = requests.get(chosen_url)
soup = bs4.BeautifulSoup(result.text, "lxml")

title_of_website = soup.select('title')[0].getText()

print(title_of_website)

This will go to the website (You need the internet) and get the source code. Then bs4 will parse through the code and find the code that shows the title and grabs the text of the title and displays it.

Step 4: Try it yourself

Now, you can try it yourself. Try grabbing something else from another website. At least for this tutorial post, we are done!

Done!

Post Views: 103