1. Project Brief
Your task for this project is to scrape the first section of the Mathematics Wikipedia page using Python, along with the requests and the BeautifulSoup libraries, and save the results in a text file. Here is the link to the page you need to scrape:
https://en.wikipedia.org/wiki/Mathematics
This is the content you need to scrape:
2. Project Expected Output
Your code should generate a text file that should contain the content of the first section of the Mathematics Wikipedia article.
3. Environment Setup Instructions (in your local IDE)
👉 Skip to the next step if you prefer to code this project in an online browser-based IDE or from your mobile phone.
Install requests:
pip install requests
Install BeautifulSoup
pip install beautifulsoup4
To run the code, execute the main.py file with:
python main.py
4. Environment Setup Instructions (in an online IDE as alternative)
Prefer an online IDE? Use this cloud IDE link to start coding immediately with a pre-configured environment.
5. Resources
You can learn about scraping Wikipedia pages in the article below:
https://pythonhow.com/how/scrape-a-wikipedia-page/
I have a little suggestion on the code...
1) The variable "title" can be concatenated on "intro" for more style. (the variable "title" is never used)
2) I have error on charmap while del file is written (with other article in wikipedia), aggregating the argument "encoding=utf-8" worked for me.