Introduction to Web Scraping with Python
Introduction
Web-scraping is an important technique, frequently employed in a lot of different contexts, especially data science and data mining. Python is largely considered the go-to language for web-scraping, the reason being the batteries-included nature of Python. With Python, you can create a simple scraping script in about 15 minutes and in under 100 lines of code. So regardless of usage, web-scraping is a skill that every Python programmer must have under his belt.
Before we start getting hands-on, we need to step back and consider what is web-scraping, when should we use it, and when to avoid using it.
As you already know, web-scraping is a technique employed to automatically extract data from websites. What’s important to understand is, web-scraping is a somewhat crude technique to extract data from various sources – typically web pages. If the developers of a website are generous enough to provide an API to extract data, that would be a lot more stable and robust way to do access the data. So, as a rule of thumb, if a website provides an API to programmatically retrieve their data, use that. If an API is not available, only then use web-scraping.
Be sure