Data Scraping

What is Data Scraping?

Data scraping, a technique in which a computer program extracts data from the output generated from another program. Data scraping is commonly manifest in web scraping, the process of using an application to extract valuable information from a website.

How does Data Scraping can be done?

The process of web scraping is fairly simple, though the implementation can be complex. Web scraping occurs in 3 steps:

• First, the piece of code used to pull the information, which we call a scraper bot, sends an HTTP GET request to a specific website.

• When the website responds, the scraper parses the HTML document for a specific pattern of data.

• Once the data is extracted, it is converted into whatever specific format the scraper bot’s author designed.

Web Scraping Tools:

• Import.io

• Dexi.io (formerly known as CloudScrape)

• Scrapinghub

• ParseHub

• Scraper

Types and Examples

Many real estate agents use web scraping to populate their database of available properties for sale or rent. For example, a real estate agency will scrape MLS listings to build an API that directly populates this information onto their website. This way, they get to act as the agent for the property when someone finds this listing on their site. Most listings that you will find on a Real Estate website are automatically generated by an API.