Internet / Web Authoring: Misc Tools

Advertisement


The web crawler is a program that automatically traverses the web by downloading the pages and following the links from page to page. A general purpose of web crawler is to download any web page that can be accessed through the links. This process is called web crawling or spidering. Many sites, in particular search engines, use spidering as a means of providing up-to-date data. Web crawlers are mainly used to create a copy of all the visited pages for later processing by a search engine that will index the downloaded pages to provide fast searches. Crawlers can also be used for automating maintenance tasks on a website, such as checking links or validating HTML code. Also, crawlers can be used to gather specific types of information from Web pages, such as harvest ing e-mail addresses (usually for spam). A web crawler is one type of bot, or software agent. In general, it starts with a list of URLs to visit, called the seeds. As the crawler visits these URLs, it identifies all the hyperlinks in the page and adds them to the list of URLs to visit, called the crawl frontier. URLs from the frontier are recursively visited according to a set of policies.Release notes: Betafirst public release [ Vietspider Web Data Extractor full changelog ]





  • License: Freeware
  • Cost: $0.00 USD
  • Size: 57.6 MB
Download    |   Publisher

Related Software

Title / Version / Description Size License
Excavator 1.0.11 - Diggernaut is a cloud based service for web scraping, data extraction and other ETL (Extract,... 45.5 MB Freeware
Just Data Extractor 1.0.17 - Just Data Extractor is the most powerful and easy-to-use data extraction software for web... 1.6 MB Demo
iSkysoft Free iPhone Data Recovery 1.3.0 - When important files are lost or deleted from your iPhone, iPad or iPod touch, you can always... 37.5 MB Freeware
Free Windows Data Recovery 14.0 - The analysis and testing of the Free Windows Data recovery software can be practiced to recover... 3.1 MB Shareware
7thShare Free Android Data Recovery 2.3.8.8 - 7thShare Free Android Data Recovery can recover Android files such as deleted text messages,... 16.2 MB Freeware
Free Card Data Recovery 2.6.6.8 - The best and reliable Free Card Data Recovery can help you recover any lost, deleted, formatted,... 11.3 MB Shareware
7thShare Free iPhone Data Recovery 2.6.8.8 - Free iPhone Data Recovery enables you to recover the lost/deleted data like Notes, Reminder,... 16.2 MB Freeware
Knowlesys Web Data Extractor 1.0 - Custom Web Data Extractor ==================== I want to run the extraction tasks in my house at... 13.0 KB
Knowlesys Screen Scraper 1.0 - Custom Web Data Screen Scraper ==================== I want to run the extraction tasks in my... 13.0 KB
Knowlesys Web Crawler 1.0 - Custom Web Data Extractor ==================== I want to run the extraction tasks in my house at... 13.0 KB
PixieRobot 2.1.0.14 - PixieRobot is a scriptable web scraping robot which can grab images and data from targeted Web... 2.0 MB Shareware

Vietspider Web Data Extractor User Reviews

More Reviews or Write Review