no way to compare when less than two revisions

Differences

This shows you the differences between two versions of the page.


—proxy_scraper:search_engine_parser [2016-03-03 14:36] (current) – created sven
Line 1: Line 1:
 +====== Search Engine Parser ======
  
 +You can reach the following screen from the Main GUI using the //TOOLS// button.
 +
 +{{ :proxy_scraper:ps_tool_search_engine_parser.png |}}
 +
 +On the Left you can set things up from Search Engines to use, Keywords to parse and the Options.
 +Once you press //START// you will see the progress in the log window at the bottom.
 +
 +**Options are as follow**:
 +
 +
 +**Use proxies with tags** - It makes sense to use e.g. only proxies who have been tested with Google. However if you use also other search engnes, then it might be faster when using all proxies.
 +
 +**Thread** - How many simultaneous queries should be made.
 +
 +**Time to wait between requests** - To not get banned by search engines, it is important to use wait a certain time between queries. This time is measured for search engine domain + proxy used by it. 
 +
 +**Use same proxy for same search engine/keyword** - It might look suspicious for a search engine if you crawl page 1 with one proxy and page 2 with another one. This option however slows things down as the timing between requests has to elapse before a new page can be parsed.