Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
proxy_scraper:options [2015-06-10 22:01] – [Automatic Export] devinproxy_scraper:options [2016-11-09 11:24] (current) – [Automatic Export] sven
Line 4: Line 4:
  
 ===== Settings ===== ===== Settings =====
 +{{ :settings.png |}}
  
 ==== Internal Proxy Server ==== ==== Internal Proxy Server ====
Line 31: Line 32:
  
 ===== Provider ===== ===== Provider =====
 +{{ :provider.png |}}
  
 A website that offers free proxy servers is called a //provider// within GSA Proxy Scraper. The software comes with over 800 providers.  A website that offers free proxy servers is called a //provider// within GSA Proxy Scraper. The software comes with over 800 providers. 
Line 41: Line 43:
 When having this option checked, it will parse all those links for new proxies. When having this option checked, it will parse all those links for new proxies.
 Often new proxy sources are found that way that you can later add here as well. In the proxy list it is shown as "//Proxy-Search Links - ...//". Often new proxy sources are found that way that you can later add here as well. In the proxy list it is shown as "//Proxy-Search Links - ...//".
 +
 +Another option called "**Use Search Engines to locate proxy lists**" will activly use search engines (google, bing...) with your queries (**Other**) or the found **IP**s itself to find new proxies.
  
  
Line 52: Line 56:
  
 ===== Automatic Export ===== ===== Automatic Export =====
 +{{ :export.png |}}
  
 There are many options for you to export proxies automatically. You can define an interval and different ways to export. There are many options for you to export proxies automatically. You can define an interval and different ways to export.
Line 58: Line 63:
   * **FTP** - Uploads a file to a FTP Server   * **FTP** - Uploads a file to a FTP Server
   * **E-Mail** - Sends an E-Mail with an attached file with proxies in it   * **E-Mail** - Sends an E-Mail with an attached file with proxies in it
-  * **WEB Upload** - This sends proxies to a webserver. You can define everything on how this should happen (POST/GET)+  * **WEB Upload** - This sends proxies to a webserver. You can define everything on how this should happen (POST/GET). Also see this [[http://pastebin.com/uikYgRyj|sample script in php to accept uploads]].
  
 You can of course Edit or Delete your previously setup export options here. Having it checked in the list means it is used on automatic export, otherwise it can still be used in the Export Toolbar. You can of course Edit or Delete your previously setup export options here. Having it checked in the list means it is used on automatic export, otherwise it can still be used in the Export Toolbar.
Line 65: Line 70:
 ===== Automatic Search ===== ===== Automatic Search =====
  
-Having this checked means the program will search all the providers for new proxies. You can define the **Interval** here as well as the conditions on when this should happen or stopped.+{{ :auto_search.png |}} 
 + 
 +Having this checked means the program will search all the providers for new proxies. You can define the **Interval** here as well as the conditions on when this should happen or stop.
  
 The box with the different Tests is in fact very important. Usually people need proxies to be anonymous, you you should pick some tests here that offer this. The box with the different Tests is in fact very important. Usually people need proxies to be anonymous, you you should pick some tests here that offer this.
 Also the //Google Search// test might be of interest for you. Other tests are also available like //StopForumSpam// or tests if a proxy works with //facebook.com//. Also the //Google Search// test might be of interest for you. Other tests are also available like //StopForumSpam// or tests if a proxy works with //facebook.com//.
  
-Of course you can add your own tests here clicking the **Add** button or **Delete** those not of interest for you.+Of course you can add your own tests here clicking the **Add** button or **Delete** the tests that are not of interest for you.
  
 Each found proxy is tested against the selected tests. Each found proxy is tested against the selected tests.
Line 80: Line 87:
 Anyway it is not suitable to keep dead proxies forever in the hope that they will work once again. You should not disable the option **Automatically remove proxies** in order to not wast memory and other resources. Anyway it is not suitable to keep dead proxies forever in the hope that they will work once again. You should not disable the option **Automatically remove proxies** in order to not wast memory and other resources.
  
-The option to store removed proxies in an file enables you to test them all again once you feel like doing it. Thats possible clicking **Add -> Previously Removed** on the main interface.+The option to store removed proxies in an file enables you to test them all again if you want to go back and try that at a later time. That's possible by clicking **Add -> Previously Removed** on the main interface.
  
 ===== Filter ===== ===== Filter =====
  
-I don't recommend you to use any of the filter options as you can always define filters for the automatic exports. However for those who want this, there is the possibility to do this.+{{ :filter.png |}} 
 + 
 +I don't recommend you to use any of the filter options as you can always define filters for the automatic exports. However for those who want this, the following options are available:
    
   * **Do not accept anonymous (no elite) proxies**: This is not keeping proxies who identify them as such but not telling the remote website what IP you have.   * **Do not accept anonymous (no elite) proxies**: This is not keeping proxies who identify them as such but not telling the remote website what IP you have.
   * **Do not accept transparent proxies**: This is not keeping proxies who identify them as such AND tell the remote website what IP you have.   * **Do not accept transparent proxies**: This is not keeping proxies who identify them as such AND tell the remote website what IP you have.
-  * **Skip suspicious proxies**: Suspicious proxies are those who probably spy on your activities on the proxy server itself. Keep in mind that many of the proxy servers you find are run by spy agencies. Anyway even if a proxy is not tagged as such, it can still spy on you. You can not trust anyone at all thees days.+  * **Skip suspicious proxies**: Suspicious proxies are those who probably spy on your activities on the proxy server itself. Keep in mind that many of the proxy servers you find are run by spy agencies. Anyway even if a proxy is not tagged as such, it can still spy on you. You can not trust anyone at all these days.
   * **Accept only if tagged as**: Keep proxies only if they match a certain test and that test tagged the proxy as such.   * **Accept only if tagged as**: Keep proxies only if they match a certain test and that test tagged the proxy as such.
-  * **Accept only the following ports** OR **Skip the following ports**: This can be usefull if your firewall does not allow you to go online by certain ports anyway or if you simply need proxies from a specific port. +  * **Accept only the following ports** OR **Skip the following ports**: This can be useful if your firewall does not allow you to go online by certain ports or if you simply need proxies from a specific port. 
-  * **Skip duplicate IPs**: Often you have proxies being on the same IP/Host but with different ports. This can be a problem if you got too many of those and do requests to search engines who only see the IP and check against that to see if you are hammering it.+  * **Skip duplicate IPs**: Often you have proxies being on the same IP/Host but with different ports. This can be a problem if you have too many and do requests to search engines who only see the IP and check against that to see if you are hammering it.
   * **Accept only the following Types**: There are different types of proxies. WEB proxies are those that work like a normal http protocol but with a bit of a modification on the request header. They are only useful for web queries. Then you have Connect proxies, socks4 and socks5 who can basically be used for any purpose, not just website parsing.   * **Accept only the following Types**: There are different types of proxies. WEB proxies are those that work like a normal http protocol but with a bit of a modification on the request header. They are only useful for web queries. Then you have Connect proxies, socks4 and socks5 who can basically be used for any purpose, not just website parsing.
-  * **Accept only the following Regions**: Proxies and there location on Earth are determinate by a small database. In some cases it is required to just take proxies from a special location which can be setup here.+  * **Accept only the following Regions**: Proxies and there location on Earth are determined by a small database. In some cases it is required to just take proxies from a special location which can be setup here.