| Both sides previous revision Previous revision Next revision | Previous revision Next revisionBoth sides next revision |
| search_engine_ranker:script_manual [2017-09-11 06:31] – [Data Extraction] sven | search_engine_ranker:script_manual [2019-06-16 15:20] – sven |
|---|
| * [[http://forum.gsa-online.de/discussion/14/tutorial-how-to-find-new-platforms-footprints-and-urls|How to find new platforms, footprints and URLs]] | * [[http://forum.gsa-online.de/discussion/14/tutorial-how-to-find-new-platforms-footprints-and-urls|How to find new platforms, footprints and URLs]] |
| * [[http://forum.gsa-online.de/discussion/18031/some-snippets-to-accelerate-ini-writing|Some snippets to accelerate .ini writing]] | * [[http://forum.gsa-online.de/discussion/18031/some-snippets-to-accelerate-ini-writing|Some snippets to accelerate .ini writing]] |
| | * [[https://github.com/seospace/gsapi|gsapi - Simple HTTP server for GSA Search Engine Ranker]] |
| ===== The Structure ===== | ===== The Structure ===== |
| |
| |seconds to wait before submission condition|This will only delay the submission if something in the variable content is found on the webpage.\\ \\ //Example: seconds to wait before submission condition=stop_spam_time//\\ \\ If the word "//stop_spam_time"// is either present in the html source or the visible text we wait with the submission a given time, else we skip this.| | |seconds to wait before submission condition|This will only delay the submission if something in the variable content is found on the webpage.\\ \\ //Example: seconds to wait before submission condition=stop_spam_time//\\ \\ If the word "//stop_spam_time"// is either present in the html source or the visible text we wait with the submission a given time, else we skip this.| |
| |post data|This is hardly used but will create a custom data that is used to submit to websites instead of using the data from <form>.\\ \\ //Example: post data=this=that&something=else//\\ \\ Have a look in the Pingback or Trackback engine where this is used.| | |post data|This is hardly used but will create a custom data that is used to submit to websites instead of using the data from <form>.\\ \\ //Example: post data=this=that&something=else//\\ \\ Have a look in the Pingback or Trackback engine where this is used.| |
| |encode post data|1 = encode the data in a proper way as used in POST protocol\\ 2 = encode it using multipart \\ 0 = take the data as it is without encoding anything| | |encode post data|1 = encode the data in a proper way as used in POST protocol\\ 2 = encode it using multipart \\ 0 = take the data as it is without encoding anything\\ 3 = encode it using json syntax| |
| |variable must be used|A form is only submitted if certain variables have been used in that form.\\ \\ //Example: variable must be used=url,login%%|%%email//\\ \\ The above example requires a form to use the variable "url" or "login" and "email".\\| | |variable must be used|A form is only submitted if certain variables have been used in that form.\\ \\ //Example: variable must be used=url,login%%|%%email//\\ \\ The above example requires a form to use the variable "url" or "login" and "email".\\| |
| |add fixed data add fixed data condition\\ \\ \\ \\ //remove fixed data//\\ \\ //remove fixed data condition//|This will add a form field if the thing in "add fixed data condition" is found on the webpage. This works only if both variables are defined. Don't worry about this to much as you will probably never use this. It is only required if some sites add data to forms by javascript to prevent automated submissions. You can use "remove fixed data" to just remove that from the posting.\\ \\ //Example: //\\ //add fixed data=code=%captcha%;hidden=0//\\ //add fixed data condition=code_sign;hidden_data//\\ \\ If the webpage has anywhere the word "code_sign" in it we add a new form field called code that will be filled with a captcha. And if the word "hidden_data" is found we also add a new form field called "hidden" with the content "0".| | |add fixed data add fixed data condition\\ \\ \\ \\ //remove fixed data//\\ \\ //remove fixed data condition//|This will add a form field if the thing in "add fixed data condition" is found on the webpage. This works only if both variables are defined. Don't worry about this to much as you will probably never use this. It is only required if some sites add data to forms by javascript to prevent automated submissions. You can use "remove fixed data" to just remove that from the posting.\\ \\ //Example: //\\ //add fixed data=code=%captcha%;hidden=0//\\ //add fixed data condition=code_sign;hidden_data//\\ \\ If the webpage has anywhere the word "code_sign" in it we add a new form field called code that will be filled with a captcha. And if the word "hidden_data" is found we also add a new form field called "hidden" with the content "0".| |
| |set cookie\\ \\ set cookie condition|Sometimes cookies are required to submit a form and they are set by javascript so that the program can not detect them. You can do that manually with these variables. "set cookie condition" doesn't have to be present.\\ \\ //Example: set cookie=passed_check=1//\\ \\ This sets a cookie named "passed_check" with the value "1".| | |set cookie\\ \\ set cookie condition|Sometimes cookies are required to submit a form and they are set by javascript so that the program can not detect them. You can do that manually with these variables. "set cookie condition" doesn't have to be present.\\ \\ //Example: set cookie=passed_check=1//\\ \\ This sets a cookie named "passed_check" with the value "1".| |
| |write file|Writes data to a file.\\ \\ //Example: write file="C:\temp\accounts.txt" "%targeturl%;%login%;%password%"//| | |write file|Writes data to a file.\\ \\ //Example: write file="C:\temp\accounts.txt" "%targeturl%;%login%;%password%"//| |
| | |recaptcha sitekey|Get the required recaptcha site key\\ \\ //Example: recaptcha sitekey=<fixed key|section|url where to get it>//\\ \\ You can either input the sitekey or the url where to extract it or a section from the script how to extract it like:\\ \\recaptcha sitekey=get_key\\ \\ [get_key]\\ type=extract\\ url=/recaptcha.js\\ front=sitekey="\\ back="| |
| |
| ===== Variables usable in SETUP and STEP Section ===== | ===== Variables usable in SETUP and STEP Section ===== |
| |
| ^Variable^Allowed values / Description^ | ^Variable^Allowed values / Description^ |
| |submit success\\ \\ submit failed\\ \\ captcha failed\\ \\ submit failed retry|At the end of each STEP* the resulting webpage content is checked first against "submit success" and than against "submit failed". And if one is matching the submission is either set as successful or failed. The text is not case sensitive and multiple items can be added with %%|%%.\\ \\ //Example:\\ submit failed=<div id='errorbox'>%%|%%wrong code%%|%%letters wrong%%|%%user name already taken\\ submit success=submitted successfully\\ captcha failed=wrong code%%|%%letters wrong\\ submit failed retry=user name already taken//\\ \\ If the text "submitted successfully" appears on the webpage (source or visible text) then the submission is taken as successful. If the text "<div id='errorbox'>" is found then the submission is aborted. | | |submit success\\ \\ submit failed\\ \\ captcha failed\\ \\ submit failed retry|At the end of each STEP* the resulting webpage content is checked first against "submit success" and than against "submit failed". And if one is matching the submission is either set as successful or failed. The text is not case sensitive and multiple items can be added with %%|%%.\\ \\ //Example:\\ submit failed=<div id='errorbox'>%%|%%wrong code%%|%%letters wrong%%|%%user name already taken\\ submit success=submitted successfully\\ captcha failed=wrong code%%|%%letters wrong\\ submit failed retry=user name already taken//\\ \\ If the text "submitted successfully" appears on the webpage (source or visible text) then the submission is taken as successful. If the text "<div id='errorbox'>" is found then the submission is aborted.\\ \\ Special keywords can be used such as //inurl:// or /httpcode://. | |
| |submit success skip verify|If a submission is detected as successful and it is e.g. a REGISTER_STEP* then we can skip the verification process if certain stings appear on the page.\\ \\ //Example: submit success skip verify=>Logout<%%|%%>Log Out<//\\ \\ After a successful registration we would normally wait till an verification email arrives to continue with login and submission. However some platforms might allow you to login without this or log you in already. In that case it would be a waste of time to wait for a email from them so we continue to login and submit the content.| | |submit success skip verify|If a submission is detected as successful and it is e.g. a REGISTER_STEP* then we can skip the verification process if certain stings appear on the page.\\ \\ //Example: submit success skip verify=>Logout<%%|%%>Log Out<//\\ \\ After a successful registration we would normally wait till an verification email arrives to continue with login and submission. However some platforms might allow you to login without this or log you in already. In that case it would be a waste of time to wait for a email from them so we continue to login and submit the content.| |
| |verify on unknown status|1 = if a submission is not detected as successful or failed it will still be taken as successful (appearing in log with "unknown submission status")\\ 0 = we assume the submission failed (default)| | |verify on unknown status|1 = if a submission is not detected as successful or failed it will still be taken as successful (appearing in log with "unknown submission status")\\ 0 = we assume the submission failed (default)| |