Hi all
I am new, and I want to share my software (no virus/trojan), at
mekaskus.tk/ScrapperMin
The idea is like this.
I have been thinking that a scripting language is needed to do POST/GET for website that makes coding easier. Why? Because it is quite complex to even get a <input> tag name/value from C#, which needed HTMLAgility pack or manually using IndexOf, then Substring etc
For every internet POST/GET, <form> tag is used to do POST/GET. Then there is security token inside the form to prevent cross site scripting, and we need to get the security token in order to do automatic web posting.
After writing several bots for website (auto post/auto bump/auto comment/auto private message). I come to think that a scripting language is needed because of all questions I can think of.
Then I write a Parser and create my own scripting language and name it "ScrapperMin".
To try the parser, I make an automation script to bump thread using script below.
Basically the script is a C like script, where Function1(function2(function3), function2b()); etc like except that you can create array of string using
Where MyFunc will accept a parameter with string[] and second parameter with string.
Back to above script, there interesting part is : WC_GetPostStringRaw
Which will get the POST string format "username=myid&password=mypass&securitytoken=AutoGetFromForm" and you just need to pass in myid and mypass, and the securitytoken etc and all other input inside the form index '1' of that page will be created, and you only pass what interest you to fill in, like the username, and password.
{PARAM0}, {PARAM1} and {PARAM2} is username, password and password again. The script detect {PARAM and whenever it detects it, it will replace with the script argument files. you run the script using
where param.txt is
Of course you need to register in the site before actually able to run the script successfully.
After having the POST string, you just call WC_PostPageComplete, to do POST using your final POST string where you have filled in the required fill such as username and password. And it will return the webpage and we check for string "Thank you for logging", if exist basically we have successfully log into the site.
What we do next is to find the thread ID by first getting the page content into PG3, and scrape the page ID that will be in the end of this string "newreply&noquote=1&p=". We use FILTER to get the first (index 0) of the occurence, and leave the rest. Then concatenante with the forum reply url using JOIN.
Finally we get the page to do posting of the forum. and using WC_GetPostStringRaw to get the form at index 3 which is the form to fill in for posting comment, we fill in "message" value pair with our input at {PARAM4} and remove the preview key pair and do a post.
And we shall see our comment posted to the forum.
ScrapperMin.exe can be scheduled in Task Scheduler and the parameter must be put correctly, then periodically it will bump your thread.
You can create your own script using ScrapperMinW.exe (windows form) and test run it, once successful you can write your own script to do Web Automation task of any website you need.
Feel free to post here to discuss.
I am new, and I want to share my software (no virus/trojan), at
mekaskus.tk/ScrapperMin
The idea is like this.
I have been thinking that a scripting language is needed to do POST/GET for website that makes coding easier. Why? Because it is quite complex to even get a <input> tag name/value from C#, which needed HTMLAgility pack or manually using IndexOf, then Substring etc
For every internet POST/GET, <form> tag is used to do POST/GET. Then there is security token inside the form to prevent cross site scripting, and we need to get the security token in order to do automatic web posting.
After writing several bots for website (auto post/auto bump/auto comment/auto private message). I come to think that a scripting language is needed because of all questions I can think of.
Then I write a Parser and create my own scripting language and name it "ScrapperMin".
To try the parser, I make an automation script to bump thread using script below.
Code:
SET('PG', WC_GetPage('hxxp://forums.hardwarezone.com.sg', ''));
SET('PS', WC_GetPostStringRaw('hxxp://forums.hardwarezone.com.sg', GET('PG'), '1', FORMAT('vb_login_username={PARAM0}&vb_login_md5password_utf={0}&vb_login_md5password={1}', SO_GetMd5Hash('{PARAM1}'),SO_GetMd5Hash('{PARAM2}')), ''));
SET('PG2', WC_PostPageComplete(WC_GetLastActionUrl(), GET('PS'), 'hxxp://forums.hardwarezone.com.sg'));
IF(GET('PG2'), 'CONTAINS', 'Thank you for logging',
LOG('Successfully Login');
SET('PG3', WC_GetPage('{PARAM3}', ''));
SET('URL', JOIN('', 'hxxp://forums.hardwarezone.com.sg/newreply.php?do=newreply&noquote=1&p=', FILTER('0', SO_TagMatch(GET('PG3'), '/newreply.php?do=newreply&noquote=1&p=', '"'))));
SET('PG4', WC_GetPage(GET('URL'), 'hxxp://forums.hardwarezone.com.sg'));
SET('PS2', WC_GetPostStringRaw(GET('URL'), GET('PG4'), '3', 'message={PARAM4}', 'preview'));
SET('PG5', WC_PostPageComplete(WC_GetLastActionUrl(), GET('PS2'), GET('URL')));
EXIT('Post successfully');
LOG(GET('PG5'));
,EXIT('Invalid Login'));Basically the script is a C like script, where Function1(function2(function3), function2b()); etc like except that you can create array of string using
Code:
MyFunc('first string';'second string', 'third string')Back to above script, there interesting part is : WC_GetPostStringRaw
Which will get the POST string format "username=myid&password=mypass&securitytoken=AutoGetFromForm" and you just need to pass in myid and mypass, and the securitytoken etc and all other input inside the form index '1' of that page will be created, and you only pass what interest you to fill in, like the username, and password.
{PARAM0}, {PARAM1} and {PARAM2} is username, password and password again. The script detect {PARAM and whenever it detects it, it will replace with the script argument files. you run the script using
Code:
scrappermin.exe -scr=HWZ.txt -args=param.txtwhere param.txt is
Code:
[param]
userid
[param]
password
[param]
password
[param]
hxxp://threadurltobump
[param]
message of bumpOf course you need to register in the site before actually able to run the script successfully.
After having the POST string, you just call WC_PostPageComplete, to do POST using your final POST string where you have filled in the required fill such as username and password. And it will return the webpage and we check for string "Thank you for logging", if exist basically we have successfully log into the site.
What we do next is to find the thread ID by first getting the page content into PG3, and scrape the page ID that will be in the end of this string "newreply&noquote=1&p=". We use FILTER to get the first (index 0) of the occurence, and leave the rest. Then concatenante with the forum reply url using JOIN.
Finally we get the page to do posting of the forum. and using WC_GetPostStringRaw to get the form at index 3 which is the form to fill in for posting comment, we fill in "message" value pair with our input at {PARAM4} and remove the preview key pair and do a post.
And we shall see our comment posted to the forum.
ScrapperMin.exe can be scheduled in Task Scheduler and the parameter must be put correctly, then periodically it will bump your thread.
You can create your own script using ScrapperMinW.exe (windows form) and test run it, once successful you can write your own script to do Web Automation task of any website you need.
Feel free to post here to discuss.
Last edited:
Some more of active integration with a JS runtime engine may very well be necessary.
