Short: Simplify mass WWW download AWEB3/DOPUS5 Author: fisherking@softhome.net (fisherking) Uploader: fisherking softhome net (fisherking) Type: comm/www Architecture: m68k-amigaos K I N G L E E C H H O G V 1 . 0b ® F I S H E R K i N G 2 0 0 0 email : fisherking@softhome.net Short : KingLeechHog is a util/script/rexx for managing mass downloads/leech using Dopus5.x, Aweb 3 and Httpresume. Dont bother using this for 4-5 files, its purpose is to hold a que for aweb3/httpresume and feed files/url's until end. I've had this util/script handle and download 1000's of files in one go and then disconnect from internet in the end. Go rent a movie, goto bed, whatever.... KingLeechHog is freeware, use at own risk, bla,bla....u know the drill.. Features : Uses Dopus5.x for 'gui'. All files/actions made are/can be controlled via dopus button bank (included ofcoz). Selection of files/filters/toggles/suffix/prefix uses dopus functions. Files listed in a own Dopus lister, with own toolbar. Having a destination lister will download files to dest (req. btw) By parsing to Httpresume, authorization/resume/timeout probs disapear. Good handling/solving of HTML page indexes, KingLeechHog will try extracting all HREF pointers, and if one failed it tryes another approatch. shittystuff : Needs a clitool (wrap) for wordwrapping lines with more that 1000 chars coz arexx cant handle reading from a files with more than 1000 chars in one line. Wrap tool included (Found on aminet) Should be in KingLeechHog: Install : 1. Make a KingLeechHog directory somewhere and assign KingLeechHog: there ex. assign KingLeechHog: DH1:Programs/Internet/KingLeechHog 2. Copy/extract all files to KingLeechHog: 3. Edit 'httpresume.rx' and change path to where you have httpresume exefile 4. Think thats it.....you can dblclick on KingLeech_Hog file (its a Dopus buttonbank) bottons pop up, and the you have control...pretty much.. ***Now, with Aweb running and page with links loaded, switch to dopus screen and click 'Rescan' button.....this will start main arexx script, and if not that html page contains 100's of links/files a new dopus lister should pop up, displaying files within a few secs. Now you can select what files to download and select download with aweb or httpresume. So in a surf session, you can browse around and when you find files you want, click rescan button, urls will pop up in a lister, and then browse around more, when you have files enough or dont find more files, start the download and goto bed (disconnect ONLY works with resume button and amitcp/genesis) Downloads with aweb will leave modem online when done, if you dont calculate approx. online time needed and use a timer (c:wait) and a reboot tool to reboot after timer example download que in aweb of 30 mb and 64k isdn line aprox 6000 chars per second 30.000.000 / 6000 = 5000 secs 5000 secs / 60 secs a minute = 84 mins c:wait 90 mins c:reboot usally works, sometimes not :) havent checked aweb docs for a cmd who checks for downloads in progress Buttons : Rescan - look behind ***(above) in so lazy writin.... Getfiles - will parse all selected files to aweb for download files will be saved in your destination dopus lister No destination lister selected a msg will pop up. Resume - Will parse files to httpresume, one by one Good for bigger files (500k and up) Also a requester will ask if you want to disconnect modem from internet after download. A second requester asking if files require authorization and if you select yes, a 3'rd requester prompts for username/password in u:p format (if username=amiga & password=rulez then enter amiga:rulez) For disconnection after download you must use Amitcp/Genesis as tcp stack (script executes amitcp:bin/stopnet) with another tcp stack, you gotta edit 'httpresume.rx' script and replace address command('execute amitcp:bin/stopnet') with whatever command your tcp stack need, or just email me. fisherking@softhome.net Destination for files are your destination lister !REQUIRED! @webgo - will parse the first selected file in the KingLeechHog lister into aweb who loads it as if you entered file as url. dunno why someone should find this useful, guess i make script for test purpose maybe remove for a future update of KingLeechHog.....?! Addlog - Saves URL of selected files into a logfile (KingLeech_Hog:log/Leech_Hog.log) Appends to file if earlyer log exists. you must delete this file manually if you dont want log to grow with files you maybe already have downloaded. Useful if you find some files, but wanna download later without loading page into aweb again and then exeute main KingLeech_Hog script. Loadlog - Loads entire logfile into your current KingLeech_Hog lister, you can then download these.... buttons have a rightbutton feature aswell..white text on black background bottom of buttons. rescan button rightclick (Toggle Selected) toggle selected/unselected files in KingLeech_Hog lister getfiles rightclick (Select by #?) Select files from wildcard (#?.lha) send mailto: disabled, n/a netstat disabled, n/a Hint/tips : problem Its not always KingLeechHog find links/files in html pages using multiple frames but in aweb links/files show. solve i've had this shit a few times, usally its a small menu frame in left of browser with links to pages who loads into the 'main' frame in right of browser. Best solved by holding mouse over link of a page and rightclick mouse and load link into new aweb window, and then restart/rescan with KingLeechHog script Pages with LOTS of banners and links to firms or buy this/that you should NOT use this util for that coz you'll get most of the files/gif's/logos into dopus lister , and with lot of lines this script slows down massive much...and lister will appear after 5-10-15 secs..... I've been working on this thing since sometime in 1998....I used Ahmatti (same kinda script for aweb) for a while but wanted more control/options and a better html scan engine. this is the result.. Been using it ALOT and its really useful for me, so why not release it for others to enjoy. Button images are ugly i know...made in a hurry....draw your own or use/make textbuttons if you draw some, send me :)