Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
I have a big list of file links for download, and I already downloaded part of the list in the past. I don't want to download again what is already exist in my computer. So I need to compare the filenames in the computer to the filenames at the end of the links, and after that to omit from the links list the duplicates.
The list of file links look like this (txt file):
http:// ... /000001.gif
http:// ... /000002.gif
http:// ... /abcdef.zip
Trying to get the filenames of the computer by Everything wasn't so easy for me.
Ctrl+Shift+C is getting FullPath, and not the Filenames.
So I used export to csv to get it.
BTW. export to csv doesn't give you to export selected files in Results. Also, there are many columns with values that doesn't have meaning for the simple user. The simple user need the values in the same Format as in Everything. Also, there is a problem with displaying unicode characters in Excel by this method.
Anyway, now that I have the both lists, links list and computer filenames list, It's difficult to me to make this comparison and get the new links list. I need help here. Thank You.
The list of file links look like this (txt file):
http:// ... /000001.gif
http:// ... /000002.gif
http:// ... /abcdef.zip
Trying to get the filenames of the computer by Everything wasn't so easy for me.
Ctrl+Shift+C is getting FullPath, and not the Filenames.
So I used export to csv to get it.
BTW. export to csv doesn't give you to export selected files in Results. Also, there are many columns with values that doesn't have meaning for the simple user. The simple user need the values in the same Format as in Everything. Also, there is a problem with displaying unicode characters in Excel by this method.
Anyway, now that I have the both lists, links list and computer filenames list, It's difficult to me to make this comparison and get the new links list. I need help here. Thank You.
Last edited by Stamimail on Wed Feb 22, 2017 10:16 am, edited 1 time in total.
Re: Help with comparing files
I'm not quite following & you may need to massage your data first, but UNIX has a uniq (unique) command.
(Ah, it's been so long, I'm not even remembering how to use it!)
Maybe its something like this...
have:
havenot:
& that should show you only the files that you do not already have, four, in this instance.
(Ah, it's been so long, I'm not even remembering how to use it!)
Code: Select all
Usage: uniq [OPTION]... [INPUT [OUTPUT]]
Discard all but one of successive identical lines from INPUT (or
standard input), writing to OUTPUT (or standard output).
Mandatory arguments to long options are mandatory for short options too.
-c, --count prefix lines by the number of occurrences
-d, --repeated only print duplicate lines
-D, --all-repeated[=delimit-method] print all duplicate lines
delimit-method={none(default),prepend,separate}
Delimiting is done with blank lines.
-f, --skip-fields=N avoid comparing the first N fields
-i, --ignore-case ignore differences in case when comparing
-s, --skip-chars=N avoid comparing the first N characters
-u, --unique only print unique lines
-w, --check-chars=N compare no more than N characters in lines
--help display this help and exit
--version output version information and exit
A field is a run of whitespace, then non-whitespace characters.
Fields are skipped before chars.
have:
Code: Select all
one
two
three
Code: Select all
one
two
three
four
Code: Select all
cat.exe have havenot > combined
sort.exe combined
uniq -d combined
Re: Help with comparing files
Please try comparing the downloaded list and the todownload list in Everything by using the !filelist: search:
The values will appear almost exactly as they do in Everything.
Size is formatted without commas.
Dates are formated with ISO8601: YYYY-MM-DD HH:MM:SS.
Excel should have no issue parsing these values.
To correctly open UTF-8 CSV files in Excel:
- In Everything, search for your current downloaded copy of the files, for example, search for:
d:\downloads\ - From the File menu, click Export....
- Change save as type to EFU Everything file list.
- Type in downloaded.efu and click Save.
- Convert your "to download" list to a list of filenames as they would appear when downloaded.
- You should a have a full path and filename on each line, for example:
Filename
d:\downloads\000001.gif
d:\downloads\000002.gif
d:\downloads\abcdef.zip - At the top of the list add a header with the word Filename (as above)
- Save the list as todownload.efu.
- Open this file in Everything.
- In Everything, from the Search menu, click Advanced Search....
- Down the bottom of the search window, to the right of Search for a list of filenames, click Browse....
- Select your downloaded.efu and click OK.
- Click OK.
- You might not be able to see the text in the text box, however there is text there.
- In the Search box, press the home key to move the caret to the start of the edit box.
- Type in a single !
- Everything should now list all the files in todownload.efu that are not in downloaded.efu.
- Now it's just a matter of converting the filenames back to their url. Which should be easy enough after exporting to a text file.
Exporting will always export all the current results, never the selection.BTW. export to csv doesn't give you to export selected files in Results.
Please make sure you export as CSV.Also, there are many columns with values that doesn't have meaning for the simple user. The simple user need the values in the same Format as in Everything.
The values will appear almost exactly as they do in Everything.
Size is formatted without commas.
Dates are formated with ISO8601: YYYY-MM-DD HH:MM:SS.
Excel should have no issue parsing these values.
Everything exports CSV as UTF-8.Also, there is a problem with displaying unicode characters in Excel by this method.
To correctly open UTF-8 CSV files in Excel:
- In Excel, from the Data menu, from the Import External Data submenu, click Import data....
- Select your exported csv and click Open.
- Change File origin to 65001 : Unicode (UTF-8).
- Click Next.
- Check Comma.
- Click Next.
- Click Finish.
- Click OK to put the data in the existing worksheet at $A$1.
Re: Help with comparing files
Stamimail
I wrote AHK script for your task and I need samples of both your files for debugging.
I wrote AHK script for your task and I need samples of both your files for debugging.
Re: Help with comparing files
From Options - Context Menu - choose "Copy Name to Clipboard" and check "show this item" then set any shortcutStamimail wrote:Trying to get the filenames of the computer by Everything wasn't so easy for me.
Ctrl+Shift+C is getting FullPath, and not the Filenames.
or
from Menu - File - choose "Copy Name to Clipboard" (must be checked one file or more) to work.
Re: Help with comparing files
Sample of todownload:ovg wrote:Stamimail
I wrote AHK script for your task and I need samples of both your files for debugging.
Code: Select all
http://cloud.addictivetips.com/wp-content/uploads/2013/04/Everything-Search-Engine.png
https://screenshots.en.sftcdn.net/en/scrn/74000/74327/everything-21.jpg
https://www.pendriveapps.com/wp-content/uploads/everything-file-search-tool.jpg
http://www.majorgeeks.com/index.php?ct
http://cloud.addictivetips.com/wp-content/uploads/2009/12/EverythingDesktopSearch.jpg
https://screenshots.en.sftcdn.net/en/scrn/74000/74327/everything-19.jpg
http://i.i.cbsi.com/cnwk.1d/i/tim/2012/09/05/Foreman_12696521_8411_ddd.jpg
http://www.reyniersaudio.com/blog/wp-content/uploads/2011/03/Search-Everything.png
http://img.brothersoft.com/screenshots/softimage/e/everything_search_engine-229653-1239257527.jpeg
http://cdn.nirmaltv.com/images/Search-Everything.png
http://www.refolder.com/wp-content/uploads/2009/02/ese-01.png
https://i.kinja-img.com/gawker-media/image/upload/s--Ed6j4W0N--/c_fit,fl_progressive,q_80,w_636/18rg09rdob9rwjpg.jpg
http://images.pcworld.com/reviews/graphics/164021-every-big_350.jpg
https://bd23.https.cdn.softlayer.net/80BD23/142.4.51.106/blog/wp-content/uploads/2011/03/everything_search.png
https://screenshots.en.sftcdn.net/en/scrn/74000/74327/everything-23.jpg
http://cloud.addictivetips.com/wp-content/uploads/2013/04/Everything-Search-Engine_Filters.png
http://www.refolder.com/wp-content/uploads/2009/02/ese-02.png
http://img.creativemark.co.uk/uploads/images/580/15580/img3File.png
https://i.ytimg.com/vi/Ykq7X6_wmGY/hqdefault.jpg
https://lookeen.com/blog/wp-content/uploads/2015/04/Lookeen-Free-Desktop-Screenshot_EN.png
https://screenshots.en.sftcdn.net/en/scrn/74000/74327/everything-8.jpg
http://cloud.addictivetips.com/wp-content/uploads/2013/04/Everything-Search-Engine_FTP.png
http://www.technosamrat.com/wp-content/uploads/2008/09/locate-your-files-easily-with-everything-search-engine-for-windows.jpg
http://blogote.com/wp-content/uploads/2011/02/everything_software_windows_screenshot_022.jpg
http://www.technixupdate.com/wp-content/uploads/images/SearchinsidethefilecontentwithEverything_13BB0/everythingsearch.jpg
http://4.bp.blogspot.com/_SIq17kU32GU/S-3cVn3iOkI/AAAAAAAAATM/toj5ItyExjU/s1600/Everything.png
https://ttcshelbyville.files.wordpress.com/2015/02/1st-screen.jpg
https://www.voidtools.com/Everything.Search.Window.png
http://cdn.gtricks.com/2011/09/Everything_Search.jpg
http://www.majorgeeks.com/index.php?ct
http://cloud.addictivetips.com/wp-content/uploads/2013/04/Everything-File-List-Editor.png
http://i1-news.softpedia-static.com/images/news2/everything-review-488745-2.jpg
http://img.technospot.net/everything-file-indexer.png
http://cdn.makeuseof.com/wp-content/uploads/2012/07/10-Everything.jpg
https://i.ytimg.com/vi/cb3MfuRnRms/maxresdefault.jpg
http://cdn.ghacks.net/wp-content/uploads/2013/06/everything-search.png
http://www.voidtools.com/images/search.mount.video.example.png
Code: Select all
C:\Everything\Everything-Search-Engine_FTP.png
C:\Everything\locate-your-files-easily-with-everything-search-engine-for-windows.jpg
C:\Everything\everything_software_windows_screenshot_022.jpg
T:\Everything\everything-file-indexer.png
D:\Users\שם משתמש\Desktop\Everything-1.4.0.713b.x64\Search-Everything.png
D:\Users\שם משתמש\Desktop\תוכנות\Everything\686 - תרגום Everything\ese-01.png
D:\Users\שם משתמש\Desktop\תוכנות\Everything\686 - תרגום Everything\makelng\everything-8.jpg
T:\Everything\עברית - 713\Foreman_12696521_8411_ddd.jpg
D:\Users\שם משתמש\Desktop\תוכנות\Everything\801 - תרגום חדש\Template\Search-Everything.png
D:\Users\שם משתמש\Desktop\תוכנות\Everything\801 - תרגום חדש\Template\תיקיה חדשה\everything_search_engine-229653-1239257527.jpeg
D:\Users\שם משתמש\Desktop\תוכנות\Everything\818\everything-review-488745-2.jpg
I had forgotten from that.
Another way to get it is by using F2 of renaming.
Re: Help with comparing files
2Stamimail
Try script. https://yadi.sk/d/rxmEGCUD3EK2Sn
Syntax: ToDownload List_of_Links List_of_Existing_Files. List of existing files should be in UTF-8 encoding.
Output: ToDownload.txt - List of Files to Download
Try script. https://yadi.sk/d/rxmEGCUD3EK2Sn
Syntax: ToDownload List_of_Links List_of_Existing_Files. List of existing files should be in UTF-8 encoding.
Output: ToDownload.txt - List of Files to Download
Re: Help with comparing files
@ovg
Almost done.
I've tested the script ToDownload.exe on the 2 lists above.
There was one miss:The file Everything.png is not listed in the List_of_Existing_Files, and this link to download Everything.png was omitted from the Output.
It will be nice to paste here the code for future tweaks.
Almost done.
I've tested the script ToDownload.exe on the 2 lists above.
There was one miss:
Code: Select all
http://4.bp.blogspot.com/_SIq17kU32GU/S-3cVn3iOkI/AAAAAAAAATM/toj5ItyExjU/s1600/Everything.png
It will be nice to paste here the code for future tweaks.
Re: Help with comparing files
2Stamimail
Source is in attachment.
PS. New (optimized) version 23-03-2017
Source is in attachment.
PS. New (optimized) version 23-03-2017
- Attachments
-
- ToDownload.zip
- (490 Bytes) Downloaded 633 times
Last edited by ovg on Thu Mar 23, 2017 10:42 am, edited 1 time in total.
Re: Help with comparing files
Thanks. I finished with my task.
Got the same results by ovg tool, and void approach but with syntax:
!"filename1.ext" !"filename2.ext" !"file-name-3.gif" !"file name_4.jpg" ... (Match WholeWord)
@ovg
Do you want to make a new thread about this tool in AutoHotkey Forum?
A tool to avoid downloading files you're already downloaded.
I want to subscribe to this thread till this tool will have a comfortable GUI. (Dialog box with Path & Browse to (the lists) txt files, and 2 TextBoxes for directly paste the lists to the dialog. Cancel & Process button to make the Output.txt)
Got the same results by ovg tool, and void approach but with syntax:
!"filename1.ext" !"filename2.ext" !"file-name-3.gif" !"file name_4.jpg" ... (Match WholeWord)
@ovg
Do you want to make a new thread about this tool in AutoHotkey Forum?
A tool to avoid downloading files you're already downloaded.
I want to subscribe to this thread till this tool will have a comfortable GUI. (Dialog box with Path & Browse to (the lists) txt files, and 2 TextBoxes for directly paste the lists to the dialog. Cancel & Process button to make the Output.txt)
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
@void
Is it possible to treat web links in seacrh box as looking for a file?
I mean, for example while enter in:
https://i.kinja-img.com/gawker-media/im ... 9rwjpg.jpg
Everything actually will search:
18rg09rdob9rwjpg.jpg
Is there any other use for web links in SearchBox?
As for multiple search list, maybe it can be made somehow by Search History, so that the user can search each item in the list one by one.
Is it possible to treat web links in seacrh box as looking for a file?
I mean, for example while enter in:
https://i.kinja-img.com/gawker-media/im ... 9rwjpg.jpg
Everything actually will search:
18rg09rdob9rwjpg.jpg
Is there any other use for web links in SearchBox?
As for multiple search list, maybe it can be made somehow by Search History, so that the user can search each item in the list one by one.
Last edited by Stamimail on Mon Mar 20, 2017 6:01 pm, edited 1 time in total.
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
Sorry for delayed reply, I've missed your post somehow ....Stamimail wrote:Do you want to make a new thread about this tool in AutoHotkey Forum?
Honestly I don't want to do this ... but if you want - you can DIY - I will not mind. You can freely use script.
In addition, I prefer cli utilities - they are very easy to integrate with Total Commander. Sorry ...
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
@ovg
No problem, and thanks for the script, it was very useful.
No problem, and thanks for the script, it was very useful.
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
2Stamimail
Glad to help you!
Glad to help you!
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
So you're saying the "file name" of a currently opened web page (URL)?Is it possible to treat web links in seacrh box as looking for a file?
So if you open the page (the URL), https://i.kinja-img.com/gawker-media/im ... 9rwjpg.jpg
You want to be able to search Everything for, "18rg09rdob9rwjpg.jpg".
Well that's basically what how to add everything as a search engine in firefox?
does, only that does a search for highlighted text on a web page, rather then the page URL itself.
I would think it could be modified to get the "basename" of the page URL?
Do you know JavaScript? (Me neither.)
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
Try this:
If its working, the alert() can come out.
https://www.squarefree.com/bookmarklets/
http://befused.com/javascript/get-filename-url
http://stackoverflow.com/questions/3820 ... javascript
Well its not going to be "exactly" right, as it fails on a URI that ends with a /, so www.example.com/test/ would not return "test", but www.example.com/test (without the trailing slash) works. (And I'm sure there are other places it will fail too
.)
Code: Select all
javascript:var%20url%20=%20window.location.pathname;%20var%20Qr%20=%20url.substring(url.lastIndexOf('/')+1);%20%20alert(Qr);if(Qr)location.href='es:'+(Qr);void(0)
Code: Select all
javascript:var%20url%20=%20window.location.pathname;%20var%20Qr%20=%20url.substring(url.lastIndexOf('/')+1);if(Qr)location.href='es:'+(Qr);void(0)
https://www.squarefree.com/bookmarklets/
http://befused.com/javascript/get-filename-url
http://stackoverflow.com/questions/3820 ... javascript
Well its not going to be "exactly" right, as it fails on a URI that ends with a /, so www.example.com/test/ would not return "test", but www.example.com/test (without the trailing slash) works. (And I'm sure there are other places it will fail too
![Wink ;-)](./images/smilies/icon_e_wink.gif)
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
2Stamimail
Try new version of script with a choice of files via standard Open/Save dialogs .... May be it will be helpful to you.
When start script without parameters, Open/Save dialogs appears. But you can still use command line parameters.
https://yadi.sk/d/wx4ilVeb3GGn9S
Try new version of script with a choice of files via standard Open/Save dialogs .... May be it will be helpful to you.
When start script without parameters, Open/Save dialogs appears. But you can still use command line parameters.
https://yadi.sk/d/wx4ilVeb3GGn9S
Last edited by ovg on Thu Mar 23, 2017 10:37 am, edited 1 time in total.
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
@therube
I meant just to copy the full URL from the browser and paste it like it is in the Searchbox of Everything.
It's more easy to select/copy the whole URL than to select just the filename at the end of the URL.
@ovg
I tested it. Nice, now it can be done without the use of CMD.
3 Comments:
1. I think there is a bug.
The last link in "List of Links" is always displayed in Result (see the last result), despite its filename was in the "List of Downloaded Files".
2. "Save result to:" Dialog box
It saves the Results file without an extension. I think Save as txt file is expected.
3. "Save result to:" Dialog box
If the user press Cancel, it creates unwanted todownload.txt file.
I meant just to copy the full URL from the browser and paste it like it is in the Searchbox of Everything.
It's more easy to select/copy the whole URL than to select just the filename at the end of the URL.
@ovg
I tested it. Nice, now it can be done without the use of CMD.
3 Comments:
1. I think there is a bug.
The last link in "List of Links" is always displayed in Result (see the last result), despite its filename was in the "List of Downloaded Files".
2. "Save result to:" Dialog box
It saves the Results file without an extension. I think Save as txt file is expected.
3. "Save result to:" Dialog box
If the user press Cancel, it creates unwanted todownload.txt file.
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
URL: http://www.voidtools.com/forum/viewtopi ... 770#p17770
"File Name":
Result: viewtopic.php
---
"Path + File Name":
Result: /forum/viewtopic.php
---
"Window URL":
Result: http://www.voidtools.com/forum/viewtopi ... 770#p17770
![Image](http://s25.postimg.org/reqnafx17/Everything_Protocol_to_find_filename_pathnam.jpg)
"File Name":
Code: Select all
javascript:var%20url%20=%20window.location.pathname;%20var%20Qr%20=%20url.substring(url.lastIndexOf('/')+1);%20%20alert(Qr);if(Qr)location.href='es:'+(Qr);void(0)
---
"Path + File Name":
Code: Select all
javascript:Qr%20=%20window.location.pathname;%20%20alert(Qr);if(Qr)location.href='es:'+(Qr);void(0)
---
"Window URL":
Code: Select all
javascript:Qr%20=%20window.location.href;%20%20alert(Qr);if(Qr)location.href='es:'+(Qr);void(0)
![Image](http://s25.postimg.org/reqnafx17/Everything_Protocol_to_find_filename_pathnam.jpg)
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
@Stamimail
Try the new version, please. I fixed the bug with the last link, and your requests are also ready.
https://yadi.sk/d/wx4ilVeb3GGn9S
Try the new version, please. I fixed the bug with the last link, and your requests are also ready.
https://yadi.sk/d/wx4ilVeb3GGn9S
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
@ovg
I've tested it a little. Yes, your program looks ready.
Thanks a lot.![Very Happy :D](./images/smilies/icon_e_biggrin.gif)
I've tested it a little. Yes, your program looks ready.
Thanks a lot.
![Very Happy :D](./images/smilies/icon_e_biggrin.gif)
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
@Stamimail
Check do you have the latest version? 1427 bytes 23-03-17 16:12.
I change Return to ExitApp and add two .Close() statements at the very end of script.
Check do you have the latest version? 1427 bytes 23-03-17 16:12.
I change Return to ExitApp and add two .Close() statements at the very end of script.
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
@Stamimail
For fun: I've added Progress Bar to indicate scanning process.
https://yadi.sk/d/jnKUs3or3GHnNv
It seems that all![Mr. Green :mrgreen:](./images/smilies/icon_mrgreen.gif)
For fun: I've added Progress Bar to indicate scanning process.
https://yadi.sk/d/jnKUs3or3GHnNv
It seems that all
![Mr. Green :mrgreen:](./images/smilies/icon_mrgreen.gif)
Re: Help with comparing files - Solved - A tool to avoid downloading files you're already downloaded
@ovg
Thank you for informing me you updated the link.
Downloaded. It works. Thanks.
Edit: [I saw you posted before me...]
Downloaded also the progessbar version. It works. Thanks.
Have Fun.
Thank you for informing me you updated the link.
Downloaded. It works. Thanks.
Edit: [I saw you posted before me...]
Downloaded also the progessbar version. It works. Thanks.
This was the original:It seems that all
I'm trying to say there is always what to improve. but don't bother yourself, unless you see another people interested.I want to subscribe to this thread till this tool will have a comfortable GUI. (Dialog box with Path & Browse to (the lists) txt files, and 2 TextBoxes for directly paste the lists to the dialog. Cancel & Process button to make the Output.txt)
Have Fun.
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
Sometimes dreams come true[color=#0000FF][b]Stamimail[/b][/color] wrote:....this tool will have a comfortable GUI. (Dialog box with Path & Browse to (the lists) txt files, and 2 TextBoxes for directly paste the lists to the dialog. Cancel & Process button to make the Output.txt)
![Very Happy :D](./images/smilies/icon_e_biggrin.gif)
Please try new GUI version of the script.
https://yadi.sk/d/Z4Qczt6l3GL6NE. Drag'n'Drop also should working ....
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
Your solution is good for searching a page URL from within a browser.therube wrote:So you're saying the "file name" of a currently opened web page (URL)?Is it possible to treat web links in seacrh box as looking for a file?
So if you open the page (the URL), https://i.kinja-img.com/gawker-media/im ... 9rwjpg.jpg
You want to be able to search Everything for, "18rg09rdob9rwjpg.jpg".
Well that's basically what how to add everything as a search engine in firefox?
does, only that does a search for highlighted text on a web page, rather then the page URL itself.
I would think it could be modified to get the "basename" of the page URL?
Do you know JavaScript? (Me neither.)
In my case it was different. I had a list of URLs in a txt file.
There are cases it is more simple to just copy the link and put it in the Search Box.
Perhaps needs to add a new option in Options > Search Tab:
[ ] treat web links in search box as looking for a file
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
How does this CMD command work for you?
If you want to use in a CMD batchfile, replace %X with %%X
(I hope I did understand you question correctly ...)
EDIT: You could combine this with WGET. WGET is used to download files off the Internet (or download complete websites):
Instead of echo %X >> remaining,txt, you could do: WGET %X
Or - if you already have the "remaining.txt" list- : WGET --input-file=remaining.txt
WGET can be found here for example: http://gnuwin32.sourceforge.net/packages/wget.htm
EDIT2: Changed the code to remove a trailing <space> at the end of the flename.
Code: Select all
FOR /f "delims=" %X IN (2download.txt) DO @findstr /i /e "\%~nxX" downloaded.txt > nul || echo %X>> remaining.txt
(I hope I did understand you question correctly ...)
EDIT: You could combine this with WGET. WGET is used to download files off the Internet (or download complete websites):
Instead of echo %X >> remaining,txt, you could do: WGET %X
Or - if you already have the "remaining.txt" list- : WGET --input-file=remaining.txt
WGET can be found here for example: http://gnuwin32.sourceforge.net/packages/wget.htm
EDIT2: Changed the code to remove a trailing <space> at the end of the flename.
Last edited by NotNull on Mon Jun 05, 2017 10:19 am, edited 1 time in total.
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
Hi NotNull,
This topic is already solved a while ago, as mentioned.
Now I just wanted to suggest a new option and bring it into a consideration.
Nevertheless I didn't want to let you down, so I did 2 tests.
In my tests I compared the results of your script with the results of ovg's tool.
The results are the same! Bravo. Except for one thing. In your results there is a single <space> that added at the end of each link (unwanted).
Thanks for the script.
This topic is already solved a while ago, as mentioned.
Now I just wanted to suggest a new option and bring it into a consideration.
Nevertheless I didn't want to let you down, so I did 2 tests.
In my tests I compared the results of your script with the results of ovg's tool.
The results are the same! Bravo. Except for one thing. In your results there is a single <space> that added at the end of each link (unwanted).
Thanks for the script.
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
Ah, OK, didn't pick up on that one (didn't read all of the massages, just a (very) quick scan).Stamimail wrote: This topic is already solved a while ago, as mentioned.
Now I just wanted to suggest a new option and bring it into a consideration.
Then you can happily ignore my message
![Smile :-)](./images/smilies/icon_e_smile.gif)
Not that it matters anymore, but the issue withe the extra space can be solved by replacing "echo %X >>" with "echo %X>>"
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
I pretty sure your code will be useful for me or for someone some day.
I think it would be better to edit and fix your code in its place above. (I guess this someone also won't read the whole messages.
)
Thank you for posting.
I think it would be better to edit and fix your code in its place above. (I guess this someone also won't read the whole messages.
![Smile :)](./images/smilies/icon_e_smile.gif)
Thank you for posting.
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
Good thinking! Changed the code ..Stamimail wrote:I pretty sure your code will be useful for me or for someone some day.
I think it would be better to edit and fix your code in its place above. (I guess this someone also won't read the whole messages.)
Thank you for posting.
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
@ovg
Thank you very much for ToDownload_GUI.
I used it many times.
I thought about more features to add that can make it better:
1. As mentioned, a TextBox or other Control for pasting the lists from Clipboard (another option to load in addition to load list from a file)
2. Currently ToDownload works like this:
First List = get a list with URLs
Second List = get a list with Filenames (FullPaths)
Third List = Output
You can make it better by getting not only one kind of list, but like this:
First List = get a list with URLs|Filenames
Second List = get a list with URLs|Filenames
Third List = Output (with URLs|Filenames)
3. In the scanning/proccessing Dialog, at the end ,a summary how many files outputed.
Thank you very much for ToDownload_GUI.
I used it many times.
I thought about more features to add that can make it better:
1. As mentioned, a TextBox or other Control for pasting the lists from Clipboard (another option to load in addition to load list from a file)
2. Currently ToDownload works like this:
First List = get a list with URLs
Second List = get a list with Filenames (FullPaths)
Third List = Output
You can make it better by getting not only one kind of list, but like this:
First List = get a list with URLs|Filenames
Second List = get a list with URLs|Filenames
Third List = Output (with URLs|Filenames)
3. In the scanning/proccessing Dialog, at the end ,a summary how many files outputed.
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
You are welcome!Stamimail wrote:Thank you very much for ToDownload_GUI.
I 'll see what I can do here. But please keep in mind:Stamimail wrote:a TextBox or other Control for pasting the lists from Clipboard
I'm newbie with AHK and implementing this can take some time.
Sorry, I didn't understand what you mean. Could you please explain more carefully?Stamimail wrote: You can make it better by getting not only one kind of list, but like this:
First List = get a list with URLs|Filenames
Second List = get a list with URLs|Filenames
Third List = Output (with URLs|Filenames)
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
Sure,
The following example is probably not going to be the common use of the script but intended only to explain what the script should does.
What I did?
I took the 2 lists above and replaced/switched the 5-8th lines between those lists.
Sample of ToDownload:
Sample of Downloaded:
When you want to download similar content from 2 websites, and you want to avoid the same content\files to be downloaded twice. In this case you have 2 lists with only URLs lines. No filenames.)
The following example is probably not going to be the common use of the script but intended only to explain what the script should does.
What I did?
I took the 2 lists above and replaced/switched the 5-8th lines between those lists.
Sample of ToDownload:
Code: Select all
http://cloud.addictivetips.com/wp-content/uploads/2013/04/Everything-Search-Engine.png
https://screenshots.en.sftcdn.net/en/scrn/74000/74327/everything-21.jpg
https://www.pendriveapps.com/wp-content/uploads/everything-file-search-tool.jpg
http://www.majorgeeks.com/index.php?ct
D:\Users\שם משתמש\Desktop\Everything-1.4.0.713b.x64\Search-Everything.png
D:\Users\שם משתמש\Desktop\תוכנות\Everything\686 - תרגום Everything\ese-01.png
D:\Users\שם משתמש\Desktop\תוכנות\Everything\686 - תרגום Everything\makelng\everything-8.jpg
T:\Everything\עברית - 713\Foreman_12696521_8411_ddd.jpg
http://img.brothersoft.com/screenshots/softimage/e/everything_search_engine-229653-1239257527.jpeg
http://cdn.nirmaltv.com/images/Search-Everything.png
http://www.refolder.com/wp-content/uploads/2009/02/ese-01.png
https://i.kinja-img.com/gawker-media/image/upload/s--Ed6j4W0N--/c_fit,fl_progressive,q_80,w_636/18rg09rdob9rwjpg.jpg
http://images.pcworld.com/reviews/graphics/164021-every-big_350.jpg
https://bd23.https.cdn.softlayer.net/80BD23/142.4.51.106/blog/wp-content/uploads/2011/03/everything_search.png
https://screenshots.en.sftcdn.net/en/scrn/74000/74327/everything-23.jpg
http://cloud.addictivetips.com/wp-content/uploads/2013/04/Everything-Search-Engine_Filters.png
http://www.refolder.com/wp-content/uploads/2009/02/ese-02.png
http://img.creativemark.co.uk/uploads/images/580/15580/img3File.png
https://i.ytimg.com/vi/Ykq7X6_wmGY/hqdefault.jpg
https://lookeen.com/blog/wp-content/uploads/2015/04/Lookeen-Free-Desktop-Screenshot_EN.png
https://screenshots.en.sftcdn.net/en/scrn/74000/74327/everything-8.jpg
http://cloud.addictivetips.com/wp-content/uploads/2013/04/Everything-Search-Engine_FTP.png
http://www.technosamrat.com/wp-content/uploads/2008/09/locate-your-files-easily-with-everything-search-engine-for-windows.jpg
http://blogote.com/wp-content/uploads/2011/02/everything_software_windows_screenshot_022.jpg
http://www.technixupdate.com/wp-content/uploads/images/SearchinsidethefilecontentwithEverything_13BB0/everythingsearch.jpg
http://4.bp.blogspot.com/_SIq17kU32GU/S-3cVn3iOkI/AAAAAAAAATM/toj5ItyExjU/s1600/Everything.png
https://ttcshelbyville.files.wordpress.com/2015/02/1st-screen.jpg
https://www.voidtools.com/Everything.Search.Window.png
http://cdn.gtricks.com/2011/09/Everything_Search.jpg
http://www.majorgeeks.com/index.php?ct
http://cloud.addictivetips.com/wp-content/uploads/2013/04/Everything-File-List-Editor.png
http://i1-news.softpedia-static.com/images/news2/everything-review-488745-2.jpg
http://img.technospot.net/everything-file-indexer.png
http://cdn.makeuseof.com/wp-content/uploads/2012/07/10-Everything.jpg
https://i.ytimg.com/vi/cb3MfuRnRms/maxresdefault.jpg
http://cdn.ghacks.net/wp-content/uploads/2013/06/everything-search.png
http://www.voidtools.com/images/search.mount.video.example.png
Code: Select all
C:\Everything\Everything-Search-Engine_FTP.png
C:\Everything\locate-your-files-easily-with-everything-search-engine-for-windows.jpg
C:\Everything\everything_software_windows_screenshot_022.jpg
T:\Everything\everything-file-indexer.png
http://cloud.addictivetips.com/wp-content/uploads/2009/12/EverythingDesktopSearch.jpg
https://screenshots.en.sftcdn.net/en/scrn/74000/74327/everything-19.jpg
http://i.i.cbsi.com/cnwk.1d/i/tim/2012/09/05/Foreman_12696521_8411_ddd.jpg
http://www.reyniersaudio.com/blog/wp-content/uploads/2011/03/Search-Everything.png
D:\Users\שם משתמש\Desktop\תוכנות\Everything\801 - תרגום חדש\Template\Search-Everything.png
D:\Users\שם משתמש\Desktop\תוכנות\Everything\801 - תרגום חדש\Template\תיקיה חדשה\everything_search_engine-229653-1239257527.jpeg
D:\Users\שם משתמש\Desktop\תוכנות\Everything\818\everything-review-488745-2.jpg
One example for a common use of such a script is:You can make it better by getting not only one kind of list, but like this:
First List = get a list with URLs|Filenames
Second List = get a list with URLs|Filenames
Third List = Output (with URLs|Filenames)
When you want to download similar content from 2 websites, and you want to avoid the same content\files to be downloaded twice. In this case you have 2 lists with only URLs lines. No filenames.)
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
2Stamimail
Please try the new version: https://yadi.sk/d/BDPILaLf3Mzwjo
Please try the new version: https://yadi.sk/d/BDPILaLf3Mzwjo
This items are doneStamimail wrote: 2. Currently ToDownload works like this:
First List = get a list with URLs
Second List = get a list with Filenames (FullPaths)
Third List = Output
You can make it better by getting not only one kind of list, but like this:
First List = get a list with URLs|Filenames
Second List = get a list with URLs|Filenames
Third List = Output (with URLs|Filenames)
3. In the scanning/proccessing Dialog, at the end ,a summary how many files outputed.
Last edited by ovg on Tue Sep 19, 2017 10:52 am, edited 1 time in total.
Re: Help with comparing files - solved - A tool to avoid downloading files you're already downloaded
2Stamimail
New version: https://yadi.sk/d/UmU0SINO3N33Xg
Item 1 is done >
New version: https://yadi.sk/d/UmU0SINO3N33Xg
Item 1 is done >
Before submit Checkbox be shure that corresponding list is in the ClipboardStamimail wrote:pasting the lists from Clipboard