Compare 2 file lists with each other?

[Biff]

Is there a way to automatically compare 2 file lists with each other?

Or a file list with a folder on a hard drive or the entire drive?

[NotNull]

Not really possible. Under certain conditions there are options. All depends on what "comparing" actually has to compare and what the result of this comparison should be.

Can you give an example (/multiple examples) on how you intend to use it?

[Biff]

Just a compare to check if the files / folders on the drive are in the list, respectively vice versa.

Or a compare of the sizes / creating / modified date of files / folders.

[therube]

distinct: (I think it is) could find files that exist in the filelist but not... or would it?
Hmm... not sure now?

Might also depend on just how you open the file list; i.e., File | Open File List vs. using -file-list switch?

[Biff]

Sorry, I understand nothing.

IS there anything I should try / do?

[void]

Is there a way to automatically compare 2 file lists with each other?

Not yet, sorry.

[Biff]

OK, many thanks!

[Biff]

Under certain conditions there are options.

What would these conditions be?

[NotNull]

Comparing 2 nearly identical folder structures (for example a folder and it's backup) and show files that are different on each side (based on name, size and/or something else that Everything has knowledge of)

[Biff]

Sounds like what I wanted. How would I do such?

Why does this only seem to work with two nearly identical folder structures?

[NotNull]

Sounds like what I wanted.

Why didn't you say so right away

• Create the following filter:
  • Name = Difference between 2 folders (or whatever you like to name it)
  • Search =

    Code: Select all

    #if:<#stricmp:<#element:<$param:,;,1>,#element:<$param:,;,2>>==0,"!">       file:   regex:[regex-escape:[element:$param:,;,1]](.*$) | regex:[regex-escape:[element:$param:,;,2]](.*$)    unique:regmatch1;size;dm
    

  • Macro = diff
• Back in the main window, set the filter to Everything
• Search for:

  Code: Select all

  diff:"X:\first folder\";"Y:\second folder\"

The resultlist will show which files are either not on one of the 'sides' or have a different name, date modified or size

(The current version of this filter was built using information and ideas from the posts below)

[Biff]

Why didn't you say so right away

Well, I guess because of difficulties with the brain and other impairments.



Wait...I am not quite sure...but these two folders are not folders from online drives, are they? Although...

[NotNull]

hese two folders are not folders from online drives, are they?

They are just placeholder foldernames; replace them with your actual folders, like c:\photos and d:\backup\photos.

[Biff]

Yes, I understand. Sorry for my bad expression: I meant are they just placeholders for foldernames which are on one or more onlilne drives? Or are they just placeholders for foldernames that are in a list in Everything?

Is this a comparison between a file list and a folder on an online drive or another file list? Or is this a comparison between two folders that are on drives being online at the moment?

[NotNull]

This is a comparison of 2 folders that are in Everything's index.
So if you type "X:\first folder\" in Everything's search bar, a list of files/folders will be shown. Same for "Y:\second folder\"

If both show a list of files/folders, you can use diff: to compare them.

[Biff]

Cannot get it working, where is the mistake:



I: is online. Y: is offline.

[NotNull]

Back in the main window:
- Set the filter to Everything

(btw: If both folders are the same, no files will be shown)

[Biff]

Back in the main window:
- Set the filter to Everything

Sorry, yes, nothing shown in the result list. What might be wrong?

So one has to execute the steps above in exactly that order?

When I click the filter, "Querying" is shown left on the status bar. When I enter while querying or after from the clipboard the search code / phrase nothing is shown in the result list.

(btw: If both folders are the same, no files will be shown)

Yes, I had already thought that. But they are not identical. So there should be shown something.

[tuska]

2NotNull

I am not getting any results either.
Please check.
 

2024-02-07_Compare 2 file lists with each other.png (43.85 KiB) Viewed 2433 times

[NotNull]

Will take a look, but not before friday. Probably the unique: function. Will try to figure out how that actually works. Luckily there's a workaround anyway.

If I do not reply this weekend, I forgot about this thread. Please bump it in that case, because this is interesting (and not only for myself)

[reitwal]

If I remove the double quotes and the trailing backslash from the paths it works for me.
Of course, this is a problem if the paths contain spaces.

[NotNull]

If I do not reply this weekend, I forgot about this thread. Please bump it in that case, because this is interesting (and not only for myself)

And indeed, I did forget ...

Updated the original search query above.

@everyone: Thanks for testing and feedback!!

[tuska]

2NotNull

Here is my negative test result.

Filter
- Search: file: regex:#regex-escape:<#element:<$param:,;,1>>(.*$) | regex:#regex-escape:<#element:<$param:,;,2>>(*$) unique:regmatch1;size;dm
- Macro: diff

Search query:
- diff:"D:\totalcmd\Test\";"C:\totalcmd\Test\"

I used this test scenario for another topic and uploaded test files to OneDrive.
These may be useful for your own tests...
 

2024-03-01_Compare 2 file lists with each other.png (81.9 KiB) Viewed 1948 times

[NotNull]

uploaded test files to OneDrive.

Which one of these EFU files or zip files do I need for testing?

(I have a script that converts the content of an EFU file to actual files, including dates so that will work too)

[tuska]

uploaded test files to OneDrive.

Which one of these EFU files or zip files do I need for testing?

Left TC window:   2024-02-16_16.47.40_COMPARISON BASE-2_Own-files_(left).zip
Right TC window:  2024-02-16_16.47.46_COMPARISON BASE-2_SharePoint_(right).zip

[NotNull]

Thanks!

That clarified the issue right away (I cannot type )

While editing I deleted a dot ("."), which has a large effect in regular expressions.
With this change your different files are reported as expected here.


Edited the original answer (still manually, fingers crossed ..)

[tuska]

2NotNull
That looks very good now!

Filter
Name:

Code: Select all

Makro: diff | diff:"X:\first folder\";"Y:\second folder\"  | Difference between 2 folders | https://www.voidtools.com/forum/viewtopic.php?p=64637#p64637 Compare 2 file lists with each other?

Macro: diff
Search:

Code: Select all

file:   regex:#regex-escape:<#element:<$param:,;,1>>(.*$) | regex:#regex-escape:<#element:<$param:,;,2>>(.*$)    unique:regmatch1;size;dm

 

2024-03-02_Filter_Difference between 2 folders.png (20.34 KiB) Viewed 1925 times

 

2024-03-02_Compare 2 file lists with each other.png (88.29 KiB) Viewed 1925 times

 
There is only one small "flaw": If the paths to be compared are identical, Everything displays search results.
Expected result ................: The search result remains empty.
Example - Search query ......: diff:"C:\totalcmd\Test\";"C:\totalcmd\Test\"

[void]

Include the following at the start of the filter to show no results when both terms are the same (case insensitive):

#if:<#stricmp:<#element:<$param:,;,1>,#element:<$param:,;,2>>==0,"!">

Code: Select all

#if:<#stricmp:<#element:<$param:,;,1>,#element:<$param:,;,2>>==0,"!">   file:   regex:#regex-escape:<#element:<$param:,;,1>>(.*$) | regex:#regex-escape:<#element:<$param:,;,2>>(.*$)    unique:regmatch1;size;dm

[NotNull]

Code: Select all

"!"

Clever!

[tuska]

2void
Thank you for your support on this issue, the complementary solution works perfectly!

2NotNull
Thanks again for the original solution!

This means that a comparison of two file lists is possible with the final code supplemented and was published here.

Here are a pictures of the final test:
 

2024-03-25_Test case - compare two identical file structures - search result remains empty.png (155.19 KiB) Viewed 1559 times

[Biff]

Sorry, so what do I have to do now to compare the content of a folder of an online drive with an index in Everything or to compare to offline folders against each other?

[tuska]

1. Create this filter in Everything
   1. Name: (Only example!)

      Code: Select all

      Makro: diff | Search query: diff:"X:\first folder\";"Y:\second folder\"  | Difference between 2 folders | https://www.voidtools.com/forum/viewtopic.php?p=65979#p65979 Compare 2 file lists with each other?

   2. Search: (Code from here)

      Code: Select all

      #if:<#stricmp:<#element:<$param:,;,1>,#element:<$param:,;,2>>==0,"!">   file:   regex:#regex-escape:<#element:<$param:,;,1>>(.*$) | regex:#regex-escape:<#element:<$param:,;,2>>(.*$)    unique:regmatch1;size;dm

   3. Macro:

      Code: Select all

      diff

2. Execute this search query in Everything, e.g. currently 1.5.0.1371a (x64) -> IMPORTANT: Change the example paths to the real paths!...

   Code: Select all

   diff:"X:\first folder\";"Y:\second folder\"

   1. The "Everything" filter must be set as the filter for the search query(!)

EDIT:
Added: The filter "Everything" must be used for the search query(!).

[Biff]

Thank you very much!

Very sorry, I cannot find the mistake I make:





What is wrong?

[NotNull]

- Set the filter to Everything

[Biff]

Now this is shown:



So one sets a different filter than the one made to compare? Why is that? Whatfor is this filter:
Makro: diff | Search query: diff:"X:\first folder\";"Y:\second folder\" | Difference between 2 folders | viewtopic.php?p=65979#p65979 Compare 2 file lists with each other?

[NotNull]

There are 2 common ways to activate a Filter:
- By selecting it from a list (Menu => Search or the Filter bar)
- using the its macro (if available)

Take for example the Picture filter: in the filter definition, in the MAcro field, it says: pic.
That means that typing pic: in the search field will give the same results as selecting the Picture filter from the list.

With some simple filters, like Picture, you can use both ways at the same time and still get the same results.
That is not the case with this complex filter: you need to choose either one of them.
I like using the macro as that gives more flexibility and is faster (typing pic: vs. selecting from a list).
(wiil describe the flexibility later, but let's keep it as simple as possible at this stage)


I would like to suggest to change the name of the filter to something easier: "Compare 2 folders: find differences".
Also: Make sure no other filter is using the diff: macro.

[Biff]

Ah, so that's what these macros are for obviously. OK, thank you.

Great, now it works:


But how do I know what files are equal, unique, different? Files not shown are equal (by date AND size, not by date OR size?)? Files shown are unique or different? And files shown with the same name could be different?

[NotNull]

This filter compares 2 folder structures. If one or more of the following is not exactly the same "on both sides", the file will reported in this list:
- relative path of the file inside the folder (i.e I:\1\folder\file.txt has relative path \folder\file.txt)
- filename (including extension)
- size of the file
- Date Modified of the file.


Files that are NOT shown in the list will have the exact same relative path AND filename AND size AND Date Modified "on both sides" and will be regarded as equal.

[Biff]

OK, I understand, thank you very much!

[NotNull]

You're welcome!

(wiil describe the flexibility later, but let's keep it as simple as possible at this stage)

Coming back to this:
Using the diff: macro instead of selecting the filter from a list gives extra options.

For example: I have two folders containing music files, but also their related lyrics text files and album covers. When only interested in the music files, I can set the filter to Audio by selecting it from the list and serach for diff:"C:\folder1\";C:\folder2\"
This will ignore all non-audio ffiles and do a comparison of just the audio files.

Or, after setting the filter to Everything, searching for ext:docx diff:"C:\folder1\";C:\folder2\" to compare only .docx files.

Not needed so far, but I like the flexibility

[Biff]

Ah, yes, indeed, that is very useful.

For example: I have two folders containing music files, but also their related lyrics text files and album covers. When only interested in the music files, I can set the filter to Audio by selecting it from the list and serach for diff:"C:\folder1\";C:\folder2\"

So you could e.g. find folders with same content by audio files only, I guess. Or leaving out 0 byte files.

Or showing what odt / docx files have changed since the last back up.

Many thanks!

[NotNull]

Note:
The search syntax provided above is probably not compatible with Everything 1.5.0.1372a due to the introduction of &PARAMSTART: and &PARAMEND:
A fix is available.

Will look at it tomorrow and update the search queries.
This message will be removed if it still work as before.

[Biff]

OK, thank you very much!

Is there a way to get notifications about updates?

[NotNull]

Is there a way to get notifications about updates?

Updates of what? If you mean this topic, you are automatically "subscribed" to updates (= new messages) when you post a message.

To subscribe to topics in which you didn't post:
At the bottom of the topic is a little wrench (?) icon. When clicked, shows a Subscribe option:

Subscribe.png (16.84 KiB) Viewed 454 times

[Biff]

Thank you for the screenshot.

Very sorry, I meant updates for Everything.

[NotNull]

Thank you for the screenshot.

Very sorry, I meant updates for Everything.

Can't answer that. I visit these forums way too often to need notifications
Someone else will likely have an answer for you.

This link shows the most recent alpha version.

[Biff]

Oops, OK, so maybe that's the punishment for not being here often enough.

The update notification of Everything, the program, does not seem to work with me.

OK, many thanks for the link! Then I'd rather come by here more often.

[NotNull]

Oops, OK, so maybe that's the punishment for not being here often enough.
OK, many thanks for the link! Then I'd rather come by here more often.

Sorry, it was not my intention to "force" you to visit the forums more often.At all.
(but you are very welcome, of course)

If I were you, I would wait on what others have to say on the matter.

[Biff]

Ah, no no, a little coercion can't hurt (If I had a program like that, I would force the users to do it too...and read 4 pages at least).
Ah, thank you!

OK, yes, I will wait. Thank you!

[tuska]

Is there a way to get notifications about updates?

Menu "Help" > Check for Updates...
but to be on the safe side I would still visit the Everything Forum often and keep an eye out for updates.