[Out of memory error]: Crashed @ 128GB & 237GB virtual memory committed

Found a bug in "Everything"? report it here
Post Reply
DerekZiemba
Posts: 36
Joined: Thu Sep 27, 2018 4:46 pm

[Out of memory error]: Crashed @ 128GB & 237GB virtual memory committed

Post by DerekZiemba »

It consumed all 64GB RAM.
Crashed at 128GB with an error I've not seen before: .\src\mem.c(1124): mem_alloc(): Fatal error: out of memory 137438953472
(137438953472 = 128Gibibyte).
& somehow Virtual Memory Committed managed to reach 237,094MB.
Untitled.png
Untitled.png (601.8 KiB) Viewed 10669 times
I was refactoring my bookmarks & macros & probably had 6-7 different search windows open plus the bookmarks, setting, & filters windows. It was definitely being abused.

I've had, I believe, the same crash happen enough times on my old PC I've learned long ago to close & reopen Everything every 30min-ish whenever creating/experimenting with macros, functions, bookmarks, & filters. The time elapsed doesn't actually matter. It can happen anytime. But changes
to bookmarks, filters, settings, etc. don't save until everything is closed & it sucks to lose a half hour of whatever I was changing.
On this new PC I neglected to force my changes to save bcus with substantially more power & ram, I thought maybe it wouldn't happen.
I was wrong.
The error on the new PC is a little different but seems the same I've always encountered - just taken to the extreme bcus of the additional resources. On the old PC there was 32GB RAM & 32GB set pagefile limit. I'd know when Everything was getting close to crashing bcus the whole PC would slow down. The new PC didn't really slow down. I did notice Explorer was slow to render & some of my chrome tabs refreshed, but I didn't think anything of it because it was still responsive.

I don't consider this too big of an issue since I'm abusing it It also only happens once in a blue moon (like when I'm abusing it).
If you're interested & have time I can supply the 1MB kernel.dmp, 10MB mini.dmp, & possibly the 2GB compressed full dump (123GB when extracted).

-------------
EDIT: Usually it would just crash and generate an event in the EventViewer & not save any of my changes. This was the first time it was a graceful shutdown with an error dialog. I clicked OK shortly after and it managed to still save the various csv files & database which was a nice surprise. No event was generated for the EventViewer.
Last edited by DerekZiemba on Mon Aug 28, 2023 2:40 pm, edited 1 time in total.
void
Developer
Posts: 15352
Joined: Fri Oct 16, 2009 11:31 pm

Re: [Out of memory error]: Crashed @ 128GB & 237GB virtual memory committed

Post by void »

Thank you for the issue report DerekZiemba,

Please send the mini crash dump to support@voidtools.com

Privacy
therube
Posts: 4610
Joined: Thu Sep 03, 2009 6:48 pm

Re: [Out of memory error]: Crashed @ 128GB & 237GB virtual memory committed

Post by therube »

(You mean the 120GB version he already has isn't good enough ;-).)

Now, as far as "Modules", does that imply that those various items have injected themselves into the Everything process space?
Revo, Icaros, Power Toys, Dropbox, MSWebp, Onedrive, Onenote...?

So the statics always load, regardless, & the dynamics only load if called?


I'll note that I have something called "IE" in mine. Gross.
.
Everything - Modules.png
Everything - Modules.png (40.3 KiB) Viewed 10583 times
(Win7.)
NotNull
Posts: 5261
Joined: Wed May 24, 2017 9:22 pm

Re: [Out of memory error]: Crashed @ 128GB & 237GB virtual memory committed

Post by NotNull »

therube wrote: Thu Aug 24, 2023 6:02 pm Now, as far as "Modules", does that imply that those various items have injected themselves into the Everything process space?
Most likely shell extensions. And if I *had* to guess: some of them don't "play nice" and leak memory or write to the wrong places in memory.

(Shell extensions can be disabled inside Everything, but the dumps might reveal which one is/are the culprit )
DerekZiemba
Posts: 36
Joined: Thu Sep 27, 2018 4:46 pm

Re: [Out of memory error]: Crashed @ 128GB & 237GB virtual memory committed

Post by DerekZiemba »

> (Shell extensions can be disabled inside Everything, but the dumps might reveal which one is/are the culprit )

I emailed the mini & kernel dumps to support@voidtools.com shortly after void requested.

> Now, as far as "Modules", does that imply that those various items have injected themselves into the Everything process space?

Yeah I included that in the screenshot for that reason. However I had to "Hide System" & "Hide knowndll images" or the list goes on forever. I only now just realized System Informer has a way to export the list. So attached is the exported lists of what's currently loaded, unfortunately when it crashed I didn't know this was an option.
----------------
Well shoot. Can't attach .csv or .md files & can't put it in the post bcus "Your message contains 285495 characters. The maximum number of allowed characters is 60000."
Guess I'll put it on github

(Note: If you scroll up to the raw formatted version of the file, it visually shows which module loaded which in a tree hierarchy)
Last edited by void on Fri Aug 25, 2023 2:10 am, edited 1 time in total.
Reason: removed links
void
Developer
Posts: 15352
Joined: Fri Oct 16, 2009 11:31 pm

Re: [Out of memory error]: Crashed @ 128GB & 237GB virtual memory committed

Post by void »

Thank you for the mini crash dump DerekZiemba,

I am still looking into the issue.

A buffer is corrupted.
The crash occurred when building the search from your filter macro while inside a list editor in the options window.

Could you please send your Filters-1.5.csv from %APPDATA%\Everything to support@voidtools.com
DerekZiemba
Posts: 36
Joined: Thu Sep 27, 2018 4:46 pm

Re: [Out of memory error]: Crashed @ 128GB & 237GB virtual memory committed

Post by DerekZiemba »

Oh wow, I only just realized my changes were saved! It appears like everything's there, right up to the crash, even the "Search History-1.5a.csv"
So I've attached all of it to that gist.

--------------------
> The crash occurred when building the search from your filter macro while inside a list editor in the options window.

I was primarily working with the Bookmarks feature so I'm a bit surprised it was the filters that caused it. The backup of `%APPDATA%\Everything` I have available on the new PC is from before I discovered bookmarks, so I was trying to get away from that & instead use bookmark macros.
Back then I was sticking a bunch of custom defined functions in the "_" filter, then referring to the `_:` macro in every other filter - as you'll see.

The Bookmarks are very much a work in progress. Trying to figure out what I can & cannot do. BTW, is there a way to do the `<ancestor:folder:attributes:HDI> "ungoogled*\"` on lines 49 to 66 of `search-history-1-5a-csv` type thing I was attempting? The goal is to filter all child files of an ancestor directory if any parent directory is marked hidden & not indexed, even if the child file has normal attributes.
Last edited by void on Fri Aug 25, 2023 2:09 am, edited 1 time in total.
Reason: removed links
void
Developer
Posts: 15352
Joined: Fri Oct 16, 2009 11:31 pm

Re: [Out of memory error]: Crashed @ 128GB & 237GB virtual memory committed

Post by void »

Thank you for the filters and bookmarks.

Bookmark macros eat all characters until a white space.
There's a few filter/bookmark macro references that explode because the parameters eaten are massive.

I'm working on improving the syntax and enforcing a hard limit of 8MB for macro expansion.



For now, please go through all your filters/bookmarks and check the search.
Please make sure all references to a nested filter/bookmark macro have a trailing space.

For example:

Name: EXEbin:<X>
Search: EXE_: <<X:>|<ignore-whitespace:Path:X:>|<appversion:X:>|<appname:X:>|<ignore-whitespace:ignore-punc:Company:X:>>

should be:

Search: EXE_: << X: >|< ignore-whitespace:Path:X: >|< appversion:X: >|< appname:X: >|< ignore-whitespace:ignore-punc:Company:X: >>


The Bookmarks are very much a work in progress. Trying to figure out what I can & cannot do. BTW, is there a way to do the `<ancestor:folder:attributes:HDI> "ungoogled*\"`
This can be done with File List Slots.

Search for:
folder:attributes:HDI "ungoogled*\"
Select all your folders and copy all the filenames to the clipboard (Ctrl + Shift + C)
Change the search to ancestorfilelist1:
Hold down Ctrl and click the ancestorfilelist1: text in the search box.
Paste your filenames (Ctrl + V)
Click OK.
DerekZiemba
Posts: 36
Joined: Thu Sep 27, 2018 4:46 pm

Re: [Out of memory error]: Crashed @ 128GB & 237GB virtual memory committed

Post by DerekZiemba »

:shock: those bookmarks are expanding to something like an 8mb query??? That blows my mind. It's magic that it can still be so fast.

--------
So that ungoogled thing was just an example. There's all kinds of other similar folders like it. In that case its an ungoogled portable chromium build & the folder just sits in my userprofile along with a bunch of other similar things such as other portable apps. Chrome in particular generates all kinds of files i don't care about.

Instead of manually targeting each folder like that everytime i do a search or by manually creating what i now imagine is a huge filter (that would break anytime a new similar thing is introduced), i thought maybe i could create a filter or bookmark/macro that'd automatically exclude anything like it. Excluding new things would be a simple as right clicking a parent folder i don't care much about, then In the windows property dialog setting hidden & not content indexed. The Everything filter when selected would then be able to not show that item and any of its descendants. I wouldn't need to edit an existing filter or add a new one.

Currently the only way i know to do that is when building the index by checking the "don't index hidden" or "don't index system files" checkboxs. Normally i don't want to see those types of files as it's all just noise. But occasionally i need to get at them (system files, hidden files, or other temp like, cached, resource, or generated files) & I'm not about to rebuild the index to do so. I just want my normal "Home" filter to exclude that type of stuff or any of the backed up stuff. Makes getting at them simply switching the filter or using a keybind.

---------
I'm pretty much trying to use the search syntax like a scripting language & am abusing the features i know about to try and get there. Feels like I'm jamming the cylinder into the triangle hole of that kids game & the only tool i know i have is a hammer.
Are there any plans to introduce maybe a scripting language? Lua perhaps?

---------
Curious why you removed the gist links. I didn't see anything in them that i feel violates my privacy. But in any case, I'll delete the gist, just in case.
NotNull
Posts: 5261
Joined: Wed May 24, 2017 9:22 pm

Re: [Out of memory error]: Crashed @ 128GB & 237GB virtual memory committed

Post by NotNull »

DerekZiemba wrote: Fri Aug 25, 2023 12:55 pm Curious why you removed the gist links. I didn't see anything in them that i feel violates my privacy.
According to void:
A full dump would contain all your filenames. (hundreds of MB)

A mini crash dump will only contain a small snapshot which includes the stack. (<10 MB)
The stack may contain a couple filenames from your system that Everything may currently be processing.
void
Developer
Posts: 15352
Joined: Fri Oct 16, 2009 11:31 pm

Re: [Out of memory error]: Crashed @ 128GB & 237GB virtual memory committed

Post by void »

I will consider an ancestorattribute: search function.
(this function would require attribute indexing -kindof like childattributes: )

Thank you for the suggestion.



Lua support is on my TODO list.

Currently bookmark macros are very basic and not really designed for nested calls.



voidtools does not collect any user data.
User data such as Search History and Everything settings are removed from the voidtools forums.
This information could be used to uniquely identify you. (Volume GUIDs are unique)
DerekZiemba
Posts: 36
Joined: Thu Sep 27, 2018 4:46 pm

Re: [Out of memory error]: Crashed @ 128GB & 237GB virtual memory committed

Post by DerekZiemba »

I've triggered it again... This time I have noticed my PC slow down significantly.
The standard windows task manager can't deal with this apparently - it's froze solid.
I imagine the leak must be all zero's or something because Physical Memory isn't really affected. In fact SoftwareInformer say's it's only using pretty much normal amounts as far as Private bytes & WorkingSet go. It is however showing 135GB Private Bytes allocated but doesn't appear to know which process to attribute it to.

Below are the the new bookmarks I was creating. I was working on the "ANNOYING" one when this occurred. It happened right after I put in the `#{: expression #}:` grouping brackets because I couldn't make heads or tails of the `<>`'s anymore. Now that I think of it, I think that type of grouping is what got me last time too.

"HOME:" , 0, "", , , , , , , , , , "WINDOWS: !SYS: !HIDDEN: !ANNOYING:" , , "", "", 0, , , , , 0, "HOME" , , ""
"WINDOWS:", 0, "", , , , , , , , , , " !< LINUX: >" , , "", "", 0, , , , , 0, "WINDOWS" , , ""
"LINUX:" , 0, "", , , , , , , , , , "<""Linux:\""|""C:\WSA""|""C:\**\wsl\""> " , , "", "", 0, , , , , 0, "LINUX" , , ""
"SYS:" , 0, "", , , , , , , , , , "< attrib:<S|HI|HSA|RHS|RHA|RHD|HDA|HDL> >|< owner:<""SYSTEM""|""TrustedInstaller""|""SERVICE""> >|< ""C:\ProgramData\Package Cache\"" >|< ""C:\Windows\"" ext:evtx >" , , "", "", 0, , , , , 0, "SYS" , , ""
"HIDDEN" , 0, "", , , , , , , , , , "< attrib:H >|< regex:path:""(?-i)\\\.[a-z]+\\"" >" , , "", "", 0, , , , , 0, "HIDDEN" , , ""
"ANNOYING", 0, "", , , , , , , , , , "!<""node_modules\""|""packages\""|""\cache\""|#{: t<""""|""e"">mp\**cache\ #}:|#{: \<""""|c|ca|app|html|dx|d3ds|dawn|font|gpu|""npm-""|""NV_""|offline|package|startup|web>cache<""""|2|d|s|data>\ #}:|""Users\*\AppData\**\User Data\""|""*google*chrom*\User Data\"" >", , "", "", 0, , , , , 0, "ANNOYING", , ""
VoidTools Memory Leak.png
VoidTools Memory Leak.png (443.05 KiB) Viewed 10081 times
--------------
While typing this, before I could get a minidump, I just lost SystemInformer. Oddly Everything seems to still be working fine...
Luckily ProcessExplorer successfully launched & I was able to grab another mini dump. I've attached it as "Everything64.zip.
SystemInformerDead.png
SystemInformerDead.png (269.17 KiB) Viewed 10081 times
------------
EDIT:
And now I lost vscode too. Odd error. Says too many files open??? Might be relevant to something Everything has done. Everything seems to be working fine tho. I just exited it & confirmed in ProcessExplorer that it had exited, but it has not freed the RAM.
vscode-rainboxcsv.png
vscode-rainboxcsv.png (38.87 KiB) Viewed 10080 times
Last edited by void on Sat Sep 02, 2023 10:14 am, edited 1 time in total.
Reason: removed dmp
void
Developer
Posts: 15352
Joined: Fri Oct 16, 2009 11:31 pm

Re: [Out of memory error]: Crashed @ 128GB & 237GB virtual memory committed

Post by void »

Thank you for the dmp DerekZiemba,

The dmp was just Everything running normally.



Everything 1.5.0.1356a makes a lot of changes here:

The preprocessor has been simplified.
This will likely break things...

Everything will now do two passes on the search.

The first pass will expand any #[function-name: ... #]: functions.
The search is then broken into search terms.
The second pass will expand any [function-name:...] functions in each search term.

Bookmark and filter searches will apply the same process.

Any [function-name:...] functions inside parameters passed to macros are now expanded and treated as quoted text.



There is now a 8MB limit on macro expansion.

To set the expansion limit:
  • In Everything 1.5, from the Tools menu, click Options.
  • Click the Advanced tab on the left.
  • To the right of Show settings containing, search for:
    max
  • Select search_max_expand_size.
  • Set the value to: 8388608
    (where 8388608 is the desired limit in bytes)
  • Click OK.
search_max_expand_size



Everything 1.5.0.1356a fixes an issue with complex subexpressions.
You might have been seeing hangs caused by this issue.
Previous versions would hang on searches like: name:<<abc 123> | <foo bar> | <baz qux>>
Highlighting performance is also improved with complex subexpressions.
The NOT operator (!) will now work inside subexpressions. for example: name:<!<<abc 123> | <foo bar> | <baz qux>>>

I highly recommend using spaces around < > and | in your filter/bookmark searches as future versions will likely eat the < > and |

For example:

case:< name:"abc 123" ext:mp3;jpg >
case:< ext:mp3;jpg name:$param: >



Please let me know if you see any hangs with this version.
Please let me know if any of your bookmark/filter macros are no longer working.
DerekZiemba
Posts: 36
Joined: Thu Sep 27, 2018 4:46 pm

Re: [Out of memory error]: Crashed @ 128GB & 237GB virtual memory committed

Post by DerekZiemba »

Thank you for Everything. I'll give that a try.
Hopefully it doesn't break too bad... Will it give me some kind of indication if I've exceeded the 8MB macro expansion limit? I may try setting it lower to identify any potential problematic macros. Just blows my mind they may be expanding that much.
The NOT operator (!) will now work inside subexpressions. for example: name:<!<<abc 123> | <foo bar> | <baz qux>>>
Hmm. I always thought it did. Guess that explains why some things haven't worked as expected.
I highly recommend using spaces around < > and | in your filter/bookmark searches as future versions will likely eat the < > and |
👍 Already reworked all my macros (Filters & Bookmarks) after you last mentioned this. Some things that have always stumped me as to why it wasn't working, now work!
The first pass will expand any #[function-name: ... #]: functions.
The search is then broken into search terms.
The second pass will expand any [function-name:...] functions in each search term.
I'm actually not sure if I use that form very much. But I'm also confused, and have always been, over what I'm actually using or doing. In most cases I'm not sure if what I'm using:
  • Is a regular built in Search Function
  • Is a built in Search Preprocessor Function? (confusion on how it differs from above)
  • Not a function, but just a Search Modifier?
  • Is none of the above, but just a regular Property?
  • Is the built in thing I'm using any of the above, or is it actually a macro? What actually constitutes a macro?
    • Is anything built in considered a macro? Or do the things listed above avoid the expansion issue?
    • Are macros only user defined? With the term only being used to reference things like: Filters & Bookmarks?
    • How does `[define:definition]` fit into all of this? The syntax to call it is different (requiring a # hashtag) & documentation refers to it as a user defined function.
      - Is this effectively just a Macro? Or is it a Function & somehow fundamentally different? How do they differ?
      - Does it differ from macros defined using `/define name=search`?
      - Does a user defined function (of either the `[define:definition]` or `/define name=search` variant) undergo expansion the same way macros (as in Filters & Bookmarks) do?
Basically I don't really know what I'm doing and just kind of piece something together until it works. I'm unsure what difference's there are or may be between the different methods of filtering listed above.
------------------
Anyway, I threw a $20 your way. Always impressed by your support, work, and commitment.

One day, could syntax highlighting be implemented (also multiline queries... a man can dream) ? Ideally to color matching braces & hint at invalid syntax. Also to potentially identify what's a:
- function
- preprocessor function
- search modifier
- property
- filter macro
- bookmark macro
- some other user defined macro/function (are there any others?)
- string (or what's being treated as a string)
- number
- operator (as in size:>10b) so the `>` would be colorized as an operator to avoid confusion with brace matching
------------------
EDIT:
case:< ext:mp3;jpg name:$param: >
I just noticed the `$param`. In what scenario do we get parameters and can use them in that form?
  • In `[define:definition]` form, params are `#arg:`. The `$` form (ie: `$columnHeading`) is used to reference column names (see viewtopic.php?f=12&t=10099#ifdef & #iflen below it) as well as (I think) metafields like $result-count:, $folder-result-count:, etc. & special variables such as $column0, $column1, ....etc.
  • In macros, AFAIK you can't really operate on the passed in argument & it can only be passed to other things.
    For example (that also showcases roughly what I envision a syntax highlighting feature would do), referencing the param is done like so:
    case:< ext:mp3;jpgname:< param: > >
    As far as I know, there is no $param: like functionality available outside of I think $column1:, $column2:, ...etc.
    • Regarding the syntax highlighting / semantic formatting example:
      • Spaces that are syntactically significant are brighter than the background color. However, spaces that are not significant to the syntax, for example any that'd be enclosed in quotes ie: "C:\Program Files (x86)" , would be treated same as any other enclosed token like characters such as : \ ( ) ..
        ➢ Spaces that are just eaten would not be colored unless touching a neighboring space that does matter - also there could be a single line of pixels @ normal background color separating them to visualize they're not necessary.
        ➢ The quote color is the same as mp3 & jpg which are interpreted as a string, but that actual quoted content should be slightly brighter so it doesn't all blur together / keeps tokens & identifiers obvious.
      • Notice space between ext: & name: clauses is helpfully slightly brighter & wider (em space U+2003) to make it obvious that space in particular is interpreted as a logical AND.
        ➢ Would be a nice QOL improvement.
        ➢ It'd need to be brighter than other spaces to differentiate from the "touching a neighboring space that does matter" spaces.
      • The <> & <> braces are almost the same color as the function/modifier/macro they belong to. In this case they're slightly brighter. This is so if another clause is nested that belongs to the same colored syntax type, the nested members brackets could instead be a darker shade. As more nesting takes place, the 1st level braces would get brighter yet, the 2nd level darker, and the 3rd level somewhere in between, etc.
        ➢ In my example they were not made obviously brighter (depending on the quality of your monitor, they're probably too close for gaming/6bit+FRC monitors to differentiate) because it'd only become necessary above 1 level nesting.
-----------
EDIT 2:
I went through & reread most of the docs I referenced above. It makes a little more sense after reading the additional comments below the initial posts. But I still have some kind of mental block that's preventing me from entirely grasping std search FNs(especially vs std properties/column names), PreProcessor FNs, modifiers, user defined FNs, MACROs, & how to and when to tie them all together. What makes sense now is PreProcessorFNs return a value & SearchFNs filter results.
What I struggled with was piping output from a PreProcessorFN that contained nested SearchFNs or Properties to another SearchFN & putting all that together in a MACRO in a way that is valid syntax, then using that MACRO from the search box. But now I think I'm realizing that's not possible bcus SearchFNs are for searching and always affect the result list. I guess I thought I could pass the results of a SearchFN to a PreProcessorFN (basically behind the scenes would built an intermediate result list) that returned something I could then pass to a SearchFN to filter the visible result list. Part of the confusion stems from columnnames, properties, & SearchFNs resembling & basically being referenced/called the same way.

I also found the dollar `$` `$param` syntax is used in Column Formulas. But still, how would you pass a parameter to a column formula? Is that somehow possible?
void
Developer
Posts: 15352
Joined: Fri Oct 16, 2009 11:31 pm

Re: [Out of memory error]: Crashed @ 128GB & 237GB virtual memory committed

Post by void »

Thank you for your donation and support DerekZiemba,
In most cases I'm not sure if what I'm using:
Things have changed quite a bit from the original preprocessor implementation.
Perhaps I need to reword a few things.

Here is the order in which Everything parses your search:



Preprocessor always looks like:

#[function:...#]:
-or-
#function:

These are expanded before anything else.
This was the only method available in the first preprocessor implementation.
With the first preprocessor implementation, #[quote:...#]: was designed to convert text to a search term.



Search modifiers are always before search functions / macros:
modifier:function:parameters



Macros:
macro:parameter
Macros override search functions.
Can expand into multiple search functions.



Search functions always look like:
function:parameter

property-name: is treated the same as search-function:
Almost all properties have an associated search function with the same name.

For example:
date-modified:



The search function parameter can include the following preprocessor style:

[function:...]

This style was recently added.
You no longer need to use the awkward #[quote:...#]: style.
The expanded text is treated as quoted text.
I will refer to this style as the "termprocessor".

Parameters can also include references to your macros.
Expanded macros are treated as quoted text.



There are shortcuts for the preprocessor and termprocessor.

A good example of this is the clipboard preprocessor function:

#clipboard: is the same as #[clipboard:#]:
clipboard: is the same as [clipboard:]

#clipboard: will treat spaces as AND
clipboard: will treat spaces literally

clipboard: preprocessor.



There are conflicting search functions and termprocessor functions.

For example, the year: search function and the year: termprocessor function.
Search functions are always used first, so:
year:year:"2023-09-09"
is the same as:
year:[year:"2023-09-09"]
=> year:2023
(The year: search function is then called with 2023)

year: preprocessor.
year: search function.

Since there's no clipboard: search function, clipboard: on its own is treated as a termprocessor function.



Eval scripts always include the following (anywhere in the term):
$property:

For example:
len($stem:)%3==0
=> match filenames where the stem length is a multiple of 3.



In short:
#[function:...#]: = preprocessor.
#function: = preprocessor.
[function:] = termprocessor.
$property: = eval script.
function: = modifier > macro > search function > termprocessor.


Is the built in thing I'm using any of the above, or is it actually a macro? What actually constitutes a macro?
#[define:...#]: or #define: or [define:] is a preprocessor define.
This is different to a filter macro, bookmark macro or just a macro.

The preprocessor does not have access to bookmarks/filters/macros.
Only use #define: if you want to access these defines from the preprocessor.


Is anything built in considered a macro? Or do the things listed above avoid the expansion issue?
No, there are no macros if you delete all your filters.
The default filters contain a few macros, for example: exe:


Are macros only user defined? With the term only being used to reference things like: Filters & Bookmarks?
Yes.
(Besides the default filters macros which can be deleted or changed)


How does `[define:definition]` fit into all of this? The syntax to call it is different (requiring a # hashtag) & documentation refers to it as a user defined function.
Defines can be a simple string or complex functions.
You can reference your defines with either #[define-name:#]: or #define-name: or [define-name:]
You can call your defined functions with either #[define-name:...#]: or #define-name:<parameters> #define-name:parameters or [define-name:parameters]



Is this effectively just a Macro? Or is it a Function & somehow fundamentally different? How do they differ?
Defines are fundamentally different to macros.
Defines can be called or referenced from the preprocessor.
The preprocessor does not have access to filters/bookmarks/macros.


Does it differ from macros defined using `/define name=search`?
My poor choice of naming here.
/define creates a macro.
It has nothing to do with the preprocessor define.


Does a user defined function (of either the `[define:definition]` or `/define name=search` variant) undergo expansion the same way macros (as in Filters & Bookmarks) do?
Yes and no.
Depends on how they are called.
#[mydefine:#]: or #mydefine: will expand spaces as AND.
[mydefine:] will expand as quoted text.

mymacro: outside a search function will expand spaces as AND
mymacro: inside a search function will expand as quoted text.

I recommend using macros over preprocessor defines.



I will consider search syntax highlighting.
Thank you for the suggestion.


Just noticed the `$param`. In what scenario do we get parameters and can use them in that form?
Support for $param: was very recently added. (1355a)
Previously, macros would have to define the parameter name.
Now you can reference the parameter without defining a parameter name.

A filter macro example:
Name: Upper Case Example
Search: case:[upper:paramname:]
Macro: up<paramname>

up:abc
=>
case:ABC

The preferred syntax is now:
Name: Upper Case Example
Search: case:[upper:$param:]
Macro: up

up:abc
=>
case:ABC



Preprocessor defines must define the parameter names.
Preprocessor defines can have more than one parameter.


In macros, AFAIK you can't really operate on the passed in argument & it can only be passed to other things.
The search in macros are expanded twice.
Once with the preprocessor and again with the termprocessor.
(exactly the same as if you had typed the macro search into the search box)



$param: is only available inside the macro search.



I do want to add support for LUA at some stage.
However, I am really trying to avoid having the user enter in syntactically correct searches.
The Everything search syntax is flexible to allow searches like: size:>128mb (don't treat > as a closing the search group) -no one will want to escape the >
Each search function has it's own syntax.
Generally, most search functions all following the basic search syntax standard.
Space is always AND.

I am considering more aliases for operators.
for example:
::and::
::or::
::not::
::group-start::
::group-end::
This might make it easier to build complex macros.



Column Formulas are different again.
Column Formulas are C expressions which evaluate to a string or number.
Column Formulas are evaluated for each result -after the search.
Use $property: to reference the property values for the current result.
You can use column-a - column-f to define custom values and reference them with $a: - $f:
column-a - column-f can also be set from your search.
You can also use regex group captures and reference them with $0: - $9:
void
Developer
Posts: 15352
Joined: Fri Oct 16, 2009 11:31 pm

Re: [Out of memory error]: Crashed @ 128GB & 237GB virtual memory committed

Post by void »

Everything 1.5.0.1359a adds a ancestor-attributes: search function.



ancestor-attritbutes: requires attribute indexing.

To enable attribute indexing:
  • In Everything, from the Tools menu, click Options.
  • Click the Indexes tab on the left.
  • Check Index attributes.
  • Click OK.


Please note: all volume roots (for example: C: ) have the Hidden and System attributes set.
Everything will ignore these attributes for volume roots when using ancestor-attribute:



Example usage:
ancestor-attr:HDI
Post Reply