Write index to disk

Have a suggestion for "Everything"? Please post it here.
Post Reply
bobm
Posts: 17
Joined: Fri Aug 21, 2015 11:27 am

Write index to disk

Post by bobm » Tue Nov 27, 2018 11:11 am

Would it be possible to add an option to automatically write the index to disk, either immediately after the scan or at a defined time interval between scans?

Alternatively, an option from the drop down menu (or an icon) could be provided to allow the operator to manually write the index, without having to close the application.

Everything would continue to use the RAM based index but the up to date disk version would be available immediately in the event of power outage or other dirty shutdown, which is an issue if Everything has been running for a long time (eg several months).

Thanks.

horst.epp
Posts: 144
Joined: Fri Apr 04, 2014 3:24 pm

Re: Write index to disk

Post by horst.epp » Tue Nov 27, 2018 11:56 am

That would be a very bad idea as the index changes all the time after an initial scan.
So there would be useless savings to disk which today may be an SDD.

bobm
Posts: 17
Joined: Fri Aug 21, 2015 11:27 am

Re: Write index to disk

Post by bobm » Tue Nov 27, 2018 11:59 am

It is better to revert to the last scan index rather than one which is several months old.
Also, in my case, the disk contains 7+ million files, is a network share and is not SSD.

NotNull
Posts: 972
Joined: Wed May 24, 2017 9:22 pm

Re: Write index to disk

Post by NotNull » Tue Nov 27, 2018 12:21 pm

When Everything is running, a separate
Everything.exe -update
will write the (RAM) database to disk.

You could run this as a scheduled task, for example.

bobm
Posts: 17
Joined: Fri Aug 21, 2015 11:27 am

Re: Write index to disk

Post by bobm » Tue Nov 27, 2018 12:31 pm

Indeed, that would work but it would be good to have that functionality in the GUI.

NotNull
Posts: 972
Joined: Wed May 24, 2017 9:22 pm

Re: Write index to disk

Post by NotNull » Tue Nov 27, 2018 12:32 pm

It is already on the to do list ...

bobm
Posts: 17
Joined: Fri Aug 21, 2015 11:27 am

Re: Write index to disk

Post by bobm » Tue Nov 27, 2018 12:34 pm

:)

void
Site Admin
Posts: 4265
Joined: Fri Oct 16, 2009 11:31 pm

Re: Write index to disk

Post by void » Wed Nov 28, 2018 6:32 am

I've added an option for the next version to periodically save the index to disk.

You will be able to disable this feature, or set how often Everything will save the index to disk.
I am experimenting with the default of 24hours. However, there are some concerns with unwanted disk access.

bobm
Posts: 17
Joined: Fri Aug 21, 2015 11:27 am

Re: Write index to disk

Post by bobm » Wed Nov 28, 2018 8:30 am

Thanks very much.

NotNull
Posts: 972
Joined: Wed May 24, 2017 9:22 pm

Re: Write index to disk

Post by NotNull » Wed Nov 28, 2018 7:02 pm

void wrote:
Wed Nov 28, 2018 6:32 am
However, there are some concerns with unwanted disk access.
Like machines waking up from stand-by?

void
Site Admin
Posts: 4265
Joined: Fri Oct 16, 2009 11:31 pm

Re: Write index to disk

Post by void » Thu Nov 29, 2018 9:22 am

Everything will never wake from stand-by to save the index to disk.
It will only occur at the specified interval, or, if the PC was in standby and the scheduled save is missed, then the save to disk will occur when the PC is woken up..

The save to disk when your computer wakes up might be undesired, more so if you have a large Everything database.., limited memory and a slow hard drive..

I need to make sure Everything will only save to disk when the disk and CPU is idle, which it currently does not do..

NotNull
Posts: 972
Joined: Wed May 24, 2017 9:22 pm

Re: Write index to disk

Post by NotNull » Thu Nov 29, 2018 9:07 pm

void wrote:
Thu Nov 29, 2018 9:22 am
I need to make sure Everything will only save to disk when the disk and CPU is idle, which it currently does not do..
I think that is very difficult; even Microsoft itself is not very successful with this (the Indexing service should only run during quiet times. In theory ..)
void wrote:
Thu Nov 29, 2018 9:22 am
The save to disk when your computer wakes up might be undesired, more so if you have a large Everything database.., limited memory and a slow hard drive..
That's my computer! ;). (with a very sluggish CPU on top of that)

You can do a WMI query to see when machine woke up from stand-by by reading the eventlog.
In CMD:

Code: Select all

WMIC.exe NTEVENT WHERE "LogFile='System' AND SourceName='Microsoft-Windows-Kernel-Power' AND Category=102" GET TimeWritten
You will only need the first result.

Replace GET TimeWritten with GET * for (a lot) more information about this event type.

void
Site Admin
Posts: 4265
Joined: Fri Oct 16, 2009 11:31 pm

Re: Write index to disk

Post by void » Fri Nov 30, 2018 11:16 am

Thanks for the WMIC suggestion. Looks like QueryUnbiasedInterruptTime could be possibility, also WM_POWERBROADCAST to save on suspend or standby.

Detecting idle state can be tricky, I have to be careful not to over engineer Everything so that it never detects an idle state...

I'm currently considering only saving the database to disk when an Everything window is closed and a 24 hour timer has elapsed from the initial load/last save..

Post Reply