TTW Performance Guide (OLD, Do not use!)

[Mystical Panda]

[quote=MajinCry]

 

What happens if you set ReservedMemorySizeMb=512?

Edit: Also, what operating system are you using? If it's Windows 10, I wouldn't be surprised if shit's just wonky because it's, well, Windows 10.

[/quote]

It's Windows 7 Ultimate (all updates applied- the Windows 10 breakdates thank goodness won't apply), Steam Version of FNV. Let me post the load order I'm testing, this way you can see the version of all the mods including NVSE, JIP, ENBSeries and everything that doesn't show up in just a load order (plugins list). It'll show all assets installed (via symlinks for loose and normal copy for plugins and bsa archives). The plugins and bsas are dated to the load order.

In this second test, you can see the bottom a 20+ minute test, same areas as before (loaded save from Mojave Outpost, killed ants then went to Nipton- in the one that crashed, I didn't quite make it through the ant quest at Mojave Outpost), but no out of memory problems.

Let me run another test, then break out and see how many host.exe are running- the VRAM usage (according to MSIAfterburner) was 2.6-2.9gb so there should be at least 2 enbhost.exe running (last time I checked they were cap at about 1.9gb before starting a new instance). I'll also set reserved back to 512.

[Mystical Panda]

@ MajinCry - Here's the results for the next test; same everything as the previous, stable (no out of mem hits), but no enbhost.exe processes in the task list. This def implies that setting EnableUnsafeMemoryHacks=true might indeed disable using enbhost processes (just buffering in VRAM). But it might also be something else, not sure just yet.

What, at least to me at the onset, looks strange is that enbhost.exe (by not being present as a running task while the game is running and using 2.5+gb of graphics memory), it's not using enbhost at all and just caching things in the card. Which, it very well could be (I don't know his code), since the card has ample enough memory in and of itself to load all the textures needed for a given area (even modded). Thus enbhost.exe wouldn't need to be fired off at all until gpu memory runs out then it buffers everything in system ram.

However, if that were the case (everything could be buffered in gpu memory), I shouldn't have gotten an out of memory the first time around (I know you were thinking burst memory wasn't enough, and that might a possibility). By leaving all the settings the same, but setting ReduceSystemMemoryUsage=true and running the same test again, we're now stable. Ok, so no enbhost.exe was running, enough graphics memory but crash when one is false and the other is true, but no crash when both are true. This, at least to me, implies there's more his code is doing that just what is documented.

Here's what I think is happening...

EnableUnsafeMemoryHacks=true should keep DX from using the game's executable virtual memory space from using it's memory to 'shadow' textures.

ReduceSystemMemoryUsage=true should be using enbhost.exe to 'shadow' textures (probably other data also, meshes for example.), instead of the game's executable.

In both cases, where reserved memory was 128, one crashed, the other was stable. No enbhost in the mix, and safe hacks should trump reduce system usage. I think it's doing both. The out of memory was due to the game itself hitting a memory wall (shadowed texture wise), which was fixed by using reduce system memory. And the other, allowed the gpu VRAM to get stuff, but never full since it had enough for what was being asked of it size wise.

Let me try another test with memory sampling and see what the results show. It should show the game's memory getting tanked (which would be my first assumption based on the above results).

[Mystical Panda]

@ MajinCry ... here's another quick test; same everything as before (only thing changed was a single enb setting- reduce system memory usage). The first test (test1.jpg) using:

EnableUnsafeMemoryHacks=true

ReduceSystemMemoryUsage=false

The second test (test2.jpg) using:

EnableUnsafeMemoryHacks=true

ReduceSystemMemoryUsage=true

You can see on the first test, the virtual memory used by the game (in green) is spiking towards the limit 3.35gb (this one had reduced system memory usage = false), and that's just loading up a game save just outside of Nipton and walking a few feet (vram @ 2.3gb). An out of memory is coming shortly.

The second test (test2.jpg) shows the same game load and distance traveled but were peaking around 2.42mb   It, at least to me, looks like each does a completely different function; one shadows the textures, the other attempts to tank out the gpu vram without using enbhost (one never fired off for me). But... this really all depends on the system, gpu and the conditionals in his code that check all that stuff.

Also, when I use the reducesystemmemoryusage=true, I do get a noticable fps increase (as opposed to setting it to false). Disabling the graphics driver in lieu of his code will give me a noticable fps drop, but I'll test in a bit and see if the memory usage maps show up different (this will tells us if that's required for enablesafememoryhacks to do job of both.

Without knowing what hooks what, and how the game hooks work together or circumvent each other, this could be something that just works in this combination, but shouldn't (not intended), but does kinda thing; enb, nvac, nvse, nvsr and any others that 'hook' the game. 

I hope of these test are helpful.

[Edit] Oddly enough, MSIAfterburner reported about the same vram being used in both tests. This to me impiles that shadowing of graphics memory in the games virtual memory space is just being 'disabled' not 'redirected'; disable shadowing since we need to page/redirect textures to enbhost any ways (depending of gpu of course).

[Mystical Panda]

Here's the result of test #3 using:

EnableUnsafeMemoryHacks=true

ReduceSystemMemoryUsage=false

DisableDriverMemoryManager=true

You see the virual memory for the game starting to spike up just after loading a saved game, and walking a few feet. The disk usage would be lower than the first test, I'd imagine, since it's using buffered data (I'm pretty sure I have set to use the drive cache- no problems with that so far. It might not matter if Windows does this automatically, therefore I could see an ini setting that isn't used by the game, yet still seems to work since windows would do it- no need for redundant code). The vram used was just about the same as the others (a few mb difference between them) as reported by MSIAfterburner. So using the disabling the graphics driver, didn't really help (the default settings) when used in conjunction with enableunsafememoryhacks.

I'm still leaning towards two separate functions, two separate uses. But that just a quick analysis from the few tests I did. For me to be stable, I need to have both reduced and unsafe set to true, though I'm sure setting unsafe to false would also work fine since the biggest problem, at least for me, isnt gpu vram, but tanking out the game's virtual memory space.

I hope that helps some- back trying to figure out a way to search 'massive...MASSIVE' catalogs quickly and efficiently without using canned things like MySql, etc.,. without time-consuming temporarily building search specific catalogs (which I might have to eventually settle on) on the fly.

[FiftyTifty]

It's bizarre that the two settings need to be used in combination for the best result. The ENBHost.exe process is tied to ReduceSystemMemoryUsage, and that's where the models and textures are mirrored into (rather than into FalloutNV.exe). And EnableUnsafeMemoryHacks disables the mirroring entirely (identical to D3D 11's usage of VRAM; no mirroring into RAM).

[Mystical Panda]

If he was shooting for both functions to do the exact same thing, in a way, it could just be a bug or anomaly (not necessarily a true bug, but an unexpected result- which still might be considered a bug, depends).

I wonder if he did a kind of 'addendum' type thing- initially conclude that all that needs to be done to 'extend' memory is offload data onto 'hosts', then later added something to disable shadowing or caching of textures in system memory. Or vice versa. It's like they both do different functions, but can be combined. Most just used reduce memory usage (which fired off multiple copies of the host), which seemed to work. Then later he designed an alternative, but built it into an already existing code structure creating a distinct difference between the two, yet the latter not being fully independent; neither do the work of both.

In some of the previous tests I did (before these) when I was looking for a good balance, I did have unsafe memory hacks disabled (used just reduce system memory enabled) and everything seemed to work fairly rock solid with no out of memory issues. It was when trying to fallback on just the unsafe hacks I hit an out of memory. 

Doesn't Boris have a 1-2gb NVidia graphics card? I wonder what the tests would be using one of those; to be stable though, it'd still have to use multiple copies of host for really really large texture sets since there just wouldn't be enough vram for an area. Otherwise an out of memory crash is pretty much guaranteed.

Here's my initial thoughts on the better fps, if this holds true...

1) By loading everything on the card, there's no copying data from the multiple hosts or system memory to the gpus memory saving cycles for other threads and processes. This would especially hold true for those cpus with a lower ipc or fewer cores (interrupts eating or compounding the overall time slice).

2) By reducing virtual memory for the game (reduce system memory usage), there's more room for other types of data like lists, arrays, objects and the like (maybe more npc data, things like that). If not tanked with tons of scripts, less the game has to juggle working size wise. If that makes sense.

3) When you look at the memory map for test1.jpg, the game figured virtual memory was just about tanked so it tried to load balance the data, and these constant 'checks' or scanning of the lists, were taking away from the fps (I'd venture a guess of about ~10% to the negative). That would seem a very logical possibility.

When I get a bit more time I can do more testing and see what kinds of results we could possibly get. It's np, just need to kinda slice back and forth between the different things as we go.

[RoyBatty]

So I did some testing, the turn hitching is gone with unsafe memory hacks enabled. But I can see absolutely no FPS increase or notice any other changes really, also the game fails to launch half the time with windowed mode enabled. I guess this is more for people with older or weaker machines, it's of no benefit to me it seems as the game runs identical other than some input lag with this enabled and the launch failing. I also noticed the anisotropic filtering forcing can be disabled and it will use the game setting instead which has less banding.

[Mystical Panda]

[quote=RoyBatty]

 

So I did some testing, the turn hitching is gone with unsafe memory hacks enabled. But I can see absolutely no FPS increase or notice any other changes really, also the game fails to launch half the time with windowed mode enabled. I guess this is more for people with older or weaker machines, it's of no benefit to me it seems as the game runs identical other than some input lag with this enabled and the launch failing. I also noticed the anisotropic filtering forcing can be disabled and it will use the game setting instead which has less banding.

[/quote]

At least you were able to get the hitching out, but added some undesirable anomalies. It's definitely not for everyone.

I also think it's for those wanting to push the game engine a bit past what 32bit systems can handle virtual memory space wise- with large enough graphics or just "too much stuff" an "out of memory" happens quite frequently regardless of how powerful or 'weak' a system might be (its about the virtual space), so off-loading of 'something' (in this case) shadowed/ cached textures allows for something else like game data to take it's place within that virtual limit.

With a more inexpensive gpu and/ or less vram, it might help when using larger texture packs, but that will more than likely also increase the 'bandwidth' requirement from the card when just rendering in higher resolutions which would more than likely also decrease fps (which, not taking either side, wouldn't necessarily be the fault of the tool).

In the end, it's kind of a balancing act.

[Mystical Panda]

One more quick note which might help out, before I forget...

@ Roy, you mentioned awhile back you were able to use hi-res textures, without an out of memory issue (except where you mentioned NVSR could potentially cause that); [reduce system memory] wouldn't apply to your situation. However, loading all the images into VRAM [unsafe memory hacks], in your situation, would result in less hitching since you have 10+ gb VRAM- no need to decompress, re-load or copy from shadowed/ cached system memory into VRAM; it's right there to use, always. That would be absolutely expected.

The other setting [reduce system memory usage] is for those hitting an out of memory issue (not necessary gpu VRAM, but application virtual ram), which is what I think I was encountering with the initial test I did using just [unsafe memory hacks]; the virtual space for the app ran out according to the memory graphics I posted as just using [unsafe memory hacks] didn't seem to work as both during my test. Which is why doubling the options worked for me; [unsafe memory hacks] wasn't necessary, but would reduce hitching yes, and if hitching is a noticeable slow (we see a 'hitch' in the frame rate), there would an fps drop of some sort at that moment; with it it improves because of no 'hitching'.

Though, as I was testing them out, I'm pretty sure they both work differently from each other and are used under different circumstances, and one can't necessarily supercede the other in all circumstances.

[FiftyTifty]

[quote=RoyBatty]

 

So I did some testing, the turn hitching is gone with unsafe memory hacks enabled. But I can see absolutely no FPS increase or notice any other changes really, also the game fails to launch half the time with windowed mode enabled. I guess this is more for people with older or weaker machines, it's of no benefit to me it seems as the game runs identical other than some input lag with this enabled and the launch failing. I also noticed the anisotropic filtering forcing can be disabled and it will use the game setting instead which has less banding.

[/quote]

 

The only way I've had launch failing with ENB's borderless window, is if you dick around during the couple seconds between launching the game and the window appearing. So if you launch obse_loader.exe, and then alt-tab to firefox, the process will just be a zombie requiring task manager to kill it. The solution is to wait the couple seconds for the window to appear.

EnableUnsafeMemoryHacks cannot give a higher FPS. But it significantly reduces hitching, including cell load hitches. The game is a damn sight worse without it.