Replies: 0 (Who?), Viewed: 356 times.
#1 22nd Sep 2020 at 8:49 PM Last edited by nitromon : 24th Sep 2020 at 9:47 PM.
Thanks: 717 in 30 Posts
Caching your TS3?This is a subtopic discussion to this thread from a while back: (How to make TS3 use > 4GB of RAM)
So I've been looking around and upgrading mine and various' people's laptops who could support a 2ndary SSD, either in mSATA, M2. SATA, or NVMe. For myself, I got a Samsung mSATA decent quality and put my TS3 on it. However, performance wise I realized it didn't make the game run better for me. The reason is because I already use a cache software called eBoostr, which cached most of the files that lag the game such as thumbnails, etc... into the RAM. So the limiting factor for my system in this case is actually the Gen 3 i7. Weird.
Now the best case scenario is to run the whole game in the RAM, such as a 32-40GB ramdisk, which is not really idealistic or realistic in most people's case. But even if you couldn't, you can still put your TS3 user folder on a ramdisk, from the link above. There are 2 reasons for this.
1) TS3 User folder is where your compositor, simcompositor, thumbnails are located and the game "constantly" writes and reads to them "while you play." Granted new SSD can sustain a lot more writing than it use to, but still it is not something people want to do.
2) The speed of ramdisk is still multitudes time faster than even the newest NVMe. Here's a comparison, these are my actual tests on my system.
This is my 7200 RPM Seagate HDD:
This is my Samsung mSATA:
This is the free Softperfect Ramdisk, one used in the tutorial:
I have a Mushkin Helix-l NVMe installed on my dad's laptop, but I didn't do the test. It is considered a high performance "entry" level NVMe, so it is not as powerful and fast as higher grade. However, I got this because it uses max of 4.5W, 1/2 of a typical NVMe.
This is from the spec:
Seq read: Up to 2010MB/s
write: Up to 1250MB/s
Even the higher ones at most 3000MB/s read and 2000MB/s write, the random read/write is also magnitudes faster than SATA SSD, but still not near RAMdisk. Also, my RAM is DDR3-1600, today's DDR4 should be a lot faster.
Now I don't know how many people do RAM caching, if you do, there are a lot of benefits to it. Unfortunately, there are no freeware caching programs out there. They do usually come with free 30 day trials if you want to try.
eBoostr: Now this is a very interesting program. Most discussions use it to compare to Readyboost, but I think that's a faulty comparison. What makes this program so good is how easily it is to customize it. I've been using it for years now, devoting 2GB of my RAM to let it cache "bits and pieces" of programs I use the most into the RAM. Naturally a lot of my Sims 3 files were cached in there, such as the thumbnails from the game files.
Recently, I've been wondering what if I devote the whole eBoostr to TS3? By caching it on default, it caches system files and other programs, which exceeds my 2GB limit (you can add a lot more if you want), that's why it wouldn't put all the TS3 caching files on there. So I disabled and blocked every drive and directory other than where I installed TS3. By doing so, the whole 2GB RAM caches only TS3 game files.
Here's the interesting result:
1) I am running all but 2 expansions (showtime and supernatural). The game only requires about 2GB of caching, anything more than that, the cache doesn't get filled. With the 2GB ram cache, there is virtually 0 HDD usage read or write (with the exception to the user folder that is cached in a ramdisk). It doesn't actually means the game only needs 3.4+2GB of RAM, just it stores all the necessary random IOPS bottleneck data into the RAM.
2) The game actually doesn't run any faster or better than when I cached it with system files, using only maybe 1GB of RAM for TS3 files. So as most caching articles said, after a certain "threshold" increasing the cache will yield very little improvement.
3) The only time it does massive loading is of course during boot up, this is where most people notice changes after switching to SSD. Of course in actual gameplay, faster SSD in both seq and random will increase the game performance. However, from the test above, you really only need 1GB of it, not the whole game.
1) If you can add a SSD, putting TS3 on there will improve your game. This is all agreed. However, if you can't add a SSD (for whatever reason) but you have extra RAM, you can use a caching program and devote 500MB to 1GB solely to TS3 and it would improve your game performance b/c it removes the slow 4k random read into RAM cache instead. Are there free caching programs out there? I don't know, I didn't really do a search. However, it needs one that is customizable, letting you only put TS3 on there.
Now this is a PoC summary, please don't go into "well just upgrade your HDD to 2.5" SSD install Windows" discussion. That's is tangent to this article and not everyone wants to do that. I can give you both pros and cons on that topic. It is NOT for everyone.
This part is pretty experimental. I just downloaded Primocache for 30 days trial. The concept of this caching is vastly different than eBoostr. There are both pros and cons to this in comparison to eBoostr, but of course what we're interested in is how does it benefit TS3.
1) So back to eBoostr, setting the whole cache to just TS3 is actually not more beneficial than simply caching the whole system. Because TS3 is a program running in your system, which still requires your OS resources. Many of those resources require to be cached to give your whole system an optimal performance when running TS3. That being said, the best setup for eBoostr cache is simply caching your whole system with TS3. I would estimate 1-2GB would give you optimal performance without redundancy. This varies greatly depending on how many programs you've installed and how often you use them and what kinds of programs. For example, I actually do not cache my Starcraft 2 b/c besides the loading screens, the game doesn't require constant loading. So with only 2GB of RAM to spare, I rather give it to system files and TS3. This customization is what makes eBoostr superior to Primocache.
2) So Primocache is a very different kind of caching. It lets you set up Level 1 RAM cache and a Level 2 disc cache via SSD. Now it is far more restrictive in sense what you can cache. You can designate which partition to cache, but not which file directory. It comes with smart algorithm too, learning which programs you use more and cache. You can set up a huge say 32GB SSD cache and then use a 2GB RAM level 1 cache. Then it will store caching files in the SSD and then transfer to Level 1 RAM when it needs to and when you call for it. It does store data in the level 1 RAM, but only when it is not full. If it is full, it will flush the data for new ones.
The level 1 and level 2 cache usage means it uses the RAM more like a buffer. That is both pro and con, depending on what you want to do. For TS3, the level 2 cache is unnecessary as you can just install the whole game on a SSD, what we're interested is level 1 cache. Since it cannot devote the Level 1 cache solely to TS3 or even stored data, since it flushes it when full, then it may not be as efficient. I don't know since I'm just starting to use it.
eBoostr also allows you to create a 2nd cache using a SSD, such as 16GB or more. But it didn't say whether it uses it in level with the RAM cache or simply "together." I suspect it still does to a varying degree b/c it does a speed test on the storage devices, so I assume it stores files used more commonly on the RAM. But again, that means you might get more system files stored while TS3 files are flushed to the SSD cache, which is not what we want. There is one notable advantage here also for eBoostr is that you can adjust the size of the cache easily while Primocache requires you to format the whole SSD into unallocated before you set the size for the cache. Once the size is set, you can divide the cache into various smaller cache for different partitions if you want and also use remaining unallocated space for a regular SSD. But, shall you desire to change the size of the whole cache again, you need to reformat the SSD.
3) Primocache differs from eBoostr in one dramatic way, it does write caching as well. While this improves your overall system, in fact makes your HDD runs as fast as a SSD with write-caching, this does not benefit TS3 at all since you can put TS3 user folder on a ramdisk. Now if you don't have enough RAM for a ramdisk and you have no SSD, then using a 1GB level 1 RAM cache with Primo as both read/write will help improve your game, even the user files. This would be the only advantage so far I can see with this program over eBoostr for TS3. However, again that would be shared with your whole system cache. Is that more beneficial than to simply create a ramdisk of 1GB? Which in the tutorial, I was able to reduce a cache down to 600MB. So this is the experimental part, but there are already a lot of free ramdisk software out there while Primocache is a commercial product. It should also be noted that Primocache requires a substantial amount of RAM overhead, 1GB for 4k blocks. It can be reduced greatly by reducing the block size down but that reduces efficiency.
So far, Primocache is much better for your whole system caching. It does both read and write and it uses the ram cache as a buffer rather than pure storage. Now again, for people who don't want to install an OS on their SSD or replace their HDD, this is a good option in using some part of the SSD as a huge level 2 cache. However, even if you have only a SSD and you put your OS on there, this is still a good product b/c it uses ram cache level 1, reducing wear and tear on your SSD.
Now again, I don't really want to get into it, but there are pros and cons of installing OS on a SSD, it all depends on what you want to do with it. Newer technologies like TRIM, garbage collection, etc... extends the SSD life, also they are cheaper now for larger size to OP. But at the same time going from SLC to MLC then to TLC greatly reduce the write cycle down multitudes times. This is yet another thing to consider whether you want to use SSD write caching for your HDD via either Intel RST, Primocache, etc... It will reduce the lifespan of your SSD.
SLC = 100,000 write cycles
MLC = 3,000 (35,000 with 3D NAND)
TLC = 300-1000 (1500-3000 with 3D NAND)
I think when you get a SSD, you have to decide upfront if you want to use it for longevity or as a cache flash. But if you install Win OS on your SSD, I highly recommend creating a ramdisk and move a lot of Win OS writing activity to the ramdisk b/c contrary to a lot of people saying how you can write 20-50GB a day to your SSD and it will last 10 years? Well according to my own data logging, it wrote 2.5GB to my OS HDD within a matter of a couple of hours and I didn't even do much other than surf the web and I already have a lot of stuff moved to the ramdisk. I'm continuously researching and trying to find culprits doing unnecessary writing to my HDD.
1) Move all your browser cache to ramdisk. These things are horrible. And a lot of them these days do background caching that most people don't disable by default. They call it smart surfing, I don't know. It caches and updates websites you frequent, so when you go there it will have the preloaded. That maybe good, but even when you're not visiting them, it gets loaded. Also, they do constant update on tabs, like every 15 seconds it replaces the data. Firefox originally had a terrible algorithm and it was killing people's SSDs, you can google the report on this. But even so, people reported Chrome does the same thing, up to 50GB a day! I'm sure they've updated and fix some of these issues as SSD becomes more popular. However, generally speaking, put your browser cache on a ramdisk!
2) Move your temp folder to a ramdisk! Again, don't underestimate what your OS and other programs do. You may look at your temp folder and see 1GB of crap and think that's it, but they do clean the temp folder and reuse it again and again, so you may be writing tons of GB to the temp folder without knowing it. Likewise Photoshop and other programs uses the temp folder as a scratch folder, it will keep caching as you use them. Contrary to popular belief however, your pagefile is fine on your SSD. According to MS's own report, the read/write ratio is 40:1 so it is quite safe. However, even so what I do is that I have a large ramdisk and I set a pagefile on there that's 16MB but can grow larger to fill the need. Yeah, people say if you have enough RAM, you don't need one. That's completely false. Some programs require it regardless of how much RAM you have, such as Adobe. It may just be a small amount, which is why I set it to 16MB and just let it grow on the ramdisk when it needs it.
3) This one I just found out recently. Win OS is vicious when it comes to meaningless logging. It logs every retarded event on your system. The log files may not seem big, some as big as 1mb, but when you get thousands of write per application a day reporting nothing but "the process has started normally." etc... It wears and tears your SDD.
Many of them can be disabled. But it is tedious as you have to go through each one. Run "Event Viewer" and under "Application log" you can see many of them and it allows you to clear and disable them. I ignore the ones under 100 events, but some of them run as high as 50k logs. Disabling them also reduces CPU usage to log them so this is good if you have the time to go through it. However, some of them cannot be disabled and they still write a lot of it.
DO NOT use junction link to move the "C:\windows\system32\Logfiles" folder to ramdisk. Task Scheduler among other programs require it to be synced. You can, however, use junction link to move "C:\windows\system32\winevt\Logs" to your ramdisk, provided you do this during safe mode and disabling "Windows Event Log" service first. Turn the service back on after you created the junction link. This service is needed for Task Scheduler and other programs.
4) For TS3, a lot of info here may or may not help you. I suspect it will help most players with older systems or limited resources. However, much of it is also to help those who upgraded to a primary SSD in either improving the game performance or reducing wear and tear on the SSD via ramdisk or ramcache.
For players that cannot opt for a SSD but have RAM to spare, your best option is to create a ramdisk for the user folder and if possible use a ramcache either for TS3 or your whole system. It would cost you very little in terms of RAM sacrifice, so it falls under the previous link on "how to make TS3 use more RAM." However, again I haven't found any caching program that are free so, it is something you may want to invest in if it benefits your system. When using RAM caching, TS3 doesn't require more than 1GB before performance increase gain is reduced and you're just wasting RAM. 500MB to 1GB seems to be the optimal sweet spot.
Btw, who is the idiot that gave a disagree to a knowledge base article? Low class kids.
Anyways, I was going to do a full 30 day evaluation, but after 48 hours, I think I got all I wanted to know about this product.
First the conclusion is that for practical usage, it is not as good as eBoostr and I'm going to explain that here. I like the practical concept of using level 1 RAM cache + level 2 SSD cache however the implementation cost too much, and I'm not talking about the money.
Primocache is basically Intel Rapid Storage Technology for "non-RAID" pure software implementation. I actually don't need it b/c I have RAID set up, but I was testing it for someone else. Like the RAID counterpart, it has 2 issues that is often overlooked and not considered.
1) The RAM overhead is much too high. To get the most efficiency out of the technology, you want it to match your typical storage device at 4k blocks. That estimated to a whopping 1.1GB of RAM overhead for just 32GB of SSD cache + 500MB of RAM cache. PoC is that if you have a lot of RAM to spare, this is a great idea to speed up your 1 TB HDD with a 32GB SSD. However, if RAM matters to you, that is a heavy price to pay.
Without the RAM cache, it leaves way more wear and tear on the SSD and of course performance reduction. 500MB seems bare min, but I would say at least 1GB would be suffice. It means that during write-cache, any files under 1GB will not use the SSD cache, but only RAM. This drastically reduce the SSD wear and tear.
So we're looking at 1GB RAM + 1.5GB overhead for a 32 GB SSD cache, 2.5GB RAM sacrifice that I could think of better usage.
2) Wear and tear is a lot more than what most people realize. Again, if you think you won't write 50GBs a day, you have not done your homework. Win OS itself constantly writes and caches, this is why it is so important to move those cache files I mentioned earlier. However, what most people don't realize is that data are not written in single tries. In real life application it is called write amplification, data is written several times for redundancy and write consistency, data verification. It is estimate a data is typically written 5 times from a user's single application write.
So If you write 10GB a day, the actual real world write cycle is 50GB. If you have browser cache and temp files on your HDD, we're looking at 100-200GB real world writing a day.
Now general SSD usage as I've mentioned before has a lot of new tech to help spread the wear, this is why the larger your SSD, the more TBW it is. A typical 250GB SSD is 150TBW. But nobody is going to use 250GB for flash cache. Not only is it impractical, but remember the RAM overhead? That's a heavy cost. IRST max is 64GB, and that is looking at a good 2-3GB RAM overhead.
With that said, it means your SSD wear and tear is calculated only by what you reserve for the cache. If you reserve 32GB from a 256GB SSD drive, the wear and tear from caching is restricted only to the 32GB cache partition.
So we're looking at:
(3000 write cycle MLC * 32 GB capacity) / (5 factor write amplification * (50GB per day * 365 days a year)) = roughly 1 year.
If you bought a cheap 32GB mSATA for $15, that's not bad at all for a year of wear to bring your HDD into SSD performance. However, if you are using a 256GB and losing 32GB of it in 1 year... and this part is where my research ends, I don't know how this affects the whole SSD. You may just lose that partition or you may lose the whole thing.
Now of course, all this is estimates, everyone's usage is different and endurance and lifespan of SSD also involves other factors, but this one is calculated purely on write cycle and estimated daily write. If you stream movies or tv shows, all that cache data is going to your temp folder so if it is on the SSD every GB written is actually written 5 times in real application. Youtube, Tiktok, etc... all wear and tear your SSD and your browser constantly caching along with your Win OS. So average user could easily accumulate 20-50GB of writing a day.
3) Now this is the one most TS3 related and I cannot be sure if Intel RST has the same issue: CPU usage. I don't know whether because it is pure software base and not RAID based, it requires some CPU usage. Granted it is not a lot and hardly noticeable for the average user. However, my idle temperature was up 5-7 degrees. Again, if you're surfing the web, watching Youtube, you won't notice anything. But we're talking about running TS3, where in my game it is average 70-75*C on a typical run and up to 80*C+ when running heavy like Consort Age mod 1AM or using MC global item seach. With this cache going, it easily goes up to 88-90*C. This was the final deal break for me.
Primocache is great for people with a lot of RAM to spare and want to use a spare flash or SSD to cache and make the HDD run like a SSD. In fact it runs better because it also allows you to use RAM cache. However, in applications where people have heavier usage it is not recommended b/c of the CPU usage, increasing your overall temperature by 5-7*C is a heavy cost. Also, it pretty much means the SSD you use for the cache will have a severe life reduced, again depending on how heavy you use it and how much RAM cache you can allocate.
The only single benefit this has over "read only" cache such as eBoostr is the write-caching. So this is great for people who do a lot of writing in their projects. But with SSD price dropping, again most people will just get a TB or 2 SSD to do that, there is no reason to use a 32GB flash to achieve the same thing with a HDD. So that basically eliminated the benefit of write-caching with this cache program.
With read only caching, eBoostr is better because:
1) more flexible and customizable
2) easier to create or modify a cache, no need to format
3) It retains the data without flushing since it is not sharing it with writing.
4) No RAM overhead, no excessive CPU usage
TS3 result summarization:
I actually tested the load time with SSD and with RAM caching. In my test? only 10 sec faster because my CPU is the limiting factor.
This means this game loads the same either with the whole game on SSD or whole game on HDD with a 1GB ram caching TS3 files in bits and pieces.
I also ran the game purely without RAM cache on the SSD and then again with the RAM cache, and the performance is pretty identical. It was a little faster and smoother with the RAM cache, but you have to actually pay attention to notice. So this again, PoC: a HDD + 2GB RAM cache (both system and TS3) performs almost identical if not slightly better than whole game purely on SSD.
So again, if you cannot get a SSD for some reason or another, maybe a laptop too old, you can get the same performance increase with a 500MB-1GB RAM cache with software like eBoostr. Again, this is just knowledge base, I don't know of any free software that does the same thing unfortunately.
Just found this really cool free tool. It's primary function is showing a software LED on your monitory indicating if your drive is operative. However, it also shows you a summary of how much read and write is done to the drive during each session (reboot session I suppose).
Here you can get a clear picture of how much is written to your SSD per session. Run it for 24 hours without reboot and you'll get a clear picture of how much written in 1 day. Remember, under write amplification, it is estimated to multiple this by 5 times to get your actual real world daily write.
Sanity is overrated.
Nitromon is a type of Pokemon encountered in the Pokemon Nitrome Version series.
There. Mystery solved.