- Site Map >
- Modding and Creation >
- Sims 3 Creation >
- Modding Discussion >
- Cracking Open Shaders_Win32.precomp? (for better shaders!)
- Site Map >
- Modding and Creation >
- Sims 3 Creation >
- Modding Discussion >
- Cracking Open Shaders_Win32.precomp? (for better shaders!)
Posts: 5,656
Thanks: 1035 in 5 Posts
TS3 could really use some higher values as well. Exterior shadows can easily be modified for higher resolutions, mine are at 8K and that means virtually perfect exterior shadows, but indoors it can be quite blurry.
A short wishlist:
Higher interior shadow level of detail
Higher cubemap resolutions
Extended ambient occlusion functionality
Basic support for global illumination with better radiance from objects
And perhaps entirely unrelated to the shaders: more cloud layers with yet more realistic illumination and perhaps TS4-style world backdrops (notice how clouds are completely different between TS3 and TS4, with both having obvious weaknesses).
( Join my dumb Discord server if you're into the whole procrastination thing. But like, maybe tomorrow. )
Posts: 1,889
Thanks: 686 in 10 Posts
Rest can probably be achieved with external programs... but they are not in the scope of game's engine.
Posts: 3,443
Thanks: 938 in 5 Posts
Well for a start it's good that that value can be increased in TS4. TS3's indoor lighting shaders are one of the game's better qualities, I think, and TS4 doesn't really live up to it in some ways (though the natural light shadows are quite cool). TS3 could really use some higher values as well. Exterior shadows can easily be modified for higher resolutions, mine are at 8K and that means virtually perfect exterior shadows, but indoors it can be quite blurry. A short wishlist: Higher interior shadow level of detail Higher cubemap resolutions Extended ambient occlusion functionality Basic support for global illumination with better radiance from objects |
Personally I'd just like extended shadow draw distances and multi-floor lighting to be fixed. Probably the two main things that will have the largest impact on improving the game's appearance. Higher shadow draw distances will keep the worlds from looking flat, while multi-floor lighting just plain makes exterior and interior spaces better looking, and eliminates the need for invisible lights. These are things that should be given for any modern game engine.
On that note- I wonder if if shaders could be edited, an approximation of TS4's natural light shadows could be done by simply making the light cast by windows more crisp? One other thing worth looking into is to see why fences aren't affected by lights at all.
And perhaps entirely unrelated to the shaders: more cloud layers with yet more realistic illumination and perhaps TS4-style world backdrops (notice how clouds are completely different between TS3 and TS4, with both having obvious weaknesses). |
TS4-style backdrops are really simple- they're just flat planes objects with a scrolling texture. This is entirely possible in TS3 without editing the shaders.
Posts: 5,656
Thanks: 1035 in 5 Posts
( Join my dumb Discord server if you're into the whole procrastination thing. But like, maybe tomorrow. )
Posts: 3,860
Thanks: 8537 in 67 Posts
Here's a great video on how shaders are made:
And materials, are mainly texture-based. They both need each other though to create the final results on a mesh
(See: https://forums.cgsociety.org/t/shad...e-same/927326/6)
Now, coming to your question. Shaders are, unfortunately, all engine based. Given how TS3 has been made with HLSL shaders (and I think GLSL for mac copies IIRC), you can't just go in and edit it, no tool can do it. You'll need to have the development build of TS3 itself to adjust the code of shaders. OR Do what this dude did with another DirectX9 game: https://forums.taleworlds.com/index.php?topic=296795.0
EDIT: Correction, TS3 uses OpenGL But does indeed have HLSL Shaders
But as you can see already, it requires 3 things. 1. C++ knowledge. 2. Knowing how the game engine works. 3. Being good at maths and algebra.
However, most people actually don't really know how much EA's shaders can already do by changing values in the MATD file or even the LITE file. For the LITE I always like to give people this example because even though it's a fixed tutorial, it covers so much on the main bits of, well, ts3 lighting http://poccisims3.blogspot.com/2013...-lightings.html
Posts: 1,889
Thanks: 686 in 10 Posts
Idk if lightmaps and shadowmaps should be considered a shader instance but I'm pretty sure they are engine hardcoded at this point. And ofc if we indeed do not have EA's dev version of the game... there's nothing we can do other than, well, reverse engineer it... which is, from what I can see, not going to happen in long long stream of years to come. A shame really. Sims 3's graphical engine, albeit very old and out of date at this point, is still quite powerful (given by the mere fact that sims 4 is also using it).
Posts: 5,656
Thanks: 1035 in 5 Posts
( Join my dumb Discord server if you're into the whole procrastination thing. But like, maybe tomorrow. )
Posts: 1,889
Thanks: 686 in 10 Posts
God EA can fuck up so bad sometimes.
Posts: 1,889
Thanks: 686 in 10 Posts
- Header with file type and probably also libraries used
- Vertex and Pixel shader data for each shader
- Value Type definition (int, float3...) for each property entry per shader, for all shaders
- Some form of parameter list but I'm not sure if it's actual parameters described previously or something else. From strings it goes like: TECH, PARM, PARM, PASS
- Final list of shaders described in the file
If this is trully the same thing as sims 4's precomp file (which I just compared and they really are 90% the same) then, it should be able to change the existing shaders and add new ones. Because few people managed to actually add new shaders and edit existing ones using the .BT file gurus provided them with.
If I am correct is, how you would do this is you would run binary remplate (.BT) file over shader precomp file and you would get a version that's readable and editable. The thing with sims 4 is that you would be able to have that decompressed file in mods folder work. For sims 3 that is most likely not the case meaning that you'd probably have to compress the file again... which I don't think it's possible.
EDIT: I read Lyra's explanation and I guess she might be right though... that would mean that people who got this to work with sims 4's precomp shader did not actually get it to work? I'm currently in the state of confusion.
I guess I'm hoping too much
Posts: 3,860
Thanks: 8537 in 67 Posts
But removing the binary bits of the hex code and re-save it after the changes is actually pretty easy to program However, this info needs to be the SAME as the info from the engine.
So when you're compiling your code for the game, you obviously want the game to be able to call its textures, shaders, rendering settings, etc. Now I'm not sure if that's common practise perse but in the TS3W.exe as well as the TSLHost.dll, it holds pretty much all the 'directions' of where the game should get its details. They're actually listed like a map structure, which was obviously needed before the game was actually being compiled for release. |
When I referred to this, I basically meant that the actual shader settings are in the TS3W.exe and TSLHost.dll (Although the TSLHost.dll is I think just to contact EA's server with the whole sharing gifts in-game with your friends). The Shader data from there uses variables and references that are defined in the Precomp file we're talking about here. I assume this is because of saving loading time and getting easier access to adding more Variables rather than re-compiling the engine over and over again. Hence the mention of "Floats3, etc"
YOu could actually check out how they did it with Minecraft's shaderlab. Minecraft was supposed to be non-crackable regarding shaders, but I think they reversed engineered it in some ways. I haven't actually checked it out though.
Posts: 5,656
Thanks: 1035 in 5 Posts
( Join my dumb Discord server if you're into the whole procrastination thing. But like, maybe tomorrow. )
Posts: 3,860
Thanks: 8537 in 67 Posts
Some of you might recognise the mashalAs stuff whenever you're working on things. Here's an example:
[DllImport("Sims3Common.dll")] [return: MarshalAs(UnmanagedType.I1)] public static extern bool World_AddObjectToObjectManagersImpl(ulong scriptHandle, ulong groupId, Vector3 position, int level, Vector3 createFacing, uint hiddenFlags);
Now, marshalAs is primarily used to import C++ dll libraries (IIRC). C++ is also usually the preferred language for making engines, because it's the almost closest language to computer language. TS3's engine is also custom, so it makes sense why they'd use C++.
Now, in the engine, if we were to look at the code I just copied in my post, it would loook like this:
ScriptCore.World::World_AddObjectToObjectManagersImpl
Which makes a lot of sense! But if we were to do this with shaders, I think we'd have to make the communication work through editing hex code, so I'd figure making an application that adds the bytes to the .exe file would be better if you were to want end-users (So in our case people who download CC) to use it and safetly install such a thing.
But, of course, currently i'm talking about about functions in the engine And not shader editing :p. But I've got my reasons!
Some useful stuffz:
http://timjones.io/blog/archive/201...-cbuffers-using
https://modthesims.info/wiki.php?ti..._Modding_Basics
Personally, I think i've got myself a lead as to how to add custom materials in a sense like this: https://docs.unity3d.com/Manual/SL-...erExamples.html
But that yet needs to be seen . If I get that working, i'll make a tutorial for it!
Posts: 5,656
Thanks: 1035 in 5 Posts
( Join my dumb Discord server if you're into the whole procrastination thing. But like, maybe tomorrow. )
Posts: 3,860
Thanks: 8537 in 67 Posts
TS4 only has functions calling the shaders. That's it. Ts3 actually has genuine functions as to how the shader should act, be, do, etc. Hence why you'd see references like "Tech" and "Pass". So those are here for a reason Else we'd have been able to find a JSON file or something that TS4 has to be able to customise it without cracking open that file over and over again and recompile the engine every single time, given how they are loading in a list. But that's TS4, not TS3
There are about 604 shaders in the 1.67 patched game. Each have their own "chunk" So to speak. They're string-wise, only mentioned twice. (For example, search for PreviewWallsAndFloors, and you see what I mean). Now, you of course got the FNV32 string version of the shader's name as well and that one is mentioned plenty of times if you were to search it. The first mention of say PreviewWallsAndFloors, in the SSET for the shader, happens to have some garbage bytes and a list of some sort. The end bit of the bytes chunk, however, has the naming of the values (which I presume you can customise to something else? As long as it's done in this file) It contains the default settings of a shader param, but also a "Min" and "Max" value. For textures this is obviously a bit different.
Later on, if we use the FNV32 Hash of PreviewWallsAndFloors, The FNV32 hash is mainly used as an "ID" for the "TECH" Which makes a lot of sense: https://docs.microsoft.com/en-us/wi...ques-and-passes
Just like how we saw that SSET has it's own "Shader" chunk of bytes, so do the functions, which is with a tag called "SBLK" Which stands for "Shader block". See this as a wrapper for the shader functions
Later at the bottom, you've got ever shader + shader data fields, which is probably for initialising everything.... Because, EA :p
So the file is actually simple af, it's just that there are 604 shaders which makes it super big and hard to read. But this is as far as Tash and me got to reading the file. We're still working on it though!
Posts: 5,656
Thanks: 1035 in 5 Posts
( Join my dumb Discord server if you're into the whole procrastination thing. But like, maybe tomorrow. )
Posts: 231
Thanks: 1024 in 5 Posts
I've reordered them from [DWORD unknown1, DWORD unknown2, DWORD stringLength, String shaderName] to [shaderName, DWORD unknown1, DWORD unknown2].
unknown2 is 0 in all entries.
All values were read in Little-Endian but were written to the file flipped to big-endian. (Thanks java)
shadernames.txt (52.7 KB, 60 downloads) | ||
Description: All values were read in Little Endian |
Posts: 3,860
Thanks: 8537 in 67 Posts
I take it this is from the mKeyNameTag bit right? (Aka the bit at the bottom, with alll the strings)? I noticed that has quite a mixture of "Shaders" and "Shader params", given how they're basically strings that are being used for reading purposes.
Anyways, for the programmers out here, Tash and me made a Byte-reading template (Which was actually done quite easily by using ts4's bt file. Hence why it says that at the start still :p) We've edited in such a way that you can now read it for ts3. One note though is that for TS3, the mheapsize is being used for the shaders, whereas for TS4 they are... well, ignored. So while it would still be possible to edit it by reading the offsets, just so that you guys are aware of this.
So far, i'm still trying to figure out the linkage of bits. All I need to figure out now is just what the mplatform entries do, and how the mVertexShaderParams as well as the mPixelShaderParams communicates with the game/package files (Or well, more specifically MLOD/MODLs). So you can use that as a reference.
I used 010 editor to run the template, so make sure you've got a program that can run templates (For 010 editor, see: https://www.sweetscape.com/010editor/manual/Running.htm )
I've also included my research on what I got, which is a "Follow the DiffuseMap through the Phong shader!". mPlatform stuff however isn't included yet.
ShaderInfo.txt (6.4 KB, 52 downloads) | ||
Description: My research on the topic, way too long for a post so here you go! | ||
Sims3_Shaders_Win32_Template.precomp.zip (2.9 KB, 37 downloads) | ||
Description: Template that you can run with 010 editor or any bytes editor. |
Posts: 231
Thanks: 1024 in 5 Posts
I was wondering why my generated FNV hashes weren't matching with what I suspected were hashes, turns out my implementation is just faulty, might be because of java's lack of unsigned integers. Should have just checked them it in S3PE to begin with.
*Edit*
Looks like I missed the bold text on the wiki stating that the string should be converted to lowercase first before hashing. Now it makes sense.
Posts: 3,860
Thanks: 8537 in 67 Posts
Currently been working on a small program that can disassemble the "mByteCode" in the VSHD/PSHD section. (see template code) With D3DCDissasembleShader. Mainly because... curiousity and seeing how far things can be edited
Posts: 109
SSETRabbitHoleHighDetail Num QWords: 3 Length of list in dwords: 104 Total size of list is: 416 Num OPTs: 10 D9FD4034 OPT_SHINY 1 0 2 F2477C6F OPT_SWITCH_SHADOWMAP_TYPE 2 0 3 136B6D20 OPT_ALPHATESTENABLED 6 0 2 96DE15F6 OPT_SKINNING 12 0 2 F8EF773B OPT_FADE_ENABLED 24 0 2 E09BAB91 MAP_NORMAL 48 0 2 8534219B MAP_SPECULAR 96 0 2 9F2A6F85 MAP_EMISSION 192 0 2 7CB9E69F OPT_THICKSNOW 384 0 3 DEF23ADB OPT_FROST 1152 0 2 num Shader Fields: 20 RabbitHoleHighDetail RabbitHole High Detail Data size length: 432 At position: 1322 diffuse 0 8 0 100 3 1 1 1 DiffuseUVScale 20 36 0 100 2 1 1 specular 44 56 0 100 3 1 1 1 shininess 68 80 0 100 1 10 SpecStyle 84 96 0 100 1 2 FresnelOffset 100 116 0 100 1 0.20000000298023 NormalUVScale 120 136 0 100 2 1 1 NormalMapScale 144 160 0 100 1 1 NormalMapUVChannel 164 184 0 100 1 0 DiffuseMapUVChannel 188 208 0 100 1 0 AmbientMapUVChannel 212 232 0 100 1 1 EmissionMapUVChannel 236 260 0 100 1 0 AlphaMaskThreshold 264 284 0 100 1 0 EmissiveLightMultiplier 288 312 0 100 1 1 EmissiveBloomMultiplier 316 340 0 100 1 1 AmbientOcclusionMap 344 364 0 100 7 0 DiffuseMap 368 380 0 100 7 0 NormalMap 384 396 0 100 7 0 EmissionMap 400 412 0 100 7 0 SpecularMap 416 428 0 100 7 0
Story books are full of fairy tales, of Kings and Queens, and the bluest skies.
Posts: 156
Posts: 3,860
Thanks: 8537 in 67 Posts
So we found out that the ghost state is shaders, fairy wings, the visual effect that is with all the effects in the file is Sims3effects, but what are visual overrides, they are also in the same place as the shaders? |
Visual overrides are actually overrides done on effects . So, a great example could be "no more confetti!". Now, some effects can be changed in colours through creating a script mod out of it. Or, you could use this tool and method: http://www.modthesims.info/showthread.php?t=450194
However, if we take the confetti visual effect as an example (and this goes quite far from what you asked), this does need a mesh and shader of course. Else, the confetti will not have 1. transparency support. 2. the way the light is defined on the confetti. So, how EA fixed this was by adding a "Particle" shader. This is basically the shader for anything confetti-like or whatever "falls" as an effect
Hope this clears it a bit! I'd recommend though if you want to know more about visual effects and editing them to start a new thread. mainly because this is about cracking open the Shader pre-comp file, and not editing visual effect.
Posts: 5,656
Thanks: 1035 in 5 Posts
Sim Hair does not cast shadows. Hair also has transparency issues when rendered against certain other shaders like the glass ones. Could you explain, by the way in which shaders are used to render Sim hair, why these issues occur?
( Join my dumb Discord server if you're into the whole procrastination thing. But like, maybe tomorrow. )
Posts: 231
Thanks: 1024 in 5 Posts
Interesting that you should mention transparency support, because that reminds me of hair. Sim Hair does not cast shadows. Hair also has transparency issues when rendered against certain other shaders like the glass ones. Could you explain, by the way in which shaders are used to render Sim hair, why these issues occur? |
That happens in most rendering engines, alpha calculations and shadow casting are both expensive rendering techniques. There's in general 2 types of alpha, smooth alpha with a gradient from fully opaque to fully transparent, or the vastly cheaper options, visible or not visible with a certain cutoff point in the alpha value.
The second option usually does cast shadows in rendering engines as it's not much more expensive to calculate than a fully opaque object, you either you render it to the shadow map or you don't. With a gradient alpha you'd have to calculate the opacity of the shadow for every pixel too.
Who Posted
|