Hi there! You are currently browsing as a guest. Why not create an account? Then you get less ads, can thank creators, post feedback, keep a list of your favourites, and more!
Replies: 91 (Who?), Viewed: 22962 times.
Page 3 of 4
Quick Reply
Search this Thread
AGuyCalledPi
Mad Poster
#51 Old 25th Jan 2020 at 8:02 PM
I'm currently playing Transport Fever 2 a lot, which despite a fairly fancy graphics engine still dithers its alphas. So if you use the cinematic camera to zoom in far enough, you might start to see some windows that aren't transparent at all but checkered......and there's also a fun rendering issue where if you look at a vehicle through glass, it turns invisible. Ever seen 200 people travelling on Superwoman's invisible train? It's a sight.

Funnily enough every video game, no matter how recent or big-budget they may be, has issues like that if you look closely enough.
insert signature here
( Join my dumb Discord server if you're into the whole procrastination thing. But like, maybe tomorrow. )
Advertisement
jje1000
Mad Poster
Original Poster
#52 Old 29th Jan 2020 at 5:27 PM
Quote: Originally posted by AGuyCalledPi
Funnily enough every video game, no matter how recent or big-budget they may be, has issues like that if you look closely enough.

I'd say that there's usually a 'good' enough for graphics where all the bases are more or less covered and any advances usually are to sweat out the finer details.

The issue with TS3 is that while it still holds up well in some regards, it falls quite short in others (i.e. defective hair shine, skin shine, multi-floor lighting [broken since WA], shadow draw distance).

It's frustratingly close, yet so far.
AGuyCalledPi
Mad Poster
#53 Old 29th Jan 2020 at 10:31 PM Last edited by AGuyCalledPi : 29th Jan 2020 at 10:43 PM.
I'd have to disagree with that, TS3 falls well short. Texture resolution, polygon budgets, (the lack of) postprocessing, primitive shaders and lighting with poor dynamic range.....anno 2020, TS3 looks flat, lifeless and blurry. Mods can take care of a fairly large part of the more visible limitations, but the fact is that the engine is showing its age. It's a decent-looking game for 2009 but I wouldn't say it holds up well. Especially not with the dreadful graphics you get out of the box.

And that's one of the things you will run into when modding any game. You can improve textures all you want, you can enhance models all you want, you could even inject new shader code to add realistic effects. But you can't do all three. Ever seen Quake or GTA San Andreas with RTX? It's a masterclass in advanced turd polishing.

Anyway - that's not to say I wouldn't be thrilled to see my custom 8K shadow maps render properly across lot boundaries et cetera et cetera. And let Lyralei have her silky smooth blanket of snow already!
insert signature here
( Join my dumb Discord server if you're into the whole procrastination thing. But like, maybe tomorrow. )
jje1000
Mad Poster
Original Poster
#54 Old 29th Jan 2020 at 10:50 PM Last edited by jje1000 : 29th Jan 2020 at 11:05 PM.
Quote: Originally posted by AGuyCalledPi
I'd have to disagree with that, TS3 falls well short. Texture resolution, polygon budgets, (the lack of) postprocessing, primitive shaders and lighting with poor dynamic range.....anno 2020, TS3 looks flat, lifeless and blurry. Mods can take care of a fairly large part of the more visible limitations, but the fact is that the engine is showing its age. It's a decent-looking game for 2009 but I wouldn't say it holds up well. Especially not with the dreadful graphics you get out of the box.

And that's one of the things you will run into when modding any game. You can improve textures all you want, you can enhance models all you want, you could even inject new shader code to add realistic effects. But you can't do all three. Ever seen Quake or GTA San Andreas with RTX? It's a masterclass in advanced turd polishing.

I don't think anyone doubts the engine is showing its age, but the fact that some of the graphic engine's elements are broken or missing just requires you to take some large suspensions of disbelief, and really casts the game (which does have some beautiful moments and decently detailed models despite its mediocre art direction) in an unfair light.

And of course in the bigger picture, you aren't really playing TS3 because it's a top-of-the-line graphics engine. We play it because it's the only easily-moddable third-person open-world click-and-point life simulator out there There are plenty of gaming communities that stick to games with outdated graphics largely because of the gameplay.
Slamyy
Lab Assistant
#55 Old 31st Jan 2020 at 3:48 AM
What's the link between shaders and objects? For example, what tells the game to apply ghost shader with certain settings to a ghost sim?
SmugTomato
Field Researcher
#56 Old 31st Jan 2020 at 12:48 PM
Quote: Originally posted by Slamyy
What's the link between shaders and objects? For example, what tells the game to apply ghost shader with certain settings to a ghost sim?


That's what materials are for, a material is an instance of a shader with certain values set for each parameter.

For example; A simple shader can be made to display an object in a single solid color, this color can also be passed as a parameter instead of being hard-coded.
A material could then be created for a blue and a red color(or any other color), both will use the exact same shader code but they use different parameter values.
Battery
Space Pony
#57 Old 31st Jan 2020 at 2:44 PM
Well here are some more concrete examples

Set the Visual Override of a Sim to the Alien one
Code: 
World.ObjectSetVisualOverride(this.Target.ObjectId, eVisualOverrideTypes.Alien, null);


Set the Ghoststate of a Sim
Code: 
World.ObjectSetGhostState(this.Target.ObjectId, 4u, (uint)target.SimDescription.AgeGenderSpecies);

4u is deathtype Electrocution while 0 would be a non ghosty appearance
Slamyy
Lab Assistant
#58 Old 31st Jan 2020 at 5:52 PM
Quote: Originally posted by SmugTomato
That's what materials are for, a material is an instance of a shader with certain values set for each parameter.

So where the materials for ghost shader are set? Are those in the precomp file?
And Battery, I was asking the step that comes right after that; the part where the game says "since this object is an electrocution type ghost, I should set the diffuse to yellow".
SmugTomato
Field Researcher
#59 Old 31st Jan 2020 at 9:06 PM
Quote: Originally posted by Slamyy
So where the materials for ghost shader are set? Are those in the precomp file?
And Battery, I was asking the step that comes right after that; the part where the game says "since this object is an electrocution type ghost, I should set the diffuse to yellow".


You wouldn't find any materials in the precomp file, materials would be stored in .package files. Probably as MTNF or MATD chunks of some sort, those have parameters like how shiny an object should be or what textures they use.
The precomp file would contain the code telling your computer how to render a ghost, the material in some .package file would tell it what color to use and which extra effects to possible use.
As for where these materials are actually stored/set, I have no clue. My guess is it would have to be in one of the FullBuild*.package files and which specific version to use would be decided by the game code depending on how a sim died.
Lyralei
Virtual gardener
staff: administrator
#60 Old 2nd Feb 2020 at 11:52 AM
Quote: Originally posted by SmugTomato
You wouldn't find any materials in the precomp file, materials would be stored in .package files. Probably as MTNF or MATD chunks of some sort, those have parameters like how shiny an object should be or what textures they use.
The precomp file would contain the code telling your computer how to render a ghost, the material in some .package file would tell it what color to use and which extra effects to possible use.
As for where these materials are actually stored/set, I have no clue. My guess is it would have to be in one of the FullBuild*.package files and which specific version to use would be decided by the game code depending on how a sim died.
Ghosts are a bit... interestingly done regarding colours and the way they should appear. Some are defined by the noise map or even the reflection of the normal map. The rest are indeed defined in the SWB_. If you were to (if S3PE doesn't crash for you to begin with :p) open the fullbuild SWB_ package, then Click "Grid", find the ghost effects and change the colours from there  This should be under EffectSections > Items > Item > ColourDelta

So, indeed, not everything is defined in the Precomp file. Some things like the vampire glowy eyes can be changed through editing the SWB. Now, one thing, you don't actually need the SWB cloner tool for this, that's only useful if you're making your own Script mod that happens to need an effect
THA
Lab Assistant
#61 Old 22nd Mar 2020 at 2:02 PM
Any updates?
Lyralei
Virtual gardener
staff: administrator
#62 Old 22nd Mar 2020 at 2:09 PM
yes and no :/

So, it's possible to, through hex editing, edit the bit where you see all the other options. (See: http://www.modthesims.info/showthread.php?p=5603975#post5603975 )

But then the next issue occurs and that's adding newly written shaders. Currently, you can only edit new ones with different default values. And honestly, that already is possible if you were to edit an object yourself, so that part feels almost pointless to make a tool out of.

Personally haven't really looked into it too much, but I might after I finish the current project i'm working on.  
Lyralei
Virtual gardener
staff: administrator
#63 Old 11th Jan 2021 at 11:54 AM
So, by pure accident, I actually just found out that TS3, TS4 and My sims uses the exact same shader layout. Now ts3 and 4 we knew, but what's great for this type of research, is that the MySims's shaders aren't packed inside some 'precomp' file. Instead, they're, well, fx/shaderparameters/fxh files. (if you have the PC versionthe path is: MySims\SimsRevData\GameData_Win32\Shaders) I also attached them to this thread

However, it still begs for the question of how we'd edit the very first bit of the precomp file, where the shaders are already pretty much cached and ready to be used in-game.
Attached files:
File Type: zip  Shaders.zip (131.4 KB, 23 downloads)
Lyralei
Virtual gardener
staff: administrator
#64 Old 13th Jan 2021 at 1:03 PM Last edited by Lyralei : 13th Jan 2021 at 1:30 PM. Reason: Added helper functions and other stucts for better readability.
Not related to my previous post, but I have done some other research that could help this!  CAW as an application is the only sims 3 thing that has Sims3Common.dll as an external dll. For TS3W.exe that's not the case. However, the CAW one also uses shaders to compile and reads from a precomp file. Now, I ran Sims3Common.dll through snowman: https://derevenets.com/ And it did do an allright job extracting code. 
Here are some snippets, warning, because it's programmatically decompiled, the variables are pretty machine-like:






But overall it's relatively easy to read. I'm just not quite sure yet if this is just *progressing* the shader data or actually reading from the precomp file. And, where some functions with params are being called from, since they must be called from somewhere. Will update this post if I know!
AGuyCalledPi
Mad Poster
#65 Old 13th Jan 2021 at 5:37 PM
Reads like a shader alright.
insert signature here
( Join my dumb Discord server if you're into the whole procrastination thing. But like, maybe tomorrow. )
Lyralei
Virtual gardener
staff: administrator
#66 Old 17th Jan 2021 at 11:56 AM
Quote: Originally posted by AGuyCalledPi
Reads like a shader alright.
Lol it does doesnt' it? but this seemingly is more code that deals with shader information for objects rather than a shader itself.

But I did want to say that I'm currently converting the .bt file, that is attached here, into a program that can at least make some easier sense of the whole hex code stuff. And hopefully making it possible to edit it eventually  
jje1000
Mad Poster
Original Poster
#67 Old 23rd Jan 2021 at 12:30 AM
There is one thing I think would be worth looking into regarding shaders, which is possibly looking at how shaders respond to lights.

An example:


- The movie screen uses the Landmark shader, so it doesn't respond to any street lights.
- The Uni sign uses the Phong shader, so while it responds to light, it only responds if the center of the object is near the light source (see how if it's farther away, it doesn't respond at all).
- The bookstore rabbithole uses the Rabbithole shader, and as such, is the only object that properly and contexurally responds to light sources.

I wonder if it's possible to transfer this behaviour between shaders? It might even solve the issue of counters and other objects (which use the Phong shader) looking choppy under lights.
Lyralei
Virtual gardener
staff: administrator
#68 Old 23rd Jan 2021 at 10:52 AM
Quote: Originally posted by jje1000
There is one thing I think would be worth looking into regarding shaders, which is possibly looking at how shaders respond to lights.

An example:


- The movie screen uses the Landmark shader, so it doesn't respond to any street lights.
- The Uni sign uses the Phong shader, so while it responds to light, it only responds if the center of the object is near the light source (see how if it's farther away, it doesn't respond at all).
- The bookstore rabbithole uses the Rabbithole shader, and as such, is the only object that properly and contexurally responds to light sources.

I wonder if it's possible to transfer this behaviour between shaders? It might even solve the issue of counters and other objects (which use the Phong shader) looking choppy under lights.
Hopefully! I was talking on discord about this issue (Although not necessarily how shaders react on it but just how badly the lights have changed over the years from the first patch till today's patch). Because I remember the days where light *used* to travel to different levels on a wall. Which now isn't the case anymore.
jje1000
Mad Poster
Original Poster
#69 Old 23rd Jan 2021 at 2:32 PM
Quote: Originally posted by Lyralei
Hopefully! I was talking on discord about this issue (Although not necessarily how shaders react on it but just how badly the lights have changed over the years from the first patch till today's patch). Because I remember the days where light *used* to travel to different levels on a wall. Which now isn't the case anymore.


Definitely this! That was an effect that was broken in World Adventures when they first introduced basements.

Strangely enough, street lights outside the lot can still cast light correctly across several levels.
bubblegumfish
Lab Assistant
#70 Old 8th Mar 2021 at 3:53 AM
Now that we have the reworked and improved lights mod by simsi45, i'm hoping the shader decompilation happens soon.
Lyralei
Virtual gardener
staff: administrator
#71 Old 10th Mar 2021 at 11:49 AM
Quote: Originally posted by bubblegumfish
Now that we have the reworked and improved lights mod by simsi45, i'm hoping the shader decompilation happens soon.
Still working on that program that makes us edit the Shaders! So we'll probably get that soon too  I'll post the progress of that here soon as well, but time isn't exactly on my side as much because real life :p
Onnya
Test Subject
#72 Old 30th Mar 2021 at 12:36 AM
Quote: Originally posted by Lyralei
Still working on that program that makes us edit the Shaders! So we'll probably get that soon too  I'll post the progress of that here soon as well, but time isn't exactly on my side as much because real life :p


I really hope your program becomes a real thing! You're super talented :D
epochtimestamp
Test Subject
#73 Old 30th Jun 2021 at 11:49 PM
I was also looking into cracking the shader precomp file. I wrote a tool that could parse it and pull out the shader names, parameters, etc...
The problem I ran into was that any modification to the precomp file causes the game to crash on startup. Even innocuous changes, like changing a letter in the ShaderSet's UI parameter name description. Since that's a description it shouldn't have any real effect on the game, and yet the game crashes on startup.
Similarly, if you try to replace the precomp file with an older version that EA shipped, the game crashes on startup.
My hunch is that there is a checksum being done on the precomp file. If it is changed in any way, the values don't match up, and game refuses to start.

Have you found a way around this problem?
Lyralei
Virtual gardener
staff: administrator
#74 Old 1st Jul 2021 at 2:02 PM
Quote: Originally posted by epochtimestamp
I was also looking into cracking the shader precomp file. I wrote a tool that could parse it and pull out the shader names, parameters, etc...
The problem I ran into was that any modification to the precomp file causes the game to crash on startup. Even innocuous changes, like changing a letter in the ShaderSet's UI parameter name description. Since that's a description it shouldn't have any real effect on the game, and yet the game crashes on startup.
Similarly, if you try to replace the precomp file with an older version that EA shipped, the game crashes on startup.
My hunch is that there is a checksum being done on the precomp file. If it is changed in any way, the values don't match up, and game refuses to start.

Have you found a way around this problem?


That was something I've noticed as well myself. So far, I think the shaders are cached somewhere. (TS3W.exe maybe?). All I've been able to do personally is tweak the parameters each shader comes with (Say, phong shader has a default of 20 on shininess IIRC, changing that to 30 won't crash the game)

So I don't think currently we've gotten to a state where we're able to *add* a new shader. Just editing parameters.

Now, I doubt this will do anything and I haven't tried it myself, but in ts4 forums, what one of the devs mentioned (or Guru rather) was to leave the precomp in the game files alone, but the 'new' one, you put in a package and then put it in your mods folder and the game should pick up on it then. I've personally seen this working with existing ini files that are outside of the game, and putting those in a package file seems to make some changes.

But that's as far as I got :/ I know some other dude did decode the shader precomp and turned it into a program, but according to him, it wasn't possible to get through and decompile the directx decompression bit (somewhere in the template file that's mentioned). or Recompile it or something along those lines.

So, it might well just be that the file we see in the game files is simply a checksum as you mentioned, and that adding it to a package and doing it that way might be a workaround of the issue

Lemme know if that works!
epochtimestamp
Test Subject
#75 Old 2nd Jul 2021 at 7:44 AM
Quote: Originally posted by Lyralei
Now, I doubt this will do anything and I haven't tried it myself, but in ts4 forums, what one of the devs mentioned (or Guru rather) was to leave the precomp in the game files alone, but the 'new' one, you put in a package and then put it in your mods folder and the game should pick up on it then. I've personally seen this working with existing ini files that are outside of the game, and putting those in a package file seems to make some changes.


I spent some time trying to figure this out. Using ghidra to disassemble and decompile, I was able to find the code that parses the precomp file in TS3W.exe. I also found a call to stat64() just before the parsing. Debugging with IDA Pro, I was able to determine that the path to stat64() is the full path to Shaders_Win32.precomp in the game directory.

So far it isn't looking good for there being a way to override the file with a package in the mods folder. That said, all is not lost. I thought I understood how the game resolved the file location, but stepping through in the debugger I found that didn't actually work how I expected. So perhaps all I'm seeing is a failed check for a mod, resulting in loading the file from disk. I'll keep digging, but I'll have to find an alternative tool, IDA Pro doesn't seem to make looking at live memory easy and I think that's what I need to do if I'm going to figure out whether or not there's potential for overriding the file.
Page 3 of 4
Back to top