Replies: 76 (Who?), Viewed: 3062 times.
Page 1 of 4
Ms. Byte
Original Poster
#1 Old 27th Dec 2020 at 12:47 AM Last edited by CmarNYC : 12th Jan 2021 at 12:38 PM.
Default LRLE converter, testing needed, update 1/3/2021 v2.3
After months of frustration I seem to have cracked LRLE version 2. A very simple tool is attached that takes an LRLE file - which you can extract with s4pe or S4S - and decodes it into an image. Testing is needed, especially with very large files that may have number coding I haven't seen in smaller files. I'm going to work next on converting png to LRLE, which should be relatively easy. Conversion to LRLE is done!

Note this does NOT work on LRLE version 1. If anyone familiar with images and RLE and reading enough hex code to last a lifetime wants to help with that, please do. They're similar in some ways but have major differences. I'm attaching a zip with the same very small, simple image of a mole in LRLE and LRLE v2.
Now supports version 1!

I think the L stands for limited - some RLE algorithms use a limit over which an instruction or index (in this case) overflows into another byte. In this case it's 0x80.

Code is included in the attachment. Later I'll document the format. Anyone is free to use the code or information any way they want but if you do please give credit. This tool is very crude, don't expect it to be pretty or not crash if there's a problem.

Please upload any LRLE files you find that crash the program or give messages about unrecognized instructions!

Update 1/3/2021: V2.3
Fixed artifacting in some images when converting to LRLE

Update 1/2/2021: V2.2
Better size of saved LRLEs
Fixed strange appearance of some loaded LRLEs due to a test I set up and forgot to remove

Update 1/2/2021: V2.1
Faster conversion to PNG (tau534)
Convert LRLE to uncompressed DDS (tau534)

Update 12/31/2020:
Added conversion of PNG to LRLE (co-authored with tau534)

Update 12/30/2020:
Added support for LRLE version 1. (credit to tau534)

Update 12/28/2020:
Added code for images with 65,536 and over distinct pixels. Unless someone finds a texture this doesn't work with, I think this thing is whipped.

Update 12/27/2020:
Added tau534's modifications.
Download - please read all instructions before downloading any files!
File Type: zip LRLEConverterV2.3.zip (31.8 KB, 82 downloads)

Please do not PM me with mod, tutorial, or general modding questions or problems; post them in the thread for the mod or tutorial or post them in the appropriate forum.

Visit my blogs for other Sims content:
Online Sims - general mods for Sims 3
Offline Sims - adult mods for Sims 3 and Sims 4
Advertisement
Test Subject
#2 Old 27th Dec 2020 at 3:37 AM
I'm willing to help test, and I've already ran the largest LRLE v2 resource through it and there is an issue with unknown opcodes on mip levels 2 and 3.
Download - please read all instructions before downloading any files!
File Type: zip error.zip (2.80 MB, 8 downloads)
Ms. Byte
Original Poster
#3 Old 27th Dec 2020 at 3:49 AM
Quote:
Originally Posted by tau534
I'm willing to help test, and I've already ran the largest LRLE v2 resource through it and there is an issue with unknown opcodes on mip levels 2 and 3.


Exactly the kind of example I was looking for, thanks!

Please do not PM me with mod, tutorial, or general modding questions or problems; post them in the thread for the mod or tutorial or post them in the appropriate forum.

Visit my blogs for other Sims content:
Online Sims - general mods for Sims 3
Offline Sims - adult mods for Sims 3 and Sims 4
Test Subject
#4 Old 27th Dec 2020 at 5:19 AM
You're welcome. After some more testing and messing around with the code I did figure out what was going wrong and saw another bug. The opcode issues were due to topping too early on some copy color runs and the other bug was multiplying/shifting the extended data the wrong amount when getting an index. I've attached a fix for both issues along with a few other changes to make things easier to follow. It should be a drop in replacement for form1.cs and you are free to do what ever with it. I might see if I can
Download - please read all instructions before downloading any files!
File Type: zip code.zip (2.1 KB, 8 downloads)
Ms. Byte
Original Poster
#5 Old 27th Dec 2020 at 9:05 AM Last edited by CmarNYC : 27th Dec 2020 at 9:38 AM.
Quote:
Originally Posted by tau534
You're welcome. After some more testing and messing around with the code I did figure out what was going wrong and saw another bug. The opcode issues were due to topping too early on some copy color runs and the other bug was multiplying/shifting the extended data the wrong amount when getting an index. I've attached a fix for both issues along with a few other changes to make things easier to follow. It should be a drop in replacement for form1.cs and you are free to do what ever with it. I might see if I can


Wow, I'm not used to someone solving my coding problems - and so quickly and elegantly (almost anything would be more elegant than my banged-together run calculations, lol) - so this is very welcome. Thank you! I hope you'll take a stab at version 1 as well.

There may still be another complication which could show up in large, complex images like photos or if we eventually start using HQ textures with LRLE - if the number of pixel values is large enough to need a 3 or 4 byte index my code won't handle it for the repeat runs. Of course I don't know for sure if EA handles that either since the textures compressed with LRLE are relatively simple and have large transparent areas. Still looking for bigger image files to test on, if there are any.

Please do not PM me with mod, tutorial, or general modding questions or problems; post them in the thread for the mod or tutorial or post them in the appropriate forum.

Visit my blogs for other Sims content:
Online Sims - general mods for Sims 3
Offline Sims - adult mods for Sims 3 and Sims 4
Ms. Byte
Original Poster
#6 Old 27th Dec 2020 at 5:49 PM
Updated with tau534's modifications.

I've found one texture with one pixel over the two byte index limit and am trying to figure out how it's being addressed. Attached in case anyone wants to take a look.
Download - please read all instructions before downloading any files!
File Type: zip 2BC04EDF!00000000!C079AEE21EFD131C.zip (2.38 MB, 7 downloads)

Please do not PM me with mod, tutorial, or general modding questions or problems; post them in the thread for the mod or tutorial or post them in the appropriate forum.

Visit my blogs for other Sims content:
Online Sims - general mods for Sims 3
Offline Sims - adult mods for Sims 3 and Sims 4
Test Subject
#7 Old 27th Dec 2020 at 8:59 PM
I was planning on looking at V1. I'll also take a look at the newest problem texture.
Instructor
#8 Old 27th Dec 2020 at 9:43 PM
Just started playing around, found an error message, image attached. This is on instance C079AEE21EFD131C which, handily enough, has an RLE2 equivalent that shows it's an illness overlay texture that puts stars all over the Sim. Whoops, I didn't look very hard at the post about the texture that was giving you trouble because I knew I wouldn't be able to contribute to the tech part, but now I see it's the same texture.

I started off working with some of the larger LRLEs and of the top dozen or so only this one and one that said 'unsupported version' gave me any trouble. (That one was DCEFB022A596846F which looks like it's the result of a fabricator dye accident (the RLE2 is a bunch of splotches in cyan, yellow, maganta, and black), and I assume that means it's version 1? Not sure how to tell whether something is version 1 or 2 besides this tool, as a layperson.)
Screenshots
Ms. Byte
Original Poster
#9 Old 27th Dec 2020 at 9:43 PM Last edited by CmarNYC : 28th Dec 2020 at 4:02 PM.
Quote:
Originally Posted by tau534
I was planning on looking at V1. I'll also take a look at the newest problem texture.


Thanks! I'll look at the new problem first, probably not today though.

Edit: After a quick look, I think an op code of 3 means a copy/color run using 4 byte indexes. That doesn't make a whole lot of sense though. I'll try again later.

Edit 2: I think the values are the actual pixels, which makes less sense.

Please do not PM me with mod, tutorial, or general modding questions or problems; post them in the thread for the mod or tutorial or post them in the appropriate forum.

Visit my blogs for other Sims content:
Online Sims - general mods for Sims 3
Offline Sims - adult mods for Sims 3 and Sims 4
Ms. Byte
Original Poster
#10 Old 27th Dec 2020 at 9:50 PM
Quote:
Originally Posted by Cidira
Just started playing around, found an error message, image attached. This is on instance C079AEE21EFD131C which, handily enough, has an RLE2 equivalent that shows it's an illness overlay texture that puts stars all over the Sim.

I started off working with some of the larger LRLEs and of the top dozen or so only this one and one that said 'unsupported version' gave me any trouble. (That one was DCEFB022A596846F which looks like it's the result of a fabricator dye accident (the RLE2 is a bunch of splotches in cyan, yellow, maganta, and black), and I assume that means it's version 1? Not sure how to tell whether something is version 1 or 2 besides this tool, as a layperson.)


Thanks, that's the same texture I ran into the same problem with. IMO that's likely the ONLY texture with the situation of > 2 byte indices. Of course EA will likely convert more textures and it'll come up again.

Yes, unsupported version means it's version 1.

Thanks for testing, and especially for pulling the RLEs for comparison.

Please do not PM me with mod, tutorial, or general modding questions or problems; post them in the thread for the mod or tutorial or post them in the appropriate forum.

Visit my blogs for other Sims content:
Online Sims - general mods for Sims 3
Offline Sims - adult mods for Sims 3 and Sims 4
Instructor
#11 Old 27th Dec 2020 at 10:27 PM
Did a few other assorted samplings of LRLEs, checked about 80 files in total (though I didn't export all of them to PNG), no other problems except for running into some more version 1 files. Looks like everything other than that one thing is running pretty smoothly. If there's something specific either of you would like me to look at, let me know, but otherwise I think I may have reached the limit of my usefulness at this stage. (When I get to convert back and forth I'll have all KINDS of things to test.)
Ms. Byte
Original Poster
#12 Old 27th Dec 2020 at 10:41 PM
Quote:
Originally Posted by Cidira
Did a few other assorted samplings of LRLEs, checked about 80 files in total (though I didn't export all of them to PNG), no other problems except for running into some more version 1 files. Looks like everything other than that one thing is running pretty smoothly. If there's something specific either of you would like me to look at, let me know, but otherwise I think I may have reached the limit of my usefulness at this stage. (When I get to convert back and forth I'll have all KINDS of things to test.)


Thanks, very good to know you tested so many images and everything else (except version 1) worked out.

Please do not PM me with mod, tutorial, or general modding questions or problems; post them in the thread for the mod or tutorial or post them in the appropriate forum.

Visit my blogs for other Sims content:
Online Sims - general mods for Sims 3
Offline Sims - adult mods for Sims 3 and Sims 4
Test Subject
#13 Old 28th Dec 2020 at 3:21 PM
Thank you very much for decoding and this tool! I tested ~100 files and found only one broken image, 9FB614E20CF6A928. Error "Unknown instruction: 0 Position: 269698 Last good instruction: 269696". The image from EP01\ClientDeltaBuild0.package and has rle2 version in EP01\ClientFullBuild0.package
Screenshots
Download - please read all instructions before downloading any files!
File Type: zip S4_2BC04EDF_00000000_9FB614E20CF6A928%%+UNKN.zip (721.1 KB, 5 downloads)
Ms. Byte
Original Poster
#14 Old 28th Dec 2020 at 4:39 PM
Quote:
Originally Posted by Alf-si
Thank you very much for decoding and this tool! I tested ~100 files and found only one broken image, 9FB614E20CF6A928. Error "Unknown instruction: 0 Position: 269698 Last good instruction: 269696". The image from EP01\ClientDeltaBuild0.package and has rle2 version in EP01\ClientFullBuild0.package


Another example helped! GOT IT SOLVED! I'll upload a new version after a little more testing.

Please do not PM me with mod, tutorial, or general modding questions or problems; post them in the thread for the mod or tutorial or post them in the appropriate forum.

Visit my blogs for other Sims content:
Online Sims - general mods for Sims 3
Offline Sims - adult mods for Sims 3 and Sims 4
Instructor
#15 Old 28th Dec 2020 at 4:50 PM
Huh, that looks like another illness overlay, doesn't it? Might be an area to take a closer look at. I'm on the clock now, but if nobody beats me to it I can track some more down tonight and see if I can find any more examples. You are FAST! Also, HUZZAH!
Ms. Byte
Original Poster
#16 Old 28th Dec 2020 at 5:15 PM
Quote:
Originally Posted by Cidira
Huh, that looks like another illness overlay, doesn't it? Might be an area to take a closer look at. I'm on the clock now, but if nobody beats me to it I can track some more down tonight and see if I can find any more examples. You are FAST! Also, HUZZAH!


Thanks! Yes, it looks like a sickness overlay.

Notes: As I suspected, there are two additional opcodes for images with large numbers of distinct pixels: 3 to copy the following pixels and 6 to repeat the following pixel, with the pixel values embedded in the instruction stream. I suspected as much as soon as I saw the second example with again a pixel count of exactly 0x00010000. Looks like pixels over that limit get embedded with the opcodes.

Thanks! May I say at this point: I CAN'T BELIEVE I DID THIS!!! With help of course, very much including the testing I often don't have much patience for, but I CAN'T BELIEVE IT WORKS!!!!! If someone finds an image that errors out I'll have to go back to the drawing board, but that's life. :D

Please do not PM me with mod, tutorial, or general modding questions or problems; post them in the thread for the mod or tutorial or post them in the appropriate forum.

Visit my blogs for other Sims content:
Online Sims - general mods for Sims 3
Offline Sims - adult mods for Sims 3 and Sims 4
Ms. Byte
Original Poster
#17 Old 28th Dec 2020 at 5:18 PM
@Alf-si
@Cidira
@tau534

Sorry for the double post. More testing is welcome, and work on version 1 would be great.

Please do not PM me with mod, tutorial, or general modding questions or problems; post them in the thread for the mod or tutorial or post them in the appropriate forum.

Visit my blogs for other Sims content:
Online Sims - general mods for Sims 3
Offline Sims - adult mods for Sims 3 and Sims 4
Instructor
#18 Old 28th Dec 2020 at 6:11 PM
Quote:
Originally Posted by CmarNYC
Thanks! May I say at this point: I CAN'T BELIEVE I DID THIS!!! With help of course, very much including the testing I often don't have much patience for, but I CAN'T BELIEVE IT WORKS!!!!! If someone finds an image that errors out I'll have to go back to the drawing board, but that's life. :D


I can totally believe you did this, but it is also a GIANT ACCOMPLISHMENT that deserves confetti and noisemakers. We're this much closer to being able to do all the things that have been stuck behind LRLE!
Test Subject
#19 Old 28th Dec 2020 at 10:03 PM
Nice work. I'm have been working on v1 and I am annoyingly close to getting it working. I'm down to one opcode to figure out before it's working. I've attached a version of form1.cs with what I have so far if anyone else want's to look at it.
Download - please read all instructions before downloading any files!
File Type: zip code.zip (2.7 KB, 4 downloads)
Ms. Byte
Original Poster
#20 Old 28th Dec 2020 at 10:09 PM Last edited by CmarNYC : 28th Dec 2020 at 11:52 PM.
Quote:
Originally Posted by tau534
Nice work. I'm have been working on v1 and I am annoyingly close to getting it working. I'm down to one opcode to figure out before it's working. I've attached a version of form1.cs with what I have so far if anyone else want's to look at it.


Great, and thanks! I'll take a look later.

Edit: V1 has a strange construction that might be for pixels that vary only in alpha, or maybe for repeating pixels, that I never could figure out. That could be the opcode 3 that's giving trouble. I'll keep looking.

Please do not PM me with mod, tutorial, or general modding questions or problems; post them in the thread for the mod or tutorial or post them in the appropriate forum.

Visit my blogs for other Sims content:
Online Sims - general mods for Sims 3
Offline Sims - adult mods for Sims 3 and Sims 4
Test Subject
#21 Old 29th Dec 2020 at 5:02 AM
One more update, I did figure what opcode 3 does, it goes into another form of RLE. What I have so far cannot parse the top level mip of DCEFB022A596846F but it can parse the rest so I know I'm on the right track.
Download - please read all instructions before downloading any files!
File Type: zip code.zip (3.0 KB, 3 downloads)
Ms. Byte
Original Poster
#22 Old 29th Dec 2020 at 4:16 PM
Quote:
Originally Posted by tau534
One more update, I did figure what opcode 3 does, it goes into another form of RLE. What I have so far cannot parse the top level mip of DCEFB022A596846F but it can parse the rest so I know I'm on the right track.


Yes, it's partially working. For my sample image it works perfectly, for others it works for some mips. It's progress!

Please do not PM me with mod, tutorial, or general modding questions or problems; post them in the thread for the mod or tutorial or post them in the appropriate forum.

Visit my blogs for other Sims content:
Online Sims - general mods for Sims 3
Offline Sims - adult mods for Sims 3 and Sims 4
Test Subject
#23 Old 29th Dec 2020 at 11:27 PM
Alright, more testing is needed but, I should have cracked V1. Uploaded are the changes.
Download - please read all instructions before downloading any files!
File Type: zip code.zip (3.0 KB, 3 downloads)
Ms. Byte
Original Poster
#24 Old 30th Dec 2020 at 12:04 AM Last edited by CmarNYC : 30th Dec 2020 at 1:07 AM.
Quote:
Originally Posted by tau534
Alright, more testing is needed but, I should have cracked V1. Uploaded are the changes.


Great, I'll take a look. Thanks!

Edit: I've found one texture that throws an error in ReadEmbeddedRLE - the calculated count is too high for the arrays - in mip 1 and 2. Everything else I tried worked perfectly. The fun never ends!

I'm working on writing version 2 now. I don't see any need to write version 1.
Download - please read all instructions before downloading any files!
File Type: zip S4_2BC04EDF_00000000_03B5CBA3D2438D48%%+UNKN.zip (2.7 KB, 4 downloads)

Please do not PM me with mod, tutorial, or general modding questions or problems; post them in the thread for the mod or tutorial or post them in the appropriate forum.

Visit my blogs for other Sims content:
Online Sims - general mods for Sims 3
Offline Sims - adult mods for Sims 3 and Sims 4
Test Subject
#25 Old 30th Dec 2020 at 2:32 AM
I agree on only writing version 2. Also have a fix for v1 reading. It fixes the most recent problem v1 image. It's something I should have thought of but didn't the first time through.
Download - please read all instructions before downloading any files!
File Type: zip code.zip (3.0 KB, 4 downloads)
Page 1 of 4
Back to top