Munch's Oddysee, ROF archives, and NIF models. -

Gretin · #31 01-06-2010, 02:44 PM

Alright, I took a look and it looks like it is the number of files that's causing the problem.
See, what it does at the moment is it checks the first byte to see how many files it needs, then it requests enough memory to hold for each file:
An integer for BEGIN
An integer for SIZE
A string for NAME

But I just tested it by putting an exception catcher and it failed to allocate the memory, so I'll have to re-work the way I do it. I think what I'll do is make it only store the details for one file in memory at a time. So basically it will work through every file and print the details individually.

As for extraction, that works fine on my Windows computer for Layout_y.nif00003.ROF, so I'm not sure what the best way to make it work on a Mac is...

Naulahauta · #32 01-06-2010, 02:49 PM

Well, as long as you've coded it in uuh, was it C++? Well, C++ or not, I'm pretty sure I'm able to compile it on my computer. If you can provide a source, I think we're both fine

You can also contact me anytime at verneri.kontto@hotmail.com

Gretin · #33 01-06-2010, 03:11 PM

Ok, I found I was wrong about part of the problem. I've re-written it to avoid the running out of memory problem anyway, but it turns out that because of the way the int type in C++ works (in my compiler at least), the $C9 char (byte) translates to $FFFFFFC9 as an unsigned int or $-36 as a signed int. It was set up as an unsigned int, so it thought there was $FFFFFFC9 files in the archive, or 4,294,967,241, so it's no wonder it couldn't allocate the necessary memory

I've fixed it by setting it as an unsigned int and then it just checks to see if it's below zero, in which case it adds $FF to it.

And the only problem with you compiling the source on your computer is that it uses Windows functions. Unless you could point me in the direction of equivalent Mac C++ coding for creating new folders?

EDIT: Attached the updated executable. I tested it and it works fine now, viewing and extracting all 201 files. Would anyone else with Windows be able to test that it works so we know for certain it's just not working because it's on a Mac?

EDIT 2: Removed buggy version, see later post for up-to-date version

Naulahauta · #34 01-06-2010, 03:54 PM

As far as I know, the command for creating a new directory in the Bourne-Again Shell (also known as 'bash'), which is the native shell of Mac OS X, is mkdir.
But, trying the new one out now. In the meantime, maybe you could write a BAT script to accompany the EXE? I know that sometimes when an EXE fails, a BAT that uses it mysteriously works.
I'll edit this post with results.

EDIT: Yeah, still fails to create the files.

Gretin · #35 01-06-2010, 04:33 PM

Alright, let's try again

I've tried one that's using mkdir to create the folders instead (as that command exists in the Windows command prompt as well), so with any luck this one might work for you. I've included a BAT as requested as well.

EDIT: Only catch is that you'll get lots of error messages saying the folder already exists, I need to add something to it to check whether the folder already exists or not before using mkdir. It shouldn't affect extraction though.

EDIT 2: Removed buggy version, see later post for up-to-date version

Naulahauta · #36 01-06-2010, 05:17 PM

Yyyyyeah, still fails. I'll try and fix up something tomorrow, but right now it's 3:20 AM and I need some sleep.
Until then.

EDIT: Maybe you could try writing a super overly-simplistic GUI for the app? Just a pane with Open, View and Extract buttons? I have a hunch that might work.

Gretin · #37 01-06-2010, 10:14 PM

Dang it! I don't think the GUI would make much difference, cause it's the function to extract the files which is where the problem is, and a GUI would still call that same function.

It appears to be either the folder creation, or file creation that fails to work... Probably the folder creation, which then causes the file to be unable to be created due to the path not existing. Just one thing you could try to confirm this is, try creating the folders manually first and see if any of the different deROF versions I uploaded work on it then.

Sorry it doesn't appear to have been very successful so far

Perhaps I should try to learn Python so I can do it the same way the other person you mentioned was going to.
It does work on Windows though, so anyone can feel free to use it that way! (but I would suggest not using the "hopefully Mac compatible" one because the earlier ones actually used proper Windows functions)

Naulahauta · #38 01-07-2010, 03:05 AM

Yeah, I guess you're right. Still, though, it's weird. I know I've used the EXE interpreter to use programs far more complex than this deROFer and they've worked fine.
I'm not saying you're a bad coder, I'm just intrigued by this mystery. Well, guess I really should get a Windows, haha.

Regardless, yeah, I'll be uploading all the ROFs to Megaupload later today, so anyone who's interested, be sure to check them out!

Gretin · #39 01-07-2010, 01:16 PM

Well, I'm uploading the most stable (for Windows) version to this post and I've removed the others, so anyone who's running Windows can feel free to try it out. I've also uploaded the contents of Effects_000000.rof to megaupload so you can see if it's extracting them correctly

EDIT: Removed this version of the extractor, as it's been updated again, see the later posts.

Did you try creating the folders manually first to see if any of the deROFer versions could create the file then? And did the console say anything at all when you tried to run the last one (the "hopefully Mac compatible" one)?
Maybe it is just my bad coding, but it seems weird that it works fine for me

Anyway, contents of Effects_000000.rof (if they've been extracted correctly) can be downloaded here.

moxco · #40 01-07-2010, 02:52 PM

Whenever I try to open DeRof.bat and DeRof.exe; a black box will flash up and dissapear straight away.

Gretin · #41 01-07-2010, 03:43 PM

As I said earlier, you'll need to open it in a command prompt. Or you can download either of those ROF files that Naulahauta uploaded and edit the batch file to have the name of that file (at the moment it says Effects_000000.rof), then make sure the EXE, the BAT, and the ROF are all in the same directory and run the BAT.

If I can be bothered I may try making a GUI for it later though, to make it easier for everyone to use it

Naulahauta · #42 01-08-2010, 02:13 AM

That reminds me..! What does the program do if there are two files of the same name inside the ROF? Upon inspecting a few I noticed that one ROF contains two NIFs, both the same name, both the same path. The files' contents are different though.
Does it add a _* to the file's name's end, or does it ask what to do?

***Wil*** · #43 01-08-2010, 02:15 AM

How can two different files have the same name and path?

Gretin · #44 01-08-2010, 05:29 PM

Oh, that's a nuisance. At the moment I think it'd just overwrite the second file, so I'll have to add some code to make it check if the file already exists first, and if so add something to the end to differentiate between different files.

And has anyone else tried out the program to see if it works for them?

Naulahauta · #45 01-09-2010, 04:43 AM

:

How can two different files have the same name and path?

They're stored inside the ROF archives, not the standard filesystem.

EDIT: Here's the link to the RAR with ALL 90 ROFs inside. Have fun with this whopping 118 (well, 271 when you decompress the RAR!) Megabytes of valuable data, and if you extract each 'n every, be sure to re-upload them somewhere so I can check on the DEF files and try to crack them as well.

Get 'em!

***Wil*** · #46 01-09-2010, 06:25 AM

I get that, but all the same.

Naulahauta · #47 01-10-2010, 03:58 PM

Good news! I visited a friend who owns a PC with Windows installed, so I got the chance to decompress the ROFs.
I'm now trying to fiddle with the DEFs. They're a real nuisance, and if anyone has any kind of idea how to crack the format, please let me know. I'm totally lost and practically this makes no sense.
I haven't encountered any kind of compression format like this before. It's not RLE-kind of LZ* kind or aaaaaaaaaaaaagh I'm lost. :<

Paul · #48 01-10-2010, 04:46 PM

You may have reached the point where you need to reverse the game engine code that reads the files..

Gretin · #49 01-10-2010, 06:18 PM

Fantastic, so the files came out like they were supposed to?

Also, I've started working on a GUI to extract them as well, just so anyone can use it. Might be a few days before I'm happy with it though.

Naulahauta · #50 01-11-2010, 08:39 AM

Well, yeah, they came out like they were supposed to, except I couldn't find a few files.
I know I extracted ALL ROFs, but still I couldn't find the aforementioned slig_bullshit files.
Maybe it skipped lines of code in the BAT or something.

Gretin · #51 01-11-2010, 01:19 PM

Which ROF's were the missing files supposed to be in? I can take a look and see what's going on. I suppose the aforementioned files with the same name will also be missing as well.

Naulahauta · #52 01-11-2010, 01:53 PM

The Bullshit WAVs were in the Oddio_postinitdssr000001.rof file, and I think I might know what the problem is (and it's my fault, d'oh!)
You see, $X is not one, but two bytes. In most other ROFs the first few bytes are, say, 5C 00, 8B 00, 02 00, etc.. but in Oddio_postinitdssr000001.rof, it's DF 01. I counted the Z groups of the file, and there are 0x1DF (=479) of them, as opposed to the previously-thought 0xDF (=223).
So, in a nutshell, $X is two bytes read backwards, not one.

EDIT: Whoa! Layout_n.nif000000.rof's $X is 92 05! There are 1426 files inside, whohohoa!

Gretin · #53 01-12-2010, 12:31 PM

Ah, that should be easy enough to fix. In fact, here you go, updated version of the command prompt extractor XD Also it now adds an underscore to the end of the file name if it already exists. It was kind of a lazy solution though, so it's added RIGHT at the end (after the extension), so you'll probably have to rename it. It's just to avoid the first file being overwritten.

Anyway, I'm not 100% sure if this'll work or not, since I haven't downloaded any of the big ROFs yet, but it should be fine

I wonder if this topic would be better in General Oddworld Discussion? Just because it doesn't seem to be really getting noticed here and this is pretty awesome stuff you're doing.

Naulahauta · #54 01-12-2010, 11:48 PM

Superb! Gotta try that out when I get the chance.
And yeah, a mod should probably move this. This started out as a help me-topic but evolved into something new.

Paul · #55 01-13-2010, 10:31 PM

:

Whenever I try to open DeRof.bat and DeRof.exe; a black box will flash up and dissapear straight away.

Look at the previous posts

Crashpunk · #56 01-14-2010, 08:58 AM

Do you recon it possible to convert the Munch's Oddysee Modals, Materials, Maps and other files to work in Garry's Mod?

Paul · #57 01-14-2010, 11:04 AM

From what I've seen of the source engine it should be able to handle it. You just need a converter for NIF files? I reckon there is prolly a maya plugin or some such? Would be cool to run around the maps as Gordon blasting sligs

Naulahauta · #58 01-14-2010, 11:07 AM

NIFs can already be converted to 3DS and OBJ. Textures are stored as DDS files which can be converted to whatever.
I'm pretty sure it's possible, but we can't get all NIFs just yet. The DEF compression must first be broken.
I'm pulling an all-nighter tonight and will check on the DEF'd NIFs as much as I can.

MojoMan220 · #59 01-14-2010, 11:37 AM

I haven't a clue what any of this technical stuff means, but keep it up! I really like the screenshots you've posted (post more!), and if this cumulates in a simple level editor or even a way to merely view the models/levels, that’d be awesome.

Crashpunk · #60 01-14-2010, 12:30 PM

If someone covert them so they work with Garry's mod, let me know as soon as possible!