Munch's Oddysee, ROF archives, and NIF models.

[Naulahauta]

Hi all. I understand it may be a bit intruding to register only to have my first post in the help section.. but I hope it's acceptable.
I made my way here through this video, that's description said "If anyone knows how to extract .rof files please send me a Private Message on YouTube or on the Oddworld Forums.".
I've looked into the ROF archives myself in the past and was happy to find someone else interested in it.

ROF files are the most common file in Munch's Oddysee. The root contains three important folders that consist four file formats.
The first one is BIK. BIK is the Bink video codec and they are the video files used in Munch's Oddysee. They're viewable with Bink Video Player.
The second/third one is WAV and its accompanying SGT. WAVs are standard waveform audio files of the Mudokon Shaman's lines you can listen in practically any player, and SGTs are their sister files that's usage is yet unknown to me. For every WAV, there's a SGT of the same name. All SGTs are 4 kb.
The last one is ROF, the one I'm after. ROFs are very likely to be (sometimes compressed) archives that contain even more files and unknown formats, including the models, the levels, the sound effects, the textures, etc. ProjectOddMods, if you're reading this, please have a look below.

All ROFs begin with a 8 byte header. In Layout_y.nif000017.rof, the biggest of the ROFs, and also the one I'll be using as a prime example has 08 00 00 00 54 01 00 00.
After that, there are 20-byte headers that are directly linked to the ASCII paths right after them. Observe the bytes after the 8-byte header (formatted for viewability):
FC 01 00 00 9F 06 88 00 00 00 00 00 34 00 00 00 00 00 00 00
9B 08 88 00 D3 9F 00 00 00 00 00 00 29 00 00 00 34 00 00 00
6E A8 88 00 45 66 79 00 00 00 00 00 26 00 00 00 5D 00 00 00
B3 0E 02 01 60 91 77 00 00 00 00 00 2E 00 00 00 83 00 00 00
13 A0 79 01 12 1B 00 00 00 00 00 00 27 00 00 00 B1 00 00 00
25 BB 79 01 3D 8E 01 00 00 00 00 00 2A 00 00 00 D8 00 00 00
62 49 7B 01 06 58 00 00 00 00 00 00 29 00 00 00 02 01 00 00
68 A1 7B 01 0F 17 00 00 00 00 00 00 29 00 00 00 2B 01 00 00

And RIGHT after them in ASCII (again, formatted):

\LAYOUT\REGION05\ROOM01\GAMESTOCKDEMO\GSDEMO_BV.NIF
\LAYOUT\REGION05\ROOM01\GROUNDCLOUDS.NIF
\LAYOUT\REGION05\ROOM01\ROOM01_BV.NIF
\LAYOUT\REGION05\ROOM01\ROOM01_VILLAGE_BV.NIF
\LAYOUT\REGION05\ROOM01\SMALLWATER.NIF
\LAYOUT\REGION05\ROOM01\SPLINTERZSIGN.NIF
\LAYOUT\REGION05\ROOM01\VILLAGEFLAGS.NIF
\LAYOUT\REGION05\ROOM01\VILLAGEWATER.NIF

It is not a coincidence that there are eight 20-byte headers and eight paths to NIF files. They are clearly linked and the headers hold the key to getting the NIFs out.

The NIFs themselves are NetImmerse/Gamebryo model files, the same model files used in games such as Civilization IV, Dark Age of Camelot, Freedom Force, Morrowind, Oblivion and Sid Meier's Pirates.

Long story short, if we break this format, we will have access to Munch's Oddysee resources. NIFs can already be opened and converted in Maya, 3DSMAX and Blender through the NIFTools plugin, and who wouldn't want their 3D models of Abes, Sligs, Slogs, Munches, Scrabs, Glukkons, levels.. maybe even unused areas?

So, my first question is, does anyone around here 3DSMAX and be interested enough to install the NIFtools plugin so we could venture into the depths of Munch's Oddysee? Or even be ready to install the NIFSkope viewer that I, as a Mac user, can't open?

Thanks in advance. Hope this turns into a group effort.

[WhiteSnoop]

I have 3DSMax, but I'm not that good at it.
I downloaded it sometimes ago for the game "Command And Conquer™ Renegade™"

But as I'm good at photo shopping, im not good in 3d moddeling

But I Can test those out for you

~WhiteSnoop

[Naulahauta]

Thanks! Also, some good news.. I was able to compile the NIFtools and extract a partial NIF out! Check this out:

It's some area from Munch's Oddysee, explorable and viewable.

[WhiteSnoop]

Looks Real Nice.
Sadly, I never played Munch's Oddysee.
Don't own the game either


Offtopic:
BTW do you know the program 3DRipper DX?
You can rip everthing from everygame :P

[Naulahauta]

Thanks. I don't own Munch's Oddysee either, not to mention I have never played the game.
And yeah, I know 3DRipper DX. it's great for small things, but Munch's Oddysee is (as far as I'm aware) unemulatable. 3D Ripper DX also produces field-of-view distortions which are a huge no-no.

EDIT!:
New discoveries of the ROF header! The first four bytes represent the offset where the data begins for that file. Observe header #398 (Stormcircle.wav) from Oddio_postinitdssr000001.rof

-4F DC 79 00 FA B8 10 00 00 00 00 00 17 00 00 00 D9 27 00 00.
-Take the first four bytes, 4F DC 79 00.
-Reverse them, so you get 00 79 DC 4F.
-Go to offset 0x0079DC4F.
-That's where the file, (which is a WAV) begins!

The second four bytes represent the size of the file. Allow me to elaborate:
-4F DC 79 00 FA B8 10 00 00 00 00 00 17 00 00 00 D9 27 00 00.
-Take those other four bytes, FA B8 10 00.
-Reverse them, so you get 00 10 B8 FA.
-That's it. Our sample is 0x0010B8FA bytes big.

With simple math, you can count that 0x0079DC4F (our sample's beginning offset) + 0x0010B8FA (the size of our sample) = 0x8A9549. That is where our sample ends.
So let's break it down, gentlemen: the area between 0x0079DC4F and 0x8A9549 is WAV data. It's as simple as that..! Here's the WAV we just practically extracted from the ROF archive

[Josh]

Wait, how are you getting MO on a PC?

[Naulahauta]

I downloaded an XBOX ISO Image file of the game. They're meant to be burned to a DVD and played on a modded XBOX, but instead I mounted the disc and got access to its files.

[WhiteSnoop]

I downloaded an XBOX ISO Image file of the game. They're meant to be burned to a DVD and played on a modded XBOX, but instead I mounted the disc and got access to its files.

Smart Thinking dude

No I know 3D Ripper DX is not that good, just a little question if ya know it and maybe tried it.
I used it on Driver Parallel Lines..
But Also Good things come with 3D Ripper DX; Windowed Mode you can play games in windowed mode if DXWND doesn't work

Nice that you downloaded and mounted it
Xbox may have different files you know. so allot of extensions are special for xbox. so maybe we never decipher it?

[Josh]

We will probally be able to do that when the Oddbox comes out.

[Naulahauta]

Well, that's true. Most of the formats are proprietary and made up by the folks at Infrogames.
However, I've cracked formats before. Not XBOX ones, but some GameCube and Game Boy ones, so I've got a slight clue, I guess.. heh. I just hope these aren't too complex.

I also found out that a lot of these seem to be encrypted/compressed with a schema called DEF. I extracted three WAVs: Slig_bullshit1.wav, Slig_bullshit2.wav, Slig_bullshit3.wav. None of them opened up.
I opened them in a hex editor to examine them, and all of them begin with the 8-byte string "defT.8..".
I've never heard of the def or deft format before, but a lot of these files seem to be mangled under it.

EDIT: That's a real good point, Josh. Any clue when Munch is going to come out?

[Josh]

In that screenshot, why is the Cliffpath to the ressurection shrine not showing up?

And well done, I wonder if anybody has ever done this before...?

[Naulahauta]

I don't know, but my best guess is that it's an external object and not showing up. No Sligs or Mudokons aren't there either.
The game calls external objects like characters and possibly the cliffpath to the area.

[WhiteSnoop]

Isn't there a good Xbox Emulator, cuz I can't stand buying a lame ass xbox.
I searched, but they let me download an emu for Home Brew games, which I don't understand. (making a Xbox Emu, and let it run self made games WTF)

BTW keep up the work, keep Hex editing. Maybe you could learn me more Hex skillz, I know some, but I can't understand some things, like

Ingame my health is 100
In hex it is 64
But Why the heck is it 0046 or 4600 (forgot) in the Hex Editor itself?
Please Explain

[Naulahauta]

The reason XBOX Emulators run homebrew games is that Homebrew games are far far simpler than 'real' games. They use only a fraction of the commands and operations that the real games do, and that's why they run well on early-development emulators.
It's also a great development curve: people make homebrew games, and if they don't work, the team reviews the game source code to check where it crashes, makes notes based on it as an effort to fix it, and if still the failure the emulator's fault, the emulator developers add support for operations and/or fix the emulator so it runs the game.
The process repeats itself until the emulator becomes perfect.

And yeah, hexadecimal isn't too difficult to grasp.
Where we, the humans, count from 1 to 10 and repeat (this is the DECIMAL system), computers understand only binary, which is always either 0 or 1. These are bits.
8 bits is one byte.
Let them all be 0, it would be 00000000. It represents the value of 0.
One bit to the end, and it'll be 00000001. It represents the value of 1.
Add one bit again, and it'll be 00000011. This represents 3. Why? Because we forgot 00000010. Bytes go through every single combination of bits. The full byte, which is 11111111, is 255.
And because typing all these 1s and 0s would be trivial as hell, we can compile them into the HEXADECIMAL system.
Binary * Hex * Decimal
0000 = 0x0 = 0
0001 = 0x1 = 1
0010 = 0x2 = 2
0011 = 0x3 = 3
0100 = 0x4 = 4
0101 = 0x5 = 5
0110 = 0x6 = 6
0111 = 0x7 = 7
1000 = 0x8 = 8
1001 = 0x9 = 9
1010 = 0xA = 10
1011 = 0xB = 11
1100 = 0xC = 12
1101 = 0xD = 13
1110 = 0xE = 14
1111 = 0xF = 15

So if something is 100 in decimal, the system WE use, it's 0x64 in hexadecimal.
You can experiment around by typing into Google search "what is 132 in hexadecimal" and it'll answer. Same goes for "what is 0xF2 in decimal", or even "what is 0x12 in binary"

Fool around and read Wikipedia. It helps. At least, it helped me.


EDIT: Good news! I found someone who can code in Python to help me develop a tool to extract the files out of the ROFs. Expect results in a day or few!

[Naulahauta]

Also, in the root, there's a file called StringTags.INF. It contains every piece of text data in the game. There are some interesting strings there, but I haven't scoured through each of them. Post if you find anything funny, unused or interesting!

EDIT: Oh, snap. Double post. Sorry about that.

[Nate]

EDIT: Oh, snap. Double post. Sorry about that.

If it's longer than 24 hours, it doesn't matter.

[Naulahauta]

Oh. Awesome, the 24 hour regulation B)
On other news, the Python guy hasn't been responding to my messages lately.. I don't know what to expect, but if there are any coders in this forum, be sure to inquire within.

[Crashpunk]

I haven't the fainist clue what your doing but its looks great!

[STM]

I concour, this is amazing, you need ot get more people on this thread!!! IF you could recreate the scenes of MO from this aerial view it would make a fantastic addition to the fan corner!

[Naulahauta]

Thanks, Scrabtrapman..!
But all in all, I can't make much of a progress until:
-The guy who said he'd code the app responds.
or
-I find someone else who can code.

I can only code in ActionScript 3.0 and it isn't really meant for file input/output.

[Gretin]

I might be able to help you out there. I can code C++, I'll probably be too lazy to make a fancy program to do it, but if you told me what you needed it to do I could probably do a basic command prompt one.

[Naulahauta]

Oh? Awesome..!
Here's a little workflow I made how the program should work. Start from the BEGIN and carefully follow the arrows. All values are hexadecimal, like the $ suggests, and decimal if there is no prefix.
Ask away if you have questions.



EDIT: More pics!



Also, the file I used for reference in that workflow instruction image was Layout_y.nif000003.ROF. I attached it. Use it to Test stuff out.
http://www.megaupload.com/?d=B31ZPPLG

[Wil]

Brilliant! I very much hope that great things come of this effort. Well done to you.

[Gretin]

That's a very useful diagram! Thanks for that, that should make programming a small application to extract the files pretty easy. I won't be able to do it until after Monday, because I'm pretty busy this weekend, but I should be able to get it done within a day once I start on it.

So thanks to you, for going to all the hard work of figuring out the file format and making a very clear diagram of your findings

[Naulahauta]

Hahahaaa! Superb!!
When I drew it I had a constant fear of it being too complex, but just hearing that you comprehend it with ease makes me cry tears of joy
This is excellent. Can't wait 'til monday!

[Paul]

Very nice work there Good to see I'm not the only one who has attempted to break the formats of Oddworld games.

What is your end goal of this work? Just to be able to use the resources from the game? Or to mod the game?

[Gretin]

Alright, it's a day later than I promised (thanks to my memory stick corrupting the source code files when I first did it ), but here it is!

A simple command prompt utility that does exactly what Naulahauta's diagram explained, nothing less and nothing more.

Just bear in mind that I only had that one file to test with, so I don't know how buggy it is. It doesn't do much error checking, so make sure you have it set up right or you could get unexpected results And I also have nothing to open the resulting files with, so I don't even know if they've been done correctly But I'll leave that to you to try out.

Just run it in a command prompt and it will tell you how to use it.

EDIT: Removed buggy version, see later post for up-to-date version

[Naulahauta]

Woo! Let's check it out B)
I'll edit this post when I'm done.
EDIT: Maybe you should include a source code. Just a suggestion. I'm using a Mac and have to run this through a WINE-like utility to run EXEs on my computer, so it's a bit more complex.
REGARDLESS, I tried it out, and viewing files works like a charm. It lists them alright, except in at least this one file, that's first byte is a whopping C9. I think that because of it contains so many files it fails. I'll look into it soon.
Extracting files fails though. It says it extracted them successfully but they're nowhere to be found. I think this is the fault of me and my buggy EXE-interpreter, though.

[Gretin]

Hmmm, that's a bit of a pain, I didn't realise you're running a Mac. I could upload the source code (though I'd better tidy it up first, add comments etc ), but it's not doing anything particularly complex.

For extracting it pretty much just analyses the path and uses the Windows function CreateDirectory to make a folder if need be, then opens that folder and carries on doing that until it reaches the end of the path, after which it uses the standard C++ "fstream" library to write the file itself. And then it uses SetCurrentDirectory to go back to the original folder and starts the process again for the next file. But running it on a Mac, I have no idea how it translates.

I'm not sure exactly what you mean by this though:

REGARDLESS, I tried it out, and viewing files works like a charm. It lists them alright, except in at least this one file, that's first byte is a whopping C9. I think that because of it contains so many files it fails. I'll look into it soon.

Do you mean the first byte of the file, meaning it contains 201 files? And what does the program actually do?

[Naulahauta]

Yeah, 201 files is a lot. I'm not sure at all if that's the reason though.
But if I, for one, type:
derof -v Layout_y.nif000003.ROF
It works fine. Lists like it should.

If I type:
derof -e Layout_y.nif000003.ROF
It 'seems' to work fine. It takes a second and then says that it was successful, but the files never appear anywhere.

That was the case with Layout_y.nif000003.ROF. If I try with this Effects_000000.rof file, which has 201 files inside, it will fail to list and/or extract.

Be sure to look into it, and if you need more ROFs, just ask away