Munch's Oddysee, ROF archives, and NIF models.
Hi all. I understand it may be a bit intruding to register only to have my first post in the help section.. but I hope it's acceptable.
I made my way here through this video, that's description said "If anyone knows how to extract .rof files please send me a Private Message on YouTube or on the Oddworld Forums.". I've looked into the ROF archives myself in the past and was happy to find someone else interested in it. ROF files are the most common file in Munch's Oddysee. The root contains three important folders that consist four file formats. The first one is BIK. BIK is the Bink video codec and they are the video files used in Munch's Oddysee. They're viewable with Bink Video Player. The second/third one is WAV and its accompanying SGT. WAVs are standard waveform audio files of the Mudokon Shaman's lines you can listen in practically any player, and SGTs are their sister files that's usage is yet unknown to me. For every WAV, there's a SGT of the same name. All SGTs are 4 kb. The last one is ROF, the one I'm after. ROFs are very likely to be (sometimes compressed) archives that contain even more files and unknown formats, including the models, the levels, the sound effects, the textures, etc. ProjectOddMods, if you're reading this, please have a look below. All ROFs begin with a 8 byte header. In Layout_y.nif000017.rof, the biggest of the ROFs, and also the one I'll be using as a prime example has 08 00 00 00 54 01 00 00. After that, there are 20-byte headers that are directly linked to the ASCII paths right after them. Observe the bytes after the 8-byte header (formatted for viewability): FC 01 00 00 9F 06 88 00 00 00 00 00 34 00 00 00 00 00 00 00 9B 08 88 00 D3 9F 00 00 00 00 00 00 29 00 00 00 34 00 00 00 6E A8 88 00 45 66 79 00 00 00 00 00 26 00 00 00 5D 00 00 00 B3 0E 02 01 60 91 77 00 00 00 00 00 2E 00 00 00 83 00 00 00 13 A0 79 01 12 1B 00 00 00 00 00 00 27 00 00 00 B1 00 00 00 25 BB 79 01 3D 8E 01 00 00 00 00 00 2A 00 00 00 D8 00 00 00 62 49 7B 01 06 58 00 00 00 00 00 00 29 00 00 00 02 01 00 00 68 A1 7B 01 0F 17 00 00 00 00 00 00 29 00 00 00 2B 01 00 00 And RIGHT after them in ASCII (again, formatted): \LAYOUT\REGION05\ROOM01\GAMESTOCKDEMO\GSDEMO_BV.NIF \LAYOUT\REGION05\ROOM01\GROUNDCLOUDS.NIF \LAYOUT\REGION05\ROOM01\ROOM01_BV.NIF \LAYOUT\REGION05\ROOM01\ROOM01_VILLAGE_BV.NIF \LAYOUT\REGION05\ROOM01\SMALLWATER.NIF \LAYOUT\REGION05\ROOM01\SPLINTERZSIGN.NIF \LAYOUT\REGION05\ROOM01\VILLAGEFLAGS.NIF \LAYOUT\REGION05\ROOM01\VILLAGEWATER.NIF It is not a coincidence that there are eight 20-byte headers and eight paths to NIF files. They are clearly linked and the headers hold the key to getting the NIFs out. The NIFs themselves are NetImmerse/Gamebryo model files, the same model files used in games such as Civilization IV, Dark Age of Camelot, Freedom Force, Morrowind, Oblivion and Sid Meier's Pirates. Long story short, if we break this format, we will have access to Munch's Oddysee resources. NIFs can already be opened and converted in Maya, 3DSMAX and Blender through the NIFTools plugin, and who wouldn't want their 3D models of Abes, Sligs, Slogs, Munches, Scrabs, Glukkons, levels.. maybe even unused areas? So, my first question is, does anyone around here 3DSMAX and be interested enough to install the NIFtools plugin so we could venture into the depths of Munch's Oddysee? Or even be ready to install the NIFSkope viewer that I, as a Mac user, can't open? Thanks in advance. Hope this turns into a group effort. |
I have 3DSMax, but I'm not that good at it.
I downloaded it sometimes ago for the game "Command And Conquer™ Renegade™" But as I'm good at photo shopping, im not good in 3d moddeling But I Can test those out for you ~WhiteSnoop |
Thanks! Also, some good news.. I was able to compile the NIFtools and extract a partial NIF out! Check this out:
http://img15.imageshack.us/img15/1509/kuva2du.png It's some area from Munch's Oddysee, explorable and viewable. |
Looks Real Nice.
Sadly, I never played Munch's Oddysee. Don't own the game either Offtopic: BTW do you know the program 3DRipper DX? You can rip everthing from everygame :P |
Thanks. I don't own Munch's Oddysee either, not to mention I have never played the game.
And yeah, I know 3DRipper DX. it's great for small things, but Munch's Oddysee is (as far as I'm aware) unemulatable. 3D Ripper DX also produces field-of-view distortions which are a huge no-no. EDIT!: New discoveries of the ROF header! The first four bytes represent the offset where the data begins for that file. Observe header #398 (Stormcircle.wav) from Oddio_postinitdssr000001.rof -4F DC 79 00 FA B8 10 00 00 00 00 00 17 00 00 00 D9 27 00 00. -Take the first four bytes, 4F DC 79 00. -Reverse them, so you get 00 79 DC 4F. -Go to offset 0x0079DC4F. -That's where the file, (which is a WAV) begins! The second four bytes represent the size of the file. Allow me to elaborate: -4F DC 79 00 FA B8 10 00 00 00 00 00 17 00 00 00 D9 27 00 00. -Take those other four bytes, FA B8 10 00. -Reverse them, so you get 00 10 B8 FA. -That's it. Our sample is 0x0010B8FA bytes big. With simple math, you can count that 0x0079DC4F (our sample's beginning offset) + 0x0010B8FA (the size of our sample) = 0x8A9549. That is where our sample ends. So let's break it down, gentlemen: the area between 0x0079DC4F and 0x8A9549 is WAV data. It's as simple as that..! Here's the WAV we just practically extracted from the ROF archive :) |
Wait, how are you getting MO on a PC?
|
I downloaded an XBOX ISO Image file of the game. They're meant to be burned to a DVD and played on a modded XBOX, but instead I mounted the disc and got access to its files.
|
:
No I know 3D Ripper DX is not that good, just a little question if ya know it and maybe tried it. I used it on Driver Parallel Lines.. But Also Good things come with 3D Ripper DX; Windowed Mode :D you can play games in windowed mode if DXWND doesn't work Nice that you downloaded and mounted it :D Xbox may have different files you know. so allot of extensions are special for xbox. so maybe we never decipher it? |
We will probally be able to do that when the Oddbox comes out.
|
Well, that's true. Most of the formats are proprietary and made up by the folks at Infrogames.
However, I've cracked formats before. Not XBOX ones, but some GameCube and Game Boy ones, so I've got a slight clue, I guess.. heh. I just hope these aren't too complex. I also found out that a lot of these seem to be encrypted/compressed with a schema called DEF. I extracted three WAVs: Slig_bullshit1.wav, Slig_bullshit2.wav, Slig_bullshit3.wav. None of them opened up. I opened them in a hex editor to examine them, and all of them begin with the 8-byte string "defT.8..". I've never heard of the def or deft format before, but a lot of these files seem to be mangled under it. EDIT: That's a real good point, Josh. Any clue when Munch is going to come out? |
In that screenshot, why is the Cliffpath to the ressurection shrine not showing up?
And well done, I wonder if anybody has ever done this before...? |
I don't know, but my best guess is that it's an external object and not showing up. No Sligs or Mudokons aren't there either.
The game calls external objects like characters and possibly the cliffpath to the area. |
Isn't there a good Xbox Emulator, cuz I can't stand buying a lame ass xbox.
I searched, but they let me download an emu for Home Brew games, which I don't understand. (making a Xbox Emu, and let it run self made games WTF) BTW keep up the work, keep Hex editing. Maybe you could learn me more Hex skillz, I know some, but I can't understand some things, like Ingame my health is 100 In hex it is 64 But Why the heck is it 0046 or 4600 (forgot) in the Hex Editor itself? Please Explain :D |
The reason XBOX Emulators run homebrew games is that Homebrew games are far far simpler than 'real' games. They use only a fraction of the commands and operations that the real games do, and that's why they run well on early-development emulators.
It's also a great development curve: people make homebrew games, and if they don't work, the team reviews the game source code to check where it crashes, makes notes based on it as an effort to fix it, and if still the failure the emulator's fault, the emulator developers add support for operations and/or fix the emulator so it runs the game. The process repeats itself until the emulator becomes perfect. And yeah, hexadecimal isn't too difficult to grasp. Where we, the humans, count from 1 to 10 and repeat (this is the DECIMAL system), computers understand only binary, which is always either 0 or 1. These are bits. 8 bits is one byte. Let them all be 0, it would be 00000000. It represents the value of 0. One bit to the end, and it'll be 00000001. It represents the value of 1. Add one bit again, and it'll be 00000011. This represents 3. Why? Because we forgot 00000010. Bytes go through every single combination of bits. The full byte, which is 11111111, is 255. And because typing all these 1s and 0s would be trivial as hell, we can compile them into the HEXADECIMAL system. Binary * Hex * Decimal 0000 = 0x0 = 0 0001 = 0x1 = 1 0010 = 0x2 = 2 0011 = 0x3 = 3 0100 = 0x4 = 4 0101 = 0x5 = 5 0110 = 0x6 = 6 0111 = 0x7 = 7 1000 = 0x8 = 8 1001 = 0x9 = 9 1010 = 0xA = 10 1011 = 0xB = 11 1100 = 0xC = 12 1101 = 0xD = 13 1110 = 0xE = 14 1111 = 0xF = 15 So if something is 100 in decimal, the system WE use, it's 0x64 in hexadecimal. You can experiment around by typing into Google search "what is 132 in hexadecimal" and it'll answer. Same goes for "what is 0xF2 in decimal", or even "what is 0x12 in binary" Fool around and read Wikipedia. It helps. At least, it helped me. :) EDIT: Good news! I found someone who can code in Python to help me develop a tool to extract the files out of the ROFs. Expect results in a day or few! |
Also, in the root, there's a file called StringTags.INF. It contains every piece of text data in the game. There are some interesting strings there, but I haven't scoured through each of them. Post if you find anything funny, unused or interesting!
EDIT: Oh, snap. Double post. Sorry about that. |
:
|
Oh. Awesome, the 24 hour regulation B)
On other news, the Python guy hasn't been responding to my messages lately.. I don't know what to expect, but if there are any coders in this forum, be sure to inquire within. |
I haven't the fainist clue what your doing but its looks great!
|
I concour, this is amazing, you need ot get more people on this thread!!! IF you could recreate the scenes of MO from this aerial view it would make a fantastic addition to the fan corner!
|
Thanks, Scrabtrapman..!
But all in all, I can't make much of a progress until: -The guy who said he'd code the app responds. or -I find someone else who can code. I can only code in ActionScript 3.0 and it isn't really meant for file input/output. |
I might be able to help you out there. I can code C++, I'll probably be too lazy to make a fancy program to do it, but if you told me what you needed it to do I could probably do a basic command prompt one.
|
Oh? Awesome..!
Here's a little workflow I made how the program should work. Start from the BEGIN and carefully follow the arrows. All values are hexadecimal, like the $ suggests, and decimal if there is no prefix. Ask away if you have questions. http://img686.imageshack.us/img686/2...uctions.th.png EDIT: More pics! http://img121.imageshack.us/img121/7594/kuva46.png http://img130.imageshack.us/img130/2195/kuva47.png Also, the file I used for reference in that workflow instruction image was Layout_y.nif000003.ROF. I attached it. Use it to Test stuff out. http://www.megaupload.com/?d=B31ZPPLG |
Brilliant! I very much hope that great things come of this effort. Well done to you.
|
That's a very useful diagram! Thanks for that, that should make programming a small application to extract the files pretty easy. I won't be able to do it until after Monday, because I'm pretty busy this weekend, but I should be able to get it done within a day once I start on it.
So thanks to you, for going to all the hard work of figuring out the file format and making a very clear diagram of your findings :D |
Hahahaaa! Superb!!
When I drew it I had a constant fear of it being too complex, but just hearing that you comprehend it with ease makes me cry tears of joy :) This is excellent. Can't wait 'til monday! |
Very nice work there :) Good to see I'm not the only one who has attempted to break the formats of Oddworld games.
What is your end goal of this work? Just to be able to use the resources from the game? Or to mod the game? |
Alright, it's a day later than I promised (thanks to my memory stick corrupting the source code files when I first did it :compmad:), but here it is!
A simple command prompt utility that does exactly what Naulahauta's diagram explained, nothing less and nothing more. Just bear in mind that I only had that one file to test with, so I don't know how buggy it is. It doesn't do much error checking, so make sure you have it set up right or you could get unexpected results :D And I also have nothing to open the resulting files with, so I don't even know if they've been done correctly :p But I'll leave that to you to try out. Just run it in a command prompt and it will tell you how to use it. EDIT: Removed buggy version, see later post for up-to-date version |
Woo! Let's check it out B)
I'll edit this post when I'm done. EDIT: Maybe you should include a source code. Just a suggestion. I'm using a Mac and have to run this through a WINE-like utility to run EXEs on my computer, so it's a bit more complex. REGARDLESS, I tried it out, and viewing files works like a charm. It lists them alright, except in at least this one file, that's first byte is a whopping C9. I think that because of it contains so many files it fails. I'll look into it soon. Extracting files fails though. It says it extracted them successfully but they're nowhere to be found. I think this is the fault of me and my buggy EXE-interpreter, though. |
Hmmm, that's a bit of a pain, I didn't realise you're running a Mac. I could upload the source code (though I'd better tidy it up first, add comments etc :p), but it's not doing anything particularly complex.
For extracting it pretty much just analyses the path and uses the Windows function CreateDirectory to make a folder if need be, then opens that folder and carries on doing that until it reaches the end of the path, after which it uses the standard C++ "fstream" library to write the file itself. And then it uses SetCurrentDirectory to go back to the original folder and starts the process again for the next file. But running it on a Mac, I have no idea how it translates. I'm not sure exactly what you mean by this though: :
|
Yeah, 201 files is a lot. I'm not sure at all if that's the reason though.
But if I, for one, type: derof -v Layout_y.nif000003.ROF It works fine. Lists like it should. If I type: derof -e Layout_y.nif000003.ROF It 'seems' to work fine. It takes a second and then says that it was successful, but the files never appear anywhere. That was the case with Layout_y.nif000003.ROF. If I try with this Effects_000000.rof file, which has 201 files inside, it will fail to list and/or extract. Be sure to look into it, and if you need more ROFs, just ask away :) |
Alright, I took a look and it looks like it is the number of files that's causing the problem.
See, what it does at the moment is it checks the first byte to see how many files it needs, then it requests enough memory to hold for each file: An integer for BEGIN An integer for SIZE A string for NAME But I just tested it by putting an exception catcher and it failed to allocate the memory, so I'll have to re-work the way I do it. I think what I'll do is make it only store the details for one file in memory at a time. So basically it will work through every file and print the details individually. As for extraction, that works fine on my Windows computer for Layout_y.nif00003.ROF, so I'm not sure what the best way to make it work on a Mac is... |
Well, as long as you've coded it in uuh, was it C++? Well, C++ or not, I'm pretty sure I'm able to compile it on my computer. If you can provide a source, I think we're both fine :)
You can also contact me anytime at verneri.kontto@hotmail.com |
Ok, I found I was wrong about part of the problem. I've re-written it to avoid the running out of memory problem anyway, but it turns out that because of the way the int type in C++ works (in my compiler at least), the $C9 char (byte) translates to $FFFFFFC9 as an unsigned int or $-36 as a signed int. It was set up as an unsigned int, so it thought there was $FFFFFFC9 files in the archive, or 4,294,967,241, so it's no wonder it couldn't allocate the necessary memory :p I've fixed it by setting it as an unsigned int and then it just checks to see if it's below zero, in which case it adds $FF to it.
And the only problem with you compiling the source on your computer is that it uses Windows functions. Unless you could point me in the direction of equivalent Mac C++ coding for creating new folders? :p EDIT: Attached the updated executable. I tested it and it works fine now, viewing and extracting all 201 files. Would anyone else with Windows be able to test that it works so we know for certain it's just not working because it's on a Mac? EDIT 2: Removed buggy version, see later post for up-to-date version |
As far as I know, the command for creating a new directory in the Bourne-Again Shell (also known as 'bash'), which is the native shell of Mac OS X, is mkdir.
But, trying the new one out now. In the meantime, maybe you could write a BAT script to accompany the EXE? I know that sometimes when an EXE fails, a BAT that uses it mysteriously works. I'll edit this post with results. EDIT: Yeah, still fails to create the files. |
Alright, let's try again :p I've tried one that's using mkdir to create the folders instead (as that command exists in the Windows command prompt as well), so with any luck this one might work for you. I've included a BAT as requested as well.
EDIT: Only catch is that you'll get lots of error messages saying the folder already exists, I need to add something to it to check whether the folder already exists or not before using mkdir. It shouldn't affect extraction though. EDIT 2: Removed buggy version, see later post for up-to-date version |
Yyyyyeah, still fails. I'll try and fix up something tomorrow, but right now it's 3:20 AM and I need some sleep.
Until then. :) EDIT: Maybe you could try writing a super overly-simplistic GUI for the app? Just a pane with Open, View and Extract buttons? I have a hunch that might work. |
Dang it! I don't think the GUI would make much difference, cause it's the function to extract the files which is where the problem is, and a GUI would still call that same function.
It appears to be either the folder creation, or file creation that fails to work... Probably the folder creation, which then causes the file to be unable to be created due to the path not existing. Just one thing you could try to confirm this is, try creating the folders manually first and see if any of the different deROF versions I uploaded work on it then. Sorry it doesn't appear to have been very successful so far :p Perhaps I should try to learn Python so I can do it the same way the other person you mentioned was going to. It does work on Windows though, so anyone can feel free to use it that way! (but I would suggest not using the "hopefully Mac compatible" one because the earlier ones actually used proper Windows functions) |
Yeah, I guess you're right. Still, though, it's weird. I know I've used the EXE interpreter to use programs far more complex than this deROFer and they've worked fine.
I'm not saying you're a bad coder, I'm just intrigued by this mystery. Well, guess I really should get a Windows, haha. Regardless, yeah, I'll be uploading all the ROFs to Megaupload later today, so anyone who's interested, be sure to check them out! |
Well, I'm uploading the most stable (for Windows) version to this post and I've removed the others, so anyone who's running Windows can feel free to try it out. I've also uploaded the contents of Effects_000000.rof to megaupload so you can see if it's extracting them correctly ;)
EDIT: Removed this version of the extractor, as it's been updated again, see the later posts. Did you try creating the folders manually first to see if any of the deROFer versions could create the file then? And did the console say anything at all when you tried to run the last one (the "hopefully Mac compatible" one)? Maybe it is just my bad coding, but it seems weird that it works fine for me :p Anyway, contents of Effects_000000.rof (if they've been extracted correctly) can be downloaded here. |
Whenever I try to open DeRof.bat and DeRof.exe; a black box will flash up and dissapear straight away.
|