menu

Proprietary Binary Packaging

With just over a week left at Uni, I’ve been finishing off items left over before I’m completely done (assuming I don’t fail of course). Meanwhile I’ve already began developing some theories based on some projects I can work on. I’ve recently began designing a modularised hybrid 3D game engine, developed by me with the exception of physics and sound libraries. Over the past few months I’ve been developing mainly with the Ogre3D graphics library, and although it’s flexible, matured and powerful, I’d like to start my own process, partially as a great educational experience – a convenient bonus on top of a bachelor degree, no?

As a small project I’ve already began, I’ve been developing my own proprietary binary packaging format that allows me to store files. Rather than using ZIP, RAR, or other common archive files I’ve developed my own, allowing me to use information stored in my own way, which is also a major security boost as no one will be able to simply extract assets out of the packages. Tthen again, modifiability is always a good thing so it would only be for core files. Again, I back my reasoning behind “an educating experience”.

Currently there is no compression being performed on the packages, and I’m contemplating whether or not to bother as file-size is not so much an issue. Uncompressed data loads faster, and compression is only really useful for distribution and such, which is already handled by compression algorithms in installation software or digital distribution platforms.

The packaging tool is currently simply a console program, allowing me to drop files or folders onto it to recursively package all highlighted files into a single package. At a not-too-distant later date I’d like to start writing a reader allowing me to interpret the files in system memory, and eventually finish the binary package format with an extractor so files can be retrieved outside of memory (which is essentially very easy if the reader is already developed). Whether I’ll develop a graphical user interface for a tool is still undecided at the time, although at the moment I do not find it particularly necessary unless I plan on distributing the application for anyone looking for a light and simple binary package format (except then everyone would be able to read the packages!).

The last item I’ll cover is the mixed architectures I’ve used. Rather than writing a system to handle file splitting and large file sizes, I’ve simply created a 32-bit and 64-bit version of the packager. Code-wise there’s very few changes except for the data types. While the packager is not intended for files larger than 2GB, the option is there should I feel the need to waste my hard drive space.

Apologies for the wall of text, but as it’s a binary format I really don’t have any images to show for this post. If you’ve skimmed over it, hopefully it may be of some interest to you.