Skeetendo

’Cause all games were better on the GBC

You are not logged in.

  • Index
  • → Help/Question
  • → Pokemon Graphics Compression Help (Updated with new question!)

#1 2014-03-22 17:35:46

Inky
New member
Registered: 2014-03-22
Post 1/3

Pokemon Graphics Compression Help (Updated with new question!)

Hello! I've been lurking around this site because I'm currently working on a Pokemon Red hack. This is my first time working with anything that wasn't basic gen 3 hacking, so a lot of this is new ground for me. Everything seemed to be going wonderfully, too, until I realized the need to compress some sprites... stag019's gen 1 graphics compressor brings me to a 404 page. So I looked around, found this topic ( http://hax.iimarck.us/post/29261/ ) and found out that it's gone. The topic says the source code is still around, but I'm not really sure what that entails. (I'm not used to coding things) Does that mean it could be reconstructed? Or am I out of luck?

Thanks in advanced! PS- Your community looks really awesome! :)

Last edited by Inky (2014-03-22 21:58:57)

Offline

#2 2014-03-22 19:39:06

Hat
Member
Registered: 2013-02-04
Post 109/118

Re: Pokemon Graphics Compression Help (Updated with new question!)

Velcome to ze forum. Take a look at zis thread:
http://hax.iimarck.us/topic/3230/16/

Offline

#3 2014-03-22 20:54:46

comet
Member
Registered: 2012-04-09
Post 410/672

Re: Pokemon Graphics Compression Help (Updated with new question!)

The source is in the thread you linked. It can be built using a C compiler such as gcc.

This comes up a lot, so someone with a permanent domain would do well to host it.

Last edited by comet (2014-03-22 20:58:56)

Offline

#4 2014-03-22 21:56:25

Inky
New member
Registered: 2014-03-22
Post 2/3

Re: Pokemon Graphics Compression Help (Updated with new question!)

Okay, I got the compressor to work! Thanks!

I have another question now. I'm trying to alter the protagonist's sprite of Pokemon Red- I changed all of the overworld sprites as well as the backsprite already. However, I noticed that the compressed front sprite comes out to 345 bytes, when the original amount should be 266 bytes...

Now, I understand theres a way to repoint bytes and all, but I'm not quite sure where to even begin with that, especially considering I'm always hearing how the original Pokemon games had barely any empty space... It can't be easy. If worse comes to worse, I'll probably have to find empty space and repoint bytes... but in the meantime- what exactly is making my file size shoot up so far? I'm fairly certain that I'm only using 4 colors, same size...etc. I can't imagine that sprite complexity is taken into account, but if it is? How would I go about shrinking my sprite's bytes?

For the reference, this is my sprite:
vLn4oXg.png

Offline

#5 2014-03-22 22:34:54

stag019
Idea Killer
Registered: 2011-01-05
Post 578/630

Re: Pokemon Graphics Compression Help (Updated with new question!)

comet wrote:

This comes up a lot, so someone with a permanent domain would do well to host it.

Yes.

Inky wrote:

I can't imagine that sprite complexity is taken into account...

It does. A lot. The Generation I compressor is highly efficient and the less complex it is, the less space it takes up and vice versa.

Where is the original pic stored? This would help me find all the pointers and find some free space to put it. I used to have my own list of them all, but with the idiot that hosted me cutting me off, I don't even know anymore...

Edit: Found it. Look in the disassembly for "RedPicFront"; those are all the pointers to it. You should hopefully have enough free space at the end of bank 4 to put it there.

Last edited by stag019 (2014-03-22 22:37:19)


You can try to hide yourself in this world of pretend; when the paper's crumpled up, it can't be perfect again.

Offline

#6 2014-03-22 23:05:54

comet
Member
Registered: 2012-04-09
Post 411/672

Re: Pokemon Graphics Compression Help (Updated with new question!)

stag019 wrote:

Look in the disassembly for "RedPicFront"; those are all the pointers to it.

Some of them are missing or outright wrong ("Unknown_72ede"). Here's a full list:

00:3597     ld de, RedPicFront
01:618c     ld de,RedPicFront
01:618f     ld bc,(Bank(RedPicFront) << 8) | $00
01:61c2     ld de,RedPicFront
01:61c5     ld bc,(Bank(RedPicFront) << 8) | $00
01:6991     ld de, RedPicFront ; $6ede
01:6994     ld b, BANK(RedPicFront)
04:749a     ld de,RedPicFront
04:749d     ld bc,(BANK(RedPicFront) << 8) | $01
0f:705b     ld a, BANK(RedPicFront)
1c:433e     ld de, RedPicFront ; $6ede
1c:4341     ld a, BANK(RedPicFront)

Edit: This has been fixed in pokered.

Last edited by comet (2014-04-06 05:01:03)

Offline

#7 2014-03-22 23:58:22

Inky
New member
Registered: 2014-03-22
Post 3/3

Re: Pokemon Graphics Compression Help (Updated with new question!)

ijjZERv.png

I took some detail out, for now, but in the future i might figure out all that pointer stuff for the full quality!
Thanks everyone! ;u;

Offline

#8 2014-03-23 10:04:56

YORAMRW
Member
From: The Netherlands
Registered: 2013-03-29
Post 1,053/1,194
Website

Re: Pokemon Graphics Compression Help (Updated with new question!)

Inky wrote:

vLn4oXg.png

Cool sprite. Even the simple version on the emulator has still much 3D effect in it.


Hacks Gameboy 8-bit music in Pokémon Gen I & II, composes 8-bit music in FamiTracker

Offline

#9 2014-03-23 11:46:03

Danny-E 33
Administrator
Registered: 2012-06-09
Post 759/1,023

Re: Pokemon Graphics Compression Help (Updated with new question!)

Here is stag's compressor as well as a batch file that he wrote to speed up compressing multiple sprites at once.

All you do is put all the .2bpp files that need to be compressed in the 'input' folder and then double-click compress.bat.


Red Hack: Pokémon Prototype

Total number of registered users: 7000+
Total number of active users: ~12

Offline

#10 2014-03-23 20:50:04

stag019
Idea Killer
Registered: 2011-01-05
Post 580/630

Re: Pokemon Graphics Compression Help (Updated with new question!)

Not that I licensed it or anything (which I should do), but Danny, can you include the source with the release?

Actually....

There I included a license with it (and I changed some source code). Do you wanna compile this for me, Danny, and include the source and zip it up for me?

The source is located here.

Edit: Oh yeah, I should probably mention that one of the alterations is that if you don't supply an output filename, it uses the input filename, and replaces the (last) extension with ".bin". So now you can, for example, drag and drop a file onto it in Windows so you don't need the command line at all.

Last edited by stag019 (2014-03-23 21:46:59)


You can try to hide yourself in this world of pretend; when the paper's crumpled up, it can't be perfect again.

Offline

#11 2014-03-31 23:23:22

Danny-E 33
Administrator
Registered: 2012-06-09
Post 764/1,023

Re: Pokemon Graphics Compression Help (Updated with new question!)

Alright, the new version of the compressor with the source code is here: https://www.mediafire.com/?f8iyfrhszx7dlvw
It's usage is super simple. Just dump all the .2bpp files in "input" and double click "compress.bat" and all the compressed .pic files will be in "output".


Red Hack: Pokémon Prototype

Total number of registered users: 7000+
Total number of active users: ~12

Offline

#12 2014-04-05 22:35:51

comet
Member
Registered: 2012-04-09
Post 420/672

Re: Pokemon Graphics Compression Help (Updated with new question!)

When porting stag's compressor to python I noticed that the majority of compressed pics (650 of 658) were identical to the originals.

The ones that didn't make it were still the same size. I wasted a bunch of time looking for a pattern in case it was a sorting problem. Then it hit me--we were looking at length in bytes, but the entire compression scheme revolves around bitstreams. Choosing the smallest pic, in bits this time, worked for almost all of them.

Incidentally, the python is really slow. The first compression mode makes order redundant, so I dummied it out. Now they all work.

Here are my changes to pkmncompress.c.

--- a/pkmncompress.c
+++ b/pkmncompress.c
@@ -253,7 +253,7 @@
     free(bitgroups);
     free(_1_RAM);
     free(_2_RAM);
-    return curbyte + 1;
+    return (curbyte + 1) * 8 + curbit;
 }
 
 int compress(uint8_t *data, int width, int height)
@@ -261,7 +261,10 @@
     uint8_t *RAM_1;
     uint8_t *RAM_2;
     int i;
+    int mode;
+    int order;
     int newsize;
+    int compressedsize;
     int size = -1;
     uint8_t *current = NULL;
 
@@ -277,29 +280,36 @@
         RAM_2[i] = data[(i << 1) | 1];
     }
 
-    for(i = 5; i > -1; i--)
+    for(mode = 1; mode < 4; mode++)
     {
-        newsize = interpret_compress(RAM_1, RAM_2, i / 2 + 1, i % 2);
-        if(size == -1 || newsize < size)
+        for(order = 0; order < 2; order++)
         {
-            if(current != NULL)
+            if(!(mode == 1 && order == 0))
             {
-                free(current);
+                newsize = interpret_compress(RAM_1, RAM_2, mode, order);
+                if(size == -1 || newsize < size)
+                {
+                    if(current != NULL)
+                    {
+                        free(current);
+                    }
+                    current = (uint8_t*) calloc(0x310, 1);
+                    memcpy(current, compressed, newsize / 8);
+                    free(compressed);
+                    size = newsize;
+                }
             }
-            current = (uint8_t*) calloc(0x310, 1);
-            memcpy(current, compressed, newsize);
-            free(compressed);
-            size = newsize;
         }
     }
     compressed = (uint8_t*) calloc(0x310, 1);
-    memcpy(compressed, current, size);
+    compressedsize = size / 8;
+    memcpy(compressed, current, compressedsize);
     free(current);
 
     free(RAM_1);
     free(RAM_2);
 
-    return size;
+    return compressedsize;
 }
 
 int main(int argc, char *argv[])
@@ -385,3 +395,4 @@
 
     return EXIT_SUCCESS;
 }

Last edited by comet (2014-04-05 22:36:05)

Offline

#13 2014-04-06 00:07:34

stag019
Idea Killer
Registered: 2011-01-05
Post 584/630

Re: Pokemon Graphics Compression Help (Updated with new question!)

comet wrote:

Then it hit me--we were looking at length in bytes, but the entire compression scheme revolves around bitstreams. Choosing the smallest pic, in bits this time...

I'm glad someone here is smart enough to think outside the box. I may be able to write a compressor in C, but I'm sure as hell too dumb to have ever thought of this.

Do you plan on including the C or your python translation in pokered? And how slow is the python?


You can try to hide yourself in this world of pretend; when the paper's crumpled up, it can't be perfect again.

Offline

#14 2014-04-06 19:02:40

comet
Member
Registered: 2012-04-09
Post 421/672

Re: Pokemon Graphics Compression Help (Updated with new question!)

$ time python extras/pokemontools/pic.py compress pic/*/*.2bpp

real    0m20.325s
user    0m20.045s
sys     0m0.188s
$ time ./pkmncompress pic/*/*.2bpp

real    0m1.352s
user    0m1.232s
sys     0m0.112s
$ time python extras/pokemontools/pic.py compress pic/{bmon,monback,trainer,other}/*.2bpp

real    0m9.134s
user    0m8.793s
sys     0m0.312s
$ time ./pkmncompress pic/{bmon,monback,trainer,other}/*.2bpp

real    0m0.649s
user    0m0.576s
sys     0m0.064s

Offline

#15 2014-04-07 13:01:31

stag019
Idea Killer
Registered: 2011-01-05
Post 585/630

Re: Pokemon Graphics Compression Help (Updated with new question!)

[04:05:29] <Sanky> what about pypy?

Also I wanna do some testing, but I'm too lazy to try to download and convert all the pngs, and all your branches either have the compressed .pics or the pngs. Would you be able to convert them all to .2bpp and zip them up in the same directory structure as the repo?


You can try to hide yourself in this world of pretend; when the paper's crumpled up, it can't be perfect again.

Offline

#16 2014-10-29 07:00:52

comet
Member
Registered: 2012-04-09
Post 488/672

Re: Pokemon Graphics Compression Help (Updated with new question!)

In the same vein as before, I'm attempting to recreate HAL's LZ implementation used in GSC. I've run into something of a wall.

How it works is the compressor tabulates a score for each possible command. It picks the one that gives the smallest output. If no commands score above a minimum threshold, the data is packed into a literal command until something gets a high enough score.

Here is a diff of the command lists between the original falkner.lz and the one created by my compressor:

@@ -1,9 +1,9 @@
 blank: 149    ec 94
 literal: 17   10 06 03 09 0f 08 07 14 27 22 1d 52 7f 52 2b 3f 1d 1d
 blank: 35     ec 22
-iterate: 3    22 01
-reverse: 8    c7 81 [01 01 00 00 00 00 00 00]
-literal: 12   0b 01 01 00 02 04 04 02 0a 08 09 07 0f
+iterate: 5    24 01
+repeat: 8     87 8a [00 00 00 00 00 00 01 01]
+literal: 10   09 00 02 04 04 02 0a 08 09 07 0f
 blank: 8      67
 literal: 20   13 03 03 0c 0f 10 1f 22 3f 04 7f 89 ff 91 ff 53 7f 23 3f 27 3f
 alternate: 4  43 17 1f

Strangely, the original has its iterate command cut short to make way for a reverse. The length is identical to the repeat in the other version. Incidentally the literal stream is 2 bytes longer in the original, but it looks like coincidence.

My first thought was to remove iterate from the scoring bracket. Bad idea. Definitely not it. The only other way I can imagine this going down is if an iterate-5 command is output, and then the compressor notices it can get a reverse in if it backtracks. It sounds too expensive and pointless to boot.

So I'm lost. I thought maybe I could rubber duck my way through this one but it hasn't worked so far.

Offline

  • Index
  • → Help/Question
  • → Pokemon Graphics Compression Help (Updated with new question!)

Board footer

Powered by FluxBB