Gba on A blog by Dennis Hilhorst

GBA Audio Data

Sat, 23 Mar 2024 00:00:00 +0000

[latexpage]

About a year ago, I was working on porting Pokemon Ruby from the GBA to the PC (including some extra m4a engine decompilation work by Karaukasan, most notably the music_player.c file). After getting it working, with emulated (threaded) graphics and audio, I realized the game was not running any faster than my GBA emulator DSHBA, which I found surprising, considering a native game should probably run faster (right?). After profiling the ported game, I saw that audio emulation was taking up most of the CPU resources (by far!). I figured that, considering I have all the decompiled code available, there should be a way to run the audio engine natively (and threaded!) as well.

This is why I decided to research into the m4a engine, and, particularly, how the GBA stores / plays audio. I mean, the mp3 songs you might listen to take up multiple megabytes even for one or two minutes of play time, while a GBA ROM can only store 2MB of data total. Of course, a highly efficient way of storing song data was used, similar to the MIDI format. Programs to export sound already exist (like agbplay or the agb2mid tool in the zelda tmc decomp, or many pret decompilation projects). I decided to write my own GBA audio extractor for the learning experience.

The Hardware

As an emulator developer, I am aware of the GBA audio hardware to a certain extent, and know how the GBA is supposed to play audio when IO registers are written to (see GBATek or even the GB Pandocs, considering the GBA uses mostly the same audio registers, besides direct audio). The GBA has 4 audio channels: 2 square wave channels, a wave channel with 32 (or 64) programmable samples, a noise channel and direct 8 bit audio. Games write to IO registers to control the channels, but how does the audio engine do this to make a song?

GBA Sound Format

A lot of work has been done (see for example an attempted pc port of pokemon emerald or agbplay), but documentation is sparse and incomplete. I will be using code from the Zelda: the Minish Cap decompilation project, as well as my own code to explain how everything works. The process works a bit like the following. There are 3 main m4a engine control functions:

void m4aSoundInit(void); // initializes the engine and the hardware, called on program startup
void m4aSoundMain(void); // mixes audio samples and sends them to the direct audio controller, called regularly (every VBlank or HBlank?)
void m4aSoundVSync(void); // triggers an audio DMA to transfer data from a buffer to the audio channels

Then there are some more controller functions, starting or stopping songs or sounds, controlling the master volume or other effects.

void m4aSongNumStart(u16 n);
void m4aSongNumStartOrContinue(u16 n);
void m4aSongNumStop(u16 n);
void m4aMPlayAllStop(void);
void m4aMPlayImmInit(MusicPlayerInfo* mplayInfo);
void m4aMPlayTempoControl(MusicPlayerInfo* mplayInfo, u16 tempo);
void m4aMPlayVolumeControl(MusicPlayerInfo* mplayInfo, u16 trackBits, u16 volume);
void m4aSoundVSyncOn(void);
void m4aSoundVSyncOff(void);

These controller functions update MusicPlayer structs, which contain the state of a single song / sound that may be played. Some comments have been added to certain fields, which will be clear later.

struct MusicPlayerInfo {
 const SongHeader* songHeader; // songheader pointing to tracks / voice group
 u32 status;
 u8 trackCount; // number of tracks
 ...
 MusicPlayerTrack* tracks; // track list
 ToneData* tone; // voice
 u32 ident; // lock
 MPlayMainFunc func;
 u32* intp;
};

struct MusicPlayerTrack {
 u8 flags; // state
 u8 wait; // counter
 u8 patternLevel; // nested pattern depth
 u8 repN;
 u8 gateTime; // time to play a note for
 u8 key; // key of the current playing note
 u8 velocity; // velocity of the current playing note
 u8 runningStatus; // last repeatable event (cmd >= 0xbd)

 // keyShift, pitch, bend, volume, pan, mod, lfo, echo, etc. stuff

 ...
 SoundChannel* chan; // soundchannel to play sound on
 ToneData tone; // voice to play sound with
 ...
 u8* cmdPtr; // current position in command stream
 u8* patternStack[3]; // PEND return addresses
};

typedef struct MusicPlayer {
 MusicPlayerInfo* info;
 MusicPlayerTrack* tracks;
 u8 nTracks;
 u16 unk_A;
} MusicPlayer;

Most notably, the music player holds info on the song (a song consists of multiple tracks with audio data, and a pointer to an array of ToneData structs, which is a table containing sound samples which the tracks may use, which we refer to as voices). Then there is some more random struct fields which are used to track the status of certain audio effects like panning, echo, pitch shifting etc.

This already gives some insight into how the data is stored: as songs, consisting of tracks, using certain voices to play tones with different sounds. A game usually contains a table of songs, called the songtable (commonly named gSongTable in decompilation projects. Songs contain an ID for a music player, which is then triggered to stop playing the song it was previously playing, and start playing the requested song. This means that the m4a engine can play only a limited amount of songs (and sound effects!) at the same time (commonly 32, so quite a few).

Songs consist of tracks and a pointer to a voice group. The tracks contain the parts of the song that is played, like in a real orchestra! A track may contain music for multiple different instruments, but only one instrument (or voice!) is playing at once. Though a game may have many voice groups stored in it’s data, only one can be selected per song, and only one voice within this group can be active at a time, per track. This means that if a song contains 7 tracks, we may hear 7 different instruments at the same time! A voice group is an array of voices. A voice contains data that describes the sound of an instrument. This may be in the form of CGB audio channel controls, wave data or a direct sound sample.

So how is all the data for a song stored? Well, a track consists of a stream of events, which may be either a note, a controller event, or a wait event. Music players have an internal, regularly ticked counter, ticking down until the next events should be played, or until the counter is incremented again by a wait event. Let us briefly look at my own implementation for the data in a song. The data defined as below. Note that this does not contain any information for actually playing a song, we will look into that later.

struct Event {
 enum class Type {
 Meta = 0,
 Goto, Tempo, VoiceChange, Controller, Note, Fine,
 };

 Type type;
 i32 tick;

 union {
 ...
 };
};

struct Track {
 std::vector<Event> events{};
 u32 length;
 ..
};

struct Song {
 ...
 u8 reverb;
 bool do_reverb;
 VoiceGroup voicegroup;
 std::vector<Track> tracks;
 ...
};

The songtable (gSongTable) contains pointers to song headers, which look like this

typedef struct SongHeader {
 u8 trackCount;
 u8 blockCount;
 u8 priority;
 u8 reverb;
 ToneData* tone;
 u8* part[1];
} SongHeader;

Where the ToneData points to the voice group, and the part points to the tracks. So we extract a song like this:

#define HEADER_NUM_TRACKS_OFFSET 0
#define HEADER_REVERB_OFFSET 3
#define HEADER_VOICEGROUP_PTR_OFFSET 4
#define HEADER_TRACK_PTR_OFFSET 8

Song Song::Extract(const u8* header) {
 const u8 num_tracks = header[HEADER_NUM_TRACKS_OFFSET];
 const u8 reverb = header[HEADER_REVERB_OFFSET];
 const u32 voicegroup_ptr = util::Read<u32>(&header[HEADER_VOICEGROUP_PTR_OFFSET]);

 Song song = Song(num_tracks, reverb, util::GetPointer(voicegroup_ptr));

 for (int i = 0; i < num_tracks; i++) {
 auto* track_ptr = util::GetPointer(util::Read<u32>(&header[HEADER_TRACK_PTR_OFFSET + sizeof(u32) * i]));
 song.AddTrack(track_ptr);
 }

 return song;
}

Events

A track is then simply a long stream of events (or commands). These events are encoded in one or multiple bytes. A byte with a value of at least 0x80 indicates a command, and a byte with a value lower than 0x80 is a data byte, which is either a parameter of a previous command byte, or indicates a repeat command, repeating the previous repeatable command, with that byte as first parameter. Events happen at certain ticks, forming a kind of “timeline”, like you can see in the preview image of this post (of my gba audio extractor). The duration of one tick depends on the tempo of the song, which should be indicated by one of the first events of the track, happening at tick 0. During the song, the tempo of a track might change. Using the very useful table found here, the agb2mid tool, the already decompiled audio engine by Kurausukun and extracted audio from pokemon ruby, we can formulate a table of audio commands, their parameters and their meaning. Some commands are missing, but I have not encountered these in the ROMS that I have tested, nor are they in agb2mid.

Command	Symbol	Parameters	Repeatable	Description
`0x00-0x7F`	n/a	n/a	Yes	Data byte OR repeat command, repeating the previous repeatable command using the command byte as first argument
`0x80-0xB0`	`W00-W96`		No	Wait
`0xB1`	`FINE`		No	Track end
`0xB2`	`GOTO`	`u32 dest`	No	Jump to specified address (absolute hardware address like 0x8XXXXXX)
`0xB3`	`PATT`	`u32 dest`	No	Pattern start (subroutine jump, cannot be nested)
`0xB4`	`PEND`		No	Pattern end
`0xBA`	`PRIO`	`u8`	No	Priority of track (0-255, the higher the value, the higher priority)
`0xBB`	`TEMPO`	`u8`	No	Tempo, specifies half the value of BPM
`0xBC`	`KEYSH`	`i8`	No	Transpose (-128-127, per-track transpose)
`0xBD`	`VOICE`	`u8`	Yes	Instrument
`0xBE`	`VOL`	`u8`	Yes	Volume
`0xBF`	`PAN`	`u8`	Yes	Pan (0: left, 64: center, 127: right)
`0xC0`	`BEND`	`u8`	Yes	Pitch bend (0-127, center is 64)
`0xC1`	`BENDR`	`u8`	Yes	Pitch bend range (specified in semitones, default is 2)
`0xC2`	`LFOS`	`u8`	No	LFO speed (higher values are faster, actual speed fluctuates with tempo)
`0xC3`	`LFODL`	`u8`	No	LFO delay (specified in ticks)
`0xC4`	`MOD`	`u8`	Yes	LFO depth
`0xC5`	`MODT`	`u8`	No	LFO type (0: pitch (default), 1: volume, 2: pan)
`0xC8`	`TUNE`	`u8`	Yes	Micro-tuning (0: one semitone lower, 64: normal: 127: one semitone higher）
`0xCD`	`XCMD`	`u8 cmd [, u8 param]`	Yes	Extended commands
`0xCE`	`EOT`	`[u8 key]`	Yes	Tie end / Note off
`0xCF`	`TIE`	`[u8 key [, u8 velo]]`	Yes	Tie / Note on (note length is undeterminated until the `0xCE` command appears)
`0xD0-0xFF`	`N01-N96`	`[u8 key [, u8 velo [, u8 length]]]`	Yes	Note (the third argument is for fine adjustment of gate time, specified in ticks)

Command table adapted from loveemu.github.io

For the full code used to parse a track, see track.cpp. The very basic idea is that we keep a counter for the tick that an event is on, which we increment on wait commands (W00-W96), and store in any other command. Note that a note command does not increment the current tick. In order to play two consecutive 32-tick length notes, one needs commands

N32, W32, N32

The available lengths for both wait and note commands are the following:

static constexpr std::array<i32, 49> LengthTable = {
 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
 17, 18, 19, 20, 21, 22, 23, 24, 28, 30, 32, 36, 40, 42, 44, 48, 52,
 54, 56, 60, 64, 66, 68, 72, 76, 78, 80, 84, 88, 90, 92, 96
};

Wait commands and timing

As mentioned, event timing is determined in terms of ticks and wait commands which dictate the delay for the following events. How long a tick takes is determined by the TEMPO command. The parameter passed to the command indicates half of the BPM. One beat in this case is 24 ticks, so a 4-beat measure in a song is made up of 96 ticks, allowing for an easy way of defining beat divisions (eighth = 12 ticks, sixteenth = 6 ticks, 32th notes = 3 ticks, triplets = 8 ticks etc.). We compute the real time per tick as

static inline double TimePerTick(u32 bpm /* == 2 * the TEMPO command parameter */) {
 // 96 ticks per 4 beats
 return (60.0 / bpm) / 24.0; // seconds
}

Indeed, this means that a bpm of 150 (so a value of 75 as parameter for TEMPO) has precisely 1 tick per frame.

Notes

Besides wait commands, which dictate the timing of all other events, the simplest events are notes. The command byte dictates the length of the note, as one of the options from the array above. Then there are some (optional) parameters:

u8 key. This parameter indicates the key of the note. If this parameter is not passed, the key of the previous note in the track is used. Of course, this means that the very first note of a track MUST indicate a key. The key value is the midi key (yes, the midi key from the well-known MidiKeyToFreq BIOS routine used to dump the GBA BIOS) that indicates the frequency of a note. How this key is used to play a sound at the correct frequency depends on the type of voice that is currently active on the track. The note corresponding to the MIDI key value can be read in the following table:

const std::array<const char*, 128> NoteNames = {
 "CnM2", "CsM2", "DnM2", "DsM2", "EnM2", "FnM2", "FsM2", "GnM2", "GsM2", "AnM2", "AsM2", "BnM2",
 "CnM", "CsM", "DnM", "DsM", "EnM", "FnM", "FsM", "GnM", "GsM", "AnM", "AsM", "BnM",
 "Cn0", "Cs0", "Dn0", "Ds0", "En0", "Fn0", "Fs0", "Gn0", "Gs0", "An0", "As0", "Bn0",
 "Cn1", "Cs1", "Dn1", "Ds1", "En1", "Fn1", "Fs1", "Gn1", "Gs1", "An1", "As1", "Bn1",
 "Cn2", "Cs2", "Dn2", "Ds2", "En2", "Fn2", "Fs2", "Gn2", "Gs2", "An2", "As2", "Bn2",
 "Cn3", "Cs3", "Dn3", "Ds3", "En3", "Fn3", "Fs3", "Gn3", "Gs3", "An3", "As3", "Bn3",
 "Cn4", "Cs4", "Dn4", "Ds4", "En4", "Fn4", "Fs4", "Gn4", "Gs4", "An4", "As4", "Bn4",
 "Cn5", "Cs5", "Dn5", "Ds5", "En5", "Fn5", "Fs5", "Gn5", "Gs5", "An5", "As5", "Bn5",
 "Cn6", "Cs6", "Dn6", "Ds6", "En6", "Fn6", "Fs6", "Gn6", "Gs6", "An6", "As6", "Bn6",
 "Cn7", "Cs7", "Dn7", "Ds7", "En7", "Fn7", "Fs7", "Gn7", "Gs7", "An7", "As7", "Bn7",
 "Cn8", "Cs8", "Dn8", "Ds8", "En8", "Fn8", "Fs8", "Gn8",
};

u8 velocity. This parameter indicates the velocity of a note. In this case, velocity does not indicate any real speed-related quantity, but rather “how hard the piano key was pressed”. That is to say, how loud the note is. The amplitude of the sound wave is linearly related to the velocity parameter. If this parameter is not passed, the velocity of the previous note in the track is used. Again, the first note in the track MUST specify a velocity. This parameter can only be passed if the key parameter is also passed.
u8 length. This parameter can be used to increment the length of the note to get more exotic note lengths, other than that from the command byte, without using a TIE command.

Ties

There is one other way to play a note, besides using a note command with a predefined length: with the TIE/EOT commands. The TIE command starts a note at the given key, which is ended on an EOT command with the same key. This means that, within a track, multiple ties may be playing at once, but only one tie can be playing per note (after all, what would it mean for two ties to be playing at the same note…). The parameters in the TIE command mean the same thing as for the note command, where the default value is determined by the previously played note (this includes tie commands, as does it for the note and EOT commands). The TIE command is commonly used for playing long notes, taking more than 96 ticks.

Voices

We know about notes, but how do we know how the note should sound? Should it be a piano or a guitar? A square wave or a sine wave? This is dictated by the voice that is active in the track. These can be controlled by the VOICE command, providing an index into the voice group for the voice that is used. There is only one voice group per song, though a voice group may be used by multiple songs. There are a few different types of voices, but all of them have some things in common: volume envelope. This term may be familiar to those who have emulated a GBA or GB before, though the actual idea may not be very clear.

Volume envelope

A brief intermezzo: the volume envelope in a voice is a way of simulating an instrument by modulating the volume of the waveform. Basically, by defining attack, decay, sustain and release parameters, we can create the following volume progression over time:

source

Basically, we can simulate the volume progression of a piano key being struck (or a string being struck, or a voice singing a tone, or a brass instrument playing a tone…). When a brass player starts playing a tone, the volume rises quickly, then decays until it hits the volume that the tone is sustained at for the duration of the note, and it might echo or decay very quickly after it has been released. The source of the image gives another very good example: the sound of a person yawning has a very long attack time. The sound volume rises pretty slowly, even when the action of the yawning has begun. Compare this to someone hitting a snare drum: the sound explodes immediately, so the attack time is very short!

This gives rise to the following data in my generic Voice struct:

struct Voice {
 enum Type : u8 {

 ...
 };

 u8 type; // value depends on subtype
 u8 base;
 u8 pan;
 u8 attack;
 u8 decay;
 u8 sustain;
 u8 release;

protected:
 ...

 double attack_time;
 double decay_time;
};

The public u8 type parameters are all in the data in the ROM, while the last 2 protected values are computed once on instantiation. For the attack, decay, sustain and release parameters, the values mean the following:

Attack: The attack volume is incremented by the attack value every frame, giving us a linear attack increment lasting attack_time = FrameTime * (255.0 / attack), meaning that an attack value of 255 (which is very common) means we reach maximum volume in a single frame.
Decay: The decay is exponential, multiplying the volume by decay / 255.0 every frame until the volume reaches the sustain level. A value of 0 means we reach the sustain volume immediately.
Sustain: This parameter indicates a level, sustaining the volume at level sustain / 255.0 until the note ends.
Release: This parameter again defines an exponential decay, reducing the volume by a factor of release / 255.0 every frame after the note has ended.

The other parameters in the Voice struct will be explained later, but they are related to audio effects. Let us now go over the different types of voices.

Types of voices

There are a couple different types of voices, all encoded differently. Every voice starts with a single byte indicating the type of voice. Every voice consists of the following (see for example music_voice.inc from pokeemerald).

1 byte indicating the voice type
1 byte indicating the voices 'base key', dictating how a specific midi key frequency is generated
2 bytes containing pan / sweep data depending on the voice type (A, B)
4 bytes of voice type dependent data (C)
1 byte with the attack value
1 byte with the decay value
1 byte with the sustain level
1 byte with the release level

The voice types are the following:

Voice Type	Byte	A	B	C
DirectSound	`0x00` or `0x10`	SBZ	pan	`WaveData` pointer
DirectSound without resampling	`0x08`	SBZ	pan	`WaveData` pointer
Square & sweep (CGB channel 1)	`0x01` or `0x09`	pan	sweep	1 byte duty cycle type 3 bytes SBZ
Square 2 (CGB channel 2)	`0x02` or `0x0a`	pan	n/a	1 byte duty cycle type 3 bytes SBZ
Waveform (CGB channel 3)	`0x03` or `0x0b`	pan	SBZ	Wave RAM data pointer
Noise (CGB channel 4)	`0x04` or `0x0c`	pan	SBZ	1 byte period type 3 bytes SBZ
Keysplit	`0x40` or `0x80`	SBZ	SBZ	Voicegroup pointer

The precise meaning of the alternative encodings is unclear to me, for audio extracting it did not seem to matter much.

Squarewave voices These are perhaps the simplest voices. They generate sound using the corresponding CGB wave channels. The duty cycle parameter means the same as the value in the corresponding IO register, meaning that the values 0, 1, 2 and 3 correspond with a wave value of 1 for 12.5%, 25%, 50% or 75% of the time.
Waveform voices These use CGB channel 3, and the wave RAM data pointer, containing 16 bytes (always 16 bytes, so 32 4-bit samples) of audio data.
Noise voices These are often used for percussion instruments, and use CGB channel 4. The period type parameter is the same as that in the corresponding IO register, meaning that a 0 indicates a 15 bit counter step width, and a 1 a 7 bit counter step width.
DirectSound voices These use DMA audio. The WaveData pointer points to a struct defined as below. I ignore the type. The data pointer points to an array of size samples. The freq parameter indicates the frequency at which the sample was recorded, dictating at what rate the waveform has to be resampled to produce the right sound (using the MidiKeyToFreq function!). The loopStart parameter indicates the index at which a possible loop in the data might start. Of course, we can’t store a waveform for an arbitrarily long tone, so we indicate the index into the wave data to which the engine has to loop back when it reaches the end of the data. The presence of such a loop is indicated by either of the 0xc000 bits in the flags parameter.

struct WaveData
{
 u16 type;
 u16 flags;
 u32 freq; // sample rate * 1024
 u32 loopStart; // index into samples at which loop starts
 u32 size; // number of samples
 i8* data; // samples
};

Keysplit voices These are special voices. Remember when I said there can only be one voicegroup per song? That was a little bit of a lie, but not entirely. Basically, keysplit voices point to another voice group. The midi key in a note now doesn’t indicate what tone should be played, but instead what index into the nested voice group should be used. The frequency is then determined either by the base key, or by the index itself (depending on the type byte of the keysplit voice type.

A precise parsing of voice data can be found in voices.cpp.

Control Flow Events

There are three types of control flow events:

GOTO: This event takes a pointer as parameter. When this event happens, the audio engine jumps to the pointer, and starts processing commands from there. Note that this is an unconditional jump, so it is only really useful to create a loop in a song, by placing a GOTO at the “end”.
PATT / PEND: These commands define a “pattern”. The engine jumps to the location indicated in the parameter of the PATT command, and starts reading from there, returning when it finds a PEND command, defining a sort of “subroutine” in the song. A song may nest patterns up to three layers deep.
FINE: This indicates the end of the track, telling the engine that there are no more commands to be read and that the song is over.

Generating audio

Before we go over to the last couple of events, I want to briefly discuss my idea behind generating audio based on the parsed data (tracks and voice types), in a GBA ROM. What I do is the following:

When parsing a track, I “unfold” all patterns and turn ties into notes with very high lengths. Basically, this simplifies a track a lot. Instead of having to jump to memory addresses, I simply have a long vector of notes and simple control events, and possibly a GOTO, which jumps to a specific time tick.
I have a TrackStatus struct, which I use to keep track of the current status of a track.

struct TrackStatus {
 // external control (enable / disable track from UI)
 struct ExternalControl {
 bool enabled = true;
 } external;

 // control variables
 i32 keyshift = 0;
 double vol = 0;
 i32 pan = 0;
 i32 bend = 0;
 i32 bend_range = 2; // default
 u32 lfo_speed = 22; // default
 u32 lfo_delay = 0;
 u32 mod = 0;
 u32 modt = 0;
 i32 tune = 0;

 // internal state
 i32 tick = 0; // current tick may vary between tracks because of GOTO events
 bool track_ended = false;

 const Event* current_event = nullptr; // points into track->events
 const Voice* voice = nullptr; // current voice
 std::list<PlayingNote> current_notes{}; // current playing notes

 ...
 };

Instead of counting down a timer, I count up a timer, which I reset on GOTO commands, and I simply handle all the events that happen at a specific tick.
- If this is a controller event, it happens immediately (usually only some parameter changes, or I set the current tick value if it is a GOTO event)
- If this is a note, I add the note to the track’s current_notes.

 struct PlayingNote {
 double time_started; // real time (seconds)
 i32 tick_started; // tick
 double time_released; // real time (seconds)
 const Note* note; // note data from track->events
 VoiceState voice; // state of voice, used to track "integrated time" for sampling waveform
 };

Whenever I request a sample, I increment the time by 1 / 44200 (the time per requested sample) and the following happens:
- If enough time has passed, a tick happens (depending on the “time per tick”, dictated by TEMPO events). If this is the case, I do the following for every track:
  - I check all the events that should happen. I set any controller event parameters in the track state and add any playing notes to the track’s current playing notes.
  - I tick the voice state of any playing notes, meaning that I keep track of the time properly, as to get the waveform at the correct frequency (as explained below).
  - I remove any notes that finish playing from the track’s current playing notes.
  - I compute a sample for the track, based on the current track’s control parameters, any time-dependent audio effects and the current playing notes.
- I blend all the track’s current samples together and return the result.

This can be seen in player.cpp.

Controller events

There are some events that manage certain “audio effects” (panning, pitch shifting etc.), or the general volume of a track. Some of these involve the LFO (Low Frequency Oscillator), which is controlled by the LFO- commands, setting the LFO speed, delay, depth and the modulation type modt. Each track has its own LFO, allowing for per-track volume / pitch control effects.

The LFO

The MODT command sets the mode of the LFO to either pitch (allowing for a “vibrato” effect), volume (also allowing for some sort of vibrato effect) or pan (allowing for a “panning” effect, making it sound like the sound origin is slowly moving from left to right). In the GBA, the LFO values are updated every frame, but by carefully studying its definitions in music_player.c by Karaukasan, we can define a direct formula for it:

\[ \texttt{modM}(t) = \begin{cases} 0 & \textrm{ if } t < \texttt{delay} \ast\texttt{FrameTime} \\ \texttt{depth} \ast\sin\left(2\pi\ast\frac{\texttt{speed}}{255}\ast\left(t - \texttt{delay}\ast\texttt{FrameTime}\right)\right) & \textrm{ if } t \geq \texttt{delay} \ast \texttt{FrameTime} \end{cases} \]

Commonly, the value of depth is 1, 2 or 3, causing these to be very slight adjustments.

Volume controller events

Some controller events affect the volume or the volume pan.

VOL sets the master volume of the track to a value between 0 and 255. We call the current track volume vol.
PAN defines a volume pan in the track, which gradually oscillates the volume on the left and right channel, creating a “panning effect”, making it sound like the sound origin is moving from left to right. We call the current track pan value pan.

The resulting (normalized) volume of the left and right channels is then computed as

PanVolume Player::TrackStatus::GetPannedVolume(double dt) const {
 double main_vol = vol;
 if (modt == 1) {
 main_vol *= (modM(dt) + 128.0) / 128.0;
 }

 double pan_vol = 2 * (pan - 0x40);
 if (modt == 2) {
 pan_vol += modM(dt);
 }
 pan_vol = std::clamp(pan_vol, -128.0, 127.0);
 
 return {
 .left = (127.0 - pan_vol) * main_vol / 256.0,
 .right = (pan_vol + 128.0) * main_vol / 256.0,
 };
}

Pitch controller events

Besides the volume, the pitch of a note can be adjusted as well. Since this does not affect the “actual waveform” of a voice, but only the frequency at which it plays, I continually compute “integrated time” in the current state of a voice, and then sample the voice’s waveform using the integrated time. That is, in my music player, I track “voice states”, and every time I request a sample (at a 44.2kHz rate), I add a very small time dt = 1.0 / 44200, which I then correct by the current frequency of the playing note (which may change through the controller events) to compute the “integrated time” to sample the waveform at. Volume envelope is not affected by the frequency of the played tone, meaning we also have to track “actual time”.

void VoiceState::Tick(i32 midi_key, double pitch_adjust, double dt) {
 if (base_key >= 0) { // only for keysplit voices
 // keep fine adjust
 midi_key = base_key;
 }

 const double freq = voice->GetFrequency(midi_key + pitch_adjust);
 integrated_time += dt * freq;
 actual_time += dt;
}

double VoiceState::GetSample(double time_since_release) const {
 const double wave_sample = voice->WaveForm(integrated_time);
 const double envelope_volume = voice->GetEnvelopeVolume(actual_time, time_since_release);
 return wave_sample * envelope_volume;
}

The pitch_adjust is affected by the controller events:

The KEYSH sets the track’s (signed) keyshift value, adjusting the midi key by full keys. This is not affected by the LFO in the pitch adjusting mode.
The TUNE command provides a constant “fine adjust” (fractional midi key) in the pitch. It is on a scale of -64 to 64, normalized to a range of -1 to 1 midi key.
The BENDR sets the track’s current bend range, providing further fine adjust. The “bend” is like a variable “fine adjust” which is controlled manually.
The BEND command sets the current track’s bend amount. This is how we manually adjust the “fine adjust”. The total “fine adjust” from the bending effect takes the bend value and multiplies it by the bend range.

In the end, a note’s midi key is determined as follows

double Player::TrackStatus::GetPitchAdjust(double dt) const {
 double adjust = 4 * (tune + bend * bend_range);
 if (modt == 0) {

 // pitch adjust: "vibrato effect"
 adjust += 16 * modM(dt);
 }

 return keyshift + (adjust / 256.0);
}

And for all the currently playing notes in a track, we get a new sample by ticking the voice state’s integrated time as

 const double pitch_adjust = status.GetPitchAdjust(global_time - note.time_started);
 const i32 keyshift = std::floor(pitch_adjust);
 const double fine_adjust = pitch_adjust - keyshift;
 const i32 key = note.note->key + keyshift;

 note.voice.Tick(key, fine_adjust, 1 / SampleRate);

Finally, the last step is how we determine the actual frequency from the midi key, the keyshift and the fine adjust. As seen before, the actual played midi key is determined by simply adding the midi key and the fine adjust caused by the keyshift and the tune / bend effects. The way we then determine the voice’s frequency depends on the voice type. Since keysplit voices act like any other voice type, depending on the selected voice through the played midi key (of course not adjusted by any pitch shifting mechanisms), we only have to consider the “CGB type voices” or the “directsound voices”.

CGB type voices (except noise) use the CgbMidiKeyToFreq function (which is not available in the BIOS). It is defined as below. The value 69 is the midi key for a neutral A (concert pitch at 440Hz), 12 means there are 12 midi keys (semitones) in an octave, meaning that an octave doubles the frequency (as expected). In the actual GBA, this value is then corrected by the right amount to be written to the corresponding CGB channel’s IO register.

static inline double CgbMidiKeyToFreq(double midi_key) {
 const double corrected_key = (midi_key - 69.0) / 12.0; // An3
 return std::pow(2.0, corrected_key) * 440;
}

Noise voices use a table, indexed by the midi key:

const std::array<double, 256> NoiseFreqTable = {
 4.57142857, 4.57142857, 4.57142857, 4.57142857, 4.57142857, 4.57142857, 4.57142857, 4.57142857,
 4.57142857, 4.57142857, 4.57142857, 4.57142857, 4.57142857, 4.57142857, 4.57142857, 4.57142857,
 4.57142857, 4.57142857, 4.57142857, 4.57142857, 4.57142857, 4.57142857, 5.33333333, 6.40000000,
 8.00000000, 9.14285714, 10.66666667, 12.80000000, 16.00000000, 18.28571429, 21.33333333, 25.60000000,
 32.00000000, 36.57142857, 42.66666667, 51.20000000, 64.00000000, 73.14285714, 85.33333333, 102.40000000,
 128.00000000, 146.28571429, 170.66666667, 204.80000000, 256.00000000, 292.57142857, 341.33333333, 409.60000000,
 512.00000000, 585.14285714, 682.66666667, 819.20000000, 1024.00000000, 1170.28571429, 1365.33333333, 1638.40000000,
 2048.00000000, 2340.57142857, 2730.66666667, 3276.80000000, 4096.00000000, 4681.14285714, 5461.33333333, 6553.60000000,
 8192.00000000, 9362.28571429, 10922.66666667, 13107.20000000, 16384.00000000, 18724.57142857, 21845.33333333, 26214.40000000,
 32768.00000000, 37449.14285714, 43690.66666667, 52428.80000000, 65536.00000000, 87381.33333333, 131072.00000000, 262144.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000, 524288.00000000,
};

Direct sound voices use the MidiKeyToFreq function, defined as below. Here, base is the sampled frequency for the sound sample in the voice (which is the sample rate times 1024). The default tuning in this case is to a neutral C, again an octave (consisting of 12 semitones) means we double the frequency, as expected.

static inline double MidiKeyToFreq(u32 base, double midi_key) {
 const double corrected_key = (midi_key - 60.0) / 12.0; // Cn3
 return std::pow(2.0, corrected_key) * (base / 1024.0);
}

Summary

I want to quickly give a summary of how the GBA plays sound:

Data is stored in terms of tracks and voices.
- Tracks are streams of events (wait events, notes, voice changes, tempos, control flow, volume / pitch control etc.). They each have their own sets of parameters describing what voice to use, what the pitch / volume adjustments are, and what audio effects should be used.
- Voices contain data describing how to generate audio, using either CGB or direct audio channels.
- The song table contains a list of the song headers that are used in the game, making it easy to find all the songs in a ROM.
Audio is generated by computing the needed samples every frame. This happens by
- Handling events by tracking currently played notes (adding new ones or removing old ones) and setting control parameters.
- Computing each track’s active notes’ waveforms by handling pitch shifting controls (possibly using the track’s LTO) and properly transforming the computed midi key and fine adjustment into a frequency at which the waveform should be sampled.
- Computing each track’s volume by handling volume changing controls (possibly using the track’s LTO) and mixing the tracks together.

Applications

So what are some other interesting purposes for all this information? Well, one that comes to mind is possible audio HLE in GBA emulation. This may be a significant performance boost, as my own emulator spends most of its time emulating audio. Since most games use this engine, it may be possible to find the main audio management functions through a byte pattern (take a look at saptapper for example, using byte patterns to find a game’s songtable and main audio functions) or some other methods, and then triggering a HLE function handling the engine’s capabilities. Most significantly, this saves a GBA game a lot of computation every VBlank for mixing audio samples from the tracks in a song.

Another application is of course audio extraction. As mentioned above, a game’s songtable (the main array of songheader pointers) may be found with a byte pattern, and used to extract data like I did in my GBA audio extractor, see mp2k_driver.cpp:

Final note

As a final note, I strongly encourage anyone to take a look at Karausukun’s music_player.c, just to see how much computation the GBA has to do to properly mix sound samples. It is good to keep in mind that all of this actually happens in the background (triggered by VBlank interrupts), and is mostly it’s own thing, running apart from the game. It should be very possible to HLE audio completely for a large library of games, since most games use this audio engine provided by Nintendo.

Sources

Important sources include:

GBA decompilation projects like zeldaret/tmc, pret/pokeruby or pret/pokeemerald
Karausukun’s work on decompiling / porting the m4a audio engine.
ipatix’s AGBPlay
Saptapper
Saptapper’s documentation

EEPROM Save Type

Mon, 01 Feb 2021 00:00:00 +0000

The GBA has games with different sorts of backup memory. There are 3 types:

SRAM, straightforward RAM on the cartridge
Flash, flash storage on the cartridge (explained here)
EEPROM, which I will attempt to explain in this document.

You can see what save type a cartridge uses most accurately by using a game database. If you don’t have one of those available, you can check for certain strings in the cartridge (also explained here).

GBATek also explains the EEPROM save type, but I found it quite brief, so I will try to add on to this documentation.

EEPROM types

There are 2 different types of EEPROM cartridges:

512 bytes / 4Kbit EEPROM
8KB / 64Kbit EEPROM

Which of these 2 a cartridge has is impossible to figure out just by looking at the cartridge. Your only options are a game database, or a trick where you check the first access made to it, which I will explain later.

Addressing and waitstates

The eeprom is connected to Bit0 of the data bus, and to the upper 1 bit (or upper 17 bits in case of large 32MB ROM) of the cartridge ROM address bus, communication with the chip takes place serially.

GBATek

This means that data transferred to or from the EEPROM chip is always 1 bit at a time. Any transfer made to EEPROM will be “masked” to only the bottom bit, and any read will just be 1 or 0.

On a large ROM (of greater than 16MB in size), ROM is restricted to 0x0800'000h-0x09ff'feff. So, EEPROM can be accessed between 0x09ff'ff00 and 0x09ff'ffff. This is also mirrored to the higher waitstate cartridge regions. Judging from the source code of certain emulators, it can really onlly be accessed in the 0x0dxx'xxxx region of ROM (second waitstate), despite what GBATek says. On smaller ROMs, it can also be accessed between 0x0d00'0000-0x0dff'ffff.

The actual address that is accessed for the EEPROM access does not matter, as the “internal address” has to be sent first, and then data can be written or read.

Data can be read or written. The initial pattern for the access is similar: the mode has to be sent, and the address as well.

This is also where the different EEPROM sizes come into play. The EEPROM can only transfer data in units of 64 bits. Addressing also works in units of 64 bits. This means, that while for a 512 byte EEPROM, you have 0x200 bytes to address, there are only 0x40 blocks of 64 bits. The address will only be in the range of 0 - 0x3f. The bus width for a 512 byte EEPROM is 6, and the address that will be sent will also be 6 bits long.

For an 8KB EEPROM, there are 0x2000 bytes, but only 0x400 blocks of 64 bits. The address will thus only be in the range 0 - 0x3ff. The bus width for a 8KB EEPROM is 14 bits, but the address only 10. The address that gets sent will be 14 bits long, but the first 4 bits should be zero, as they don’t correspond with any blocks.

It is important to actually have the addressing happen in blocks of 8 bytes / 64 bits. I did this wrong in my emulator at first, and it caused some sneaky corrupted saves.

Reading data

When you want to read data from the EEPROM, you have to send the following sequence of bits:

2 bits "11" (Read Request)
n bits eeprom address (MSB first, 6 or 14 bits, depending on EEPROM)
1 bit "0"
(GBATek)

EEPROM size detection

This n is what you can deduce the EEPROM size from. A trick to detect the EEPROM size is to keep it ambiguous until the first (read) request is made. Requests have to be done by DMA, since normal transfers via LDRH/STRH are too slow, and don’t keep the right bits set during the transfer.

Since DMA channel 3 is the only DMA channel that can access ROM, you could, on the first (read) access, check the transfer length of DMA channel 3. If it’s of length 9, a 6 bit address will be sent, and the EEPROM is (likely) a 512 byte EEPROM. If it’s of length 17, a 14 bit address will be sent, and the EEPROM is (likely) an 8KB EEPROM.

This method is not perfect though. Some games, like the NES classic series, try to trick you into thinking it’s the wrong EEPROM size, by doing a transfer of the “wrong” length. Your best bet will be a game database, or some sort of hybrid approach.

The transfer

Since it’s annoying to have to place individual bits at a (half)word interval in memory to then transfer data, a common approach for games/programs is to “shift” them into memory. Basically, if I wanted to send a read request to an 8KB EEPROM to block 0x123, I would need to transfer: (0b11 << 15) | (0x123 << 1) | 0 = 0x18246. Suppose r1 holds a pointer to a the end of a 17 halfword buffer where we want to store our bits, and r0 holds 0x18246. One could simply do

strh r0, [r1], #-2
lsr r0, #1

17 times, and the buffer will then be filled with

0x0001
...
0x3048
0x6091
0xc123
0x8246

such that the 0th bit of each halfword exactly reads out 0x18246 (MSB first). I can then transfer this buffer to EEPROM. Since only bit 0 is connected to the data bus, it does not matter that there is other data in the other 15 bits of each halfword.

This is how the address is transferred for a read access. After the address is transferred, we can read back the data. This again has to happen by DMA. There are 68 bytes to be read back with DMA:

4 bits - ignore these
64 bits - data (conventionally MSB first)
(GBATek)

These accesses have to be made in the same region as the address has to be written to, but just read instead of written. After this, it is up to the game how it handles the individual bits returned by the DMA.

Writing data

The written data immediately follows the address that is written. I have never encountered a game doing a write access before a read access, but it’s possible, so it might be good to check if this is the case when trying to detect the EEPROM’s size.

The data that has to be written is:

2 bits "10" (Write Request)
n bits eeprom address (MSB first, 6 or 14 bits, depending on EEPROM)
64 bits data (conventionally MSB first)
1 bit "0"
(GBATek)

So the start is similar to the read request, except a different “code”. Then the address, again different sizes depending on the EEPROM’s size. Then 64 bits / 8 bytes of data, and one bit to end the transfer.

These transfers might also be made in a similar way to the read access, where the data is first “shifted” to a buffer, and then transferred by DMA.

After a write transfer, games likely check if the transfer is complete. They do this by reading from the EEPROM and waiting until it returns 1. Martin Korth describes in GBATek how it’s important to set a timeout if the EEPROM does not respond, but some games might just hang if you don’t return 1 on reads after a write access.

Writing a cached interpreter

Sun, 31 Jan 2021 00:00:00 +0000

A while ago, I rewrote my GBA emulator (GBAC-) in C++ (DSHBA) to make it faster. I wanted to add a hardware renderer, and really focus on optimizing it all the way. After I was “done”, and had reached framerates higher than I had ever hoped before, I recently came back to it, and wanted to try to write a cached interpreter. I have heard a lot of talks about JITs and cached interpreters for performance gain, and since I had reached pretty insane framerates already, I wanted to go even further.

Note: this is not a general guide for writing a cached interpreter, and this approach might not be viable on every system. In this post I will explain why that is, and focus on my thought process and the implementation for my cached interpreter.

Why not a JIT?

Something you might ask is: “why did you write a cached interpreter and not a JIT”. The short answer to this is this: accuracy. The GBA is a fairly old system. Usually for older systems, timings are important. Where for modern systems, you don’t have to worry about cycles for the most part, other than being in the right ballpark (this is mainly because for more modern systems, the programmer himself can also rely less on specific timings due to pipelining, OoO execution and other advanced techniques), for the GBA you still sort of have to. A lot of tests, even in the official AGS Aging Cartridge test from Nintendo, some tests require accuracy within 16 cycles (in a frame of ~1200 cycles, or 1 scanline), or some even perfect cycle accuracy. Games may also rely on specific timings.

My old GBA emulator and my new one had the same performance in terms of accuracy. I passed most AGS tests, a lot of Endrift’s tests, but not any of the crazy perfect cycle accuracy ones. This was good enough for me, but I wouldn’t want to reduce accuracy for the sake of speed. These timings are easiest to emulate in an interpreter. In a JIT, you want to mostly just keep track of your GPRs (General Purpose Registers), and not of PPU events happening, or audio samples being requested, and most importantly: IRQs firing at the right times. These things, will often require me to check the scheduler, or break blocks early at “random” moments. These things combined led me to write a cached interpreter and not a JIT.

How does a cached interpreter work?

First of all, this is how my interpreter loop looked before my cached interpreter:

check scheduler:
 if there are events to handle: do events 
 otherwise: step once

Then let’s first look at the classic interpreter. The idea for this is very simple:

fetch opcode from memory
decode opcode 
run instruction handler

Alright, simple enough. This seems very straight forward, but there is a step here that we can remove. If we decode an instruction, and later on we get to the same address, we wouldn’t have to decode it again if we store the information on the instruction. This is “caching” the instruction. We can make it even better than this though, we can make entire blocks of instructions at once, so that we dont have to look up the cached instruction every time, but just look up a block and run for multiple instructions at once!

Now, the GBA memory map has some mirroring, and code cannot be ran from all regions. So I decided to split things up a bit. There are 3 regions I allow caching in:

the BIOS: this region cannot be written to, so once we make a cached block, we never have to delete it again!
ROM: this region is also not writeable, so we can save the cache blocks forever!
iWRAM: code often gets run from here because it takes very few cycles to fetch data from here. We have to handle this region slightly differently though, because code can get overwritten.

Cache “blocks” are blocks of instructions following each other that can be executed after each other. These blocks have to end somewhere, and the most logical places are branches or on certain boundaries. I set these boundaries at every 256 bytes. This number is rather arbitrary, but setting this number allows you to only delete certain blocks when writes to iWRAM happen.

The idea for the main loop of the emulator becomes slightly different:

check if current address has a cached block 
if it doesn't have one, and none can be created: 
 run interpreter normally, until it can 
if it doesn't have one, and one can be created:
 make a cache block and run interpreter normally*, but record the instructions
if it has one:
 run the cache block**

Now this is not the entire story. As I mentioned before, there is still a scheduler, and IRQs might happen that interrupt a block. Now what I could naively do (and did at first) is just break whenever we are interrupted by the scheduler. I don’t need to always break though. Some events, like adding a sample to the audio buffer, shouldn’t need to interrupt the CPU at all, since they don’t have anything to do with it!

I changed up my scheduler slightly, and made every event return a boolean, whether they directly affected the CPU state or not. Events that do are events like IRQs, or DMAs where an IRQ happened inbetween. The extra steps will be:

(*): check if the scheduler interrupted the CPU, and if so, return from making the block

(**): check if the scheduler interrupted the CPU, and if so, return from running the block

So in the end, the general idea will be this:

while true:
 check current address
 if we are not in a cacheable region (not in ROM/BIOS/iWRAM), do:
 while we are not in a cacheable region:
 check scheduler and run events 
 step, and check if we are in a cacheable region now.
 if we are in a cacheable region, but no cache exists:
 make a cache 
 while true:
 check the scheduler and run events
 if the scheduler affected our CPU: break the block and destroy it 
 fetch 
 decode and store in cache 
 run 
 if the block should end (branch): break 
 if we are in a cacheable region, and a cache exists:
 get the cache 
 for every instruction in the cache:
 check the scheduler and run events
 if the scheduler affected our CPU: break
 run the instruction (in ARM mode: only if condition is true)

This might seem a bit abstract, so here is some actual code. Before I added the cached interpreter, the main loop essentially looked like this (some details like pipeline stuff are left out):

while (true) {
 while (!Scheduler.ShouldDoEvents()) {
 CPU.Step();
 }

 Scheduler.DoEvents();
}

with

ARM7TDMI::Step() {
 u32 instruction;
 if (ARMMode){ 
 instruction = Mem->Read<u32>(pc - 8);
 if (CheckCondition(instruction >> 28)) {
 (this->*ARMInstructions[ARMHash(instruction)])(instruction);
 }
 }
 else {
 instruction = Mem->Read<u16>(pc - 4);
 (this->*THUMBInstructions[THUMBHash(instruction)])(instruction);
 }
}

A classic interpreter loop. Now we want to add something to store our cache blocks in.

// a simple struct holding the instruction and the handler
struct CachedInstruction {
 const u32 Instruction;
 const void __fastcall (ARM7TDMI::*Pointer)(u32 instruction);
}

// a struct holding a block of instructions
struct InstructionCache {
 const bool ARM;
 const bool Deletable; // true in iWRAM
 const u16 AccessTime;
 std::vector<CachedInstruction> Instructions;
}

And we want some lookup tables for the cache blocks in our CPU.

std::array<std::unique_ptr<InstructionCache>, (Mem::BIOSSize >> 1)> BIOSCache = {};
std::array<std::unique_ptr<InstructionCache>, ((Mem::iWRAMSize - Mem::StackSize) >> 1)> iWRAMCache = {};
std::array<std::unique_ptr<InstructionCache>, (Mem::ROMSize >> 1)> ROMCache = {};

You might notice some things here. First of all, the unique_ptrs. I know that dynamic allocation is slow in general. I rewrote it to not use dynamic allocation, and I did not notice a difference. Main reason for this might be that blocks will never be deleted in ROM and in the BIOS, and it might be rare for blocks to be deleted in iWRAM. Another benefit this has is that I don’t have to worry about writing a bump allocator (a big region of memory that I push instructions to until it overflows, and then reset the entire thing), the unique_ptrs will handle memory allocation for me!

Another thing to note is that I shift the size of the respective regions by 1. This is because all instructions must be aligned by 4 in ARM mode, and 2 in THUMB mode.

Last thing, for the iWRAM cache I subtract Mem::StackSize. This is an arbitrary number (that I think I put at 0x400). I did this, because when writes to iWRAM happen, I have to check out which blocks have to be deleted. Since the stack is also in iWRAM, a lot of pushes will happen, and I don’t want to check which blocks to delete on every push/pop.

One last thing then. We need to keep track of our current cache.

std::unique_ptr<InstructionCache>* CurrentCache = nullptr;

The way this works is very convenient. This might seem like 2 layers of indirection, but while running the cache we can just look at the object that is finally pointed at. In general this is what we will be doing anyway. This is convenient, because we can use nullptr to encode different states:

CurrentCache == nullptr: no cache present, no cache can be made
CurrentCache == nullptr*: no cache present, cache can be made
otherwise: cache present!

We also need a way to update the current cache (simplified to remove some range checks etc.):

 constexpr std::unique_ptr<InstructionCache>* ARM7TDMI::GetCache(const u32 address) {
 switch (static_cast<MemoryRegion>(address >> 24)) {
 case MemoryRegion::BIOS: {
 const u32 index = (address & (Mem::BIOSSize - 1)) >> 1;
 // check if cache exists
 if (BIOSCache[index]) {
 return &BIOSCache[index];
 }

 // show that no cache exists, but can be created
 return &BIOSCache[index];
 }
 case MemoryRegion::iWRAM: {
 const u32 index = (address & (Mem::iWRAMSize - 1)) >> 1;
 if ((address & (Mem::iWRAMSize - 1)) < (Mem::iWRAMSize - Mem::StackSize)) {
 if (iWRAMCache[index]) {
 return &iWRAMCache[index];
 }
 // mark index as filled for faster page clearing
 iWRAMCacheFilled[(address & (Mem::iWRAMSize - 1)) / Mem::InstructionCacheBlockSizeBytes].push_back(index);
 return &iWRAMCache[index];
 }
 return nullptr;
 }
 case MemoryRegion::ROM: {
 const u32 index = (address & (Mem::ROMSize - 1)) >> 1;
 if (ROMCache[index]) {
 return &ROMCache[index];
 }
 return &ROMCache[index];
 }
 default:
 return nullptr;
 }
}

Ignore the iWRAMCacheFilled line for now. Alright, so now we have everything set up to make caches! First of all, we need different run loops:

ARM7TDMI::Run();
ARM7TDMI::RunMakeCache();
ARM7TDMI::RunCache();

Running the cache is the simplest method:

void ARM7TDMI::RunCache() {
 // run created cache
 const InstructionCache& cache = **CurrentCache;

 if (cache.ARM) {
 // ARM mode, we need to check the condition now too
 for (const auto& instr : cache.Instructions) {
 if (unlikely(Scheduler->ShouldDoEvents())) {
 if (unlikely(Scheduler->DoEvents())) {
 return; // CPU state affected
 }
 }

 *timer += cache.AccessTime; // add time we would otherwise spend fetching the opcode from memory
 if (CheckCondition(instr.Instruction >> 28)) {
 (this->*instr.Pointer)(instr.Instruction);
 }
 pc += 4;

 // block was destroyed (very unlikely)
 if (cache.Deletable && unlikely(!CurrentCache)) {
 return;
 }
 }
 }
 else {
 // THUMB mode, no need to check condition
 /*
 * the code here is essentially the same as above
 */
 }
}

Making the block looks very similar to the old loop, but then with a twist. I templated my basic ARM7TDMI::Step() to take a boolean MakeCache. If we are not making a cache, I made the step function return a boolean saying whether we are in a cacheable region or not. That means that Step<false> is the same as the original, basic interpreter step function, except it returns

InCacheRegion(corrected_pc); // corrected_pc = pc - (ARMMode ? 8 : 4)

If we are making a cache, we need to record the instructions we find. Then it looks a bit more complex:

template<bool MakeCache>
bool ARM7TDMI::Step<true>() {
 u32 instruction;

 bool block_end = false;
 if (ARMMode) {
 instruction = Memory->Read<u32, true>(pc - 8);
 auto instr = CachedInstruction(instruction, ARMInstructions[ARMHash(instruction)]);
 (*CurrentCache)->Instructions.push_back(instr);
 
 // check if instruction is branch or an operation with Rd == pc (does not detect multiplies with pc as destination)
 block_end = IsARMBranch[ARMHash(instruction)]
 || ((instruction & 0x0000f000) == 0x0000f000) // any operation with PC as destination
 || ((instruction & 0x0e108000) == 0x08108000); // ldm r, { .. pc }
 if (CheckCondition(instruction >> 28)) {
 (this->*ARMInstructions[ARMHash(instruction)])(instruction);
 }

 pc += 4;
 }
 else {
 // THUMB mode
 instruction = Memory->Read<u16, true>(pc - 4);
 auto instr = CachedInstruction(instruction, THUMBInstructions[THUMBHash((u16)instruction)]);
 (*CurrentCache)->Instructions.push_back(instr);
 
 // also check for hi-reg-ops with PC as destination
 block_end = IsTHUMBBranch[THUMBHash((u16)instruction)] || ((instruction & 0xfc87) == 0x4487);
 (this->*THUMBInstructions[THUMBHash((u16)instruction)])(instruction);
 pc += 2;
 }
 
 if (block_end || !(corrected_pc & (Mem::InstructionCacheBlockSizeBytes - 1))) {
 // branch or block alignment
 return true;
 }
 return false;
}

Basically, if MakeCache is true, Step returns whether the block has ended or not. If MakeCache is false, Step() returns whether a block can be made, or might be available. The RunMakeCache() method will use Step<true>, and the basic Run() method will use Step<false>, while we are not in a region where caches can be made. The RunMakeCache() looks like

void ARM7TDMI::RunMakeCache() {
 // run the interpreter "normally", but record the instructions we execute in the current cache
 while (true) {
 if (unlikely(Scheduler->ShouldDoEvents())) {
 if (unlikely(Scheduler->DoEvents())) {
 // CPU state affected
 // just destroy the cache, we want the caches to be as big as possible
 *CurrentCache = nullptr;
 return;
 }
 }

 if (unlikely(Step<true>())) {
 // cache done
 if (unlikely((*CurrentCache)->Instructions.empty())) {
 // empty caches will hang the emulator
 // they might happen when branches close to pc get written (self modifying code)
 *CurrentCache = nullptr;
 }
 return;
 }

 if (unlikely(!CurrentCache)) {
 // cache destroyed by near write
 return;
 }
 }
}

You’ll notice that I destroy the cache if we get interrupted by the scheduler. This is because I never recreate any caches in ROM and BIOS, so I want them to be as long as possible. Worst case scenario, a lot of events happen in a row, and we might make 10 blocks of 1 instruction each, while we could have easily made 1 block of 10 instructions, and saving a lot more cache block lookup time.

If we end the block because Step returned true, we save the block unless there are no instructions in it. Normally, you’d say this cannot happen. But since the ARM7TDMI has a pipeline that can hold instructions that might be overwritten in memory, it can! I might have started a block with instructions in the pipeline and ended it before it was empty. We don’t want wrong instructions in our cache block, so I cannot add those pipelined instructions, and the block will be invalid.

And then there is the normal running:

while (true) {
 if (unlikely(Scheduler->ShouldDoEvents())) {
 Scheduler->DoEvents();
 }
 
 // if Step returns true, we are in a region where we can generate a cache
 if (Step<false>()) {
 break;
 }
}

Basically the same as the classic interpreter loop. The entire run function becomes this:

ARM7TDMI::Run() {
 while(true) {
 CurrentCache = GetCache(corrected_pc);
 if (unlikely(!CurrentCache)) {
 // nullptr: no cache (not iWRAM / ROM / BIOS)
 // run interpreter normally, without recording cache
 while (true) {
 if (unlikely(Scheduler->ShouldDoEvents())) {
 Scheduler->DoEvents();
 }
 // if Step returns true, we are in a region where we can generate a cache
 if (Step<false>()) {
 break;
 }
 }
 }
 else if (unlikely(!(*CurrentCache))) {
 // possible cache, but none present
 // make new one
 if (ARMMode) {
 const auto access_time = Memory->GetAccessTime<u32>(static_cast<MemoryRegion>(pc >> 24));
 *CurrentCache = std::make_unique<InstructionCache>(access_time, true, static_cast<MemoryRegion>(pc >> 24) == MemoryRegion::iWRAM);
 }
 else {
 const auto access_time = Memory->GetAccessTime<u16>(static_cast<MemoryRegion>(pc >> 24));
 *CurrentCache = std::make_unique<InstructionCache>(access_time, false, static_cast<MemoryRegion>(pc >> 24) == MemoryRegion::iWRAM);
 }
 RunMakeCache();
 }
 else {
 // cache possible and present
 RunCache();
 }
 }
}

That is essentially it for making and running caches. There is one last, important thing we need to account for though: writes to iWRAM. When a write to iWRAM happens, it’s possible that it’s overwriting code that was there before. When we jump to an address, we don’t want to be running code from our caches that is not there anymore! So, when a write to iWRAM happens, we have to clear a section of our caches in iWRAM. We could do this naively, and loop over all 128 entries in the block of the write, check if there is a cache and destroy it. This means a lot of checking! Something else we can do, is whenever we make a cache block, store the index of that cache block to a cache block page table. Then when a write happens to iWRAM, we just have to look up what page this is in (only a shift and a mask), and then destroy the blocks at the indices that are in the page table. This is what the

iWRAMCacheFilled[(address & (Mem::iWRAMSize - 1)) / Mem::InstructionCacheBlockSizeBytes].push_back(index);

line meant in an earlier code block. Writing to iWRAM then has the following callback:

void ARM7TDMI::iWRAMWrite(u32 address) {
 // clear all instruction caches in a CacheBlockSizeBytes sized region
 const u32 cache_page_index = (address & 0x7fff) / Mem::InstructionCacheBlockSizeBytes;

 for (u32 index : iWRAMCacheFilled[cache_page_index]) {
 iWRAMCache[index] = nullptr;
 }
 iWRAMCacheFilled[cache_page_index].clear();

 if ((corrected_pc & ~(Mem::InstructionCacheBlockSizeBytes - 1)) == (address & ~(Mem::InstructionCacheBlockSizeBytes - 1))) {
 // current block destroyed
 if (CurrentCache) {
 *CurrentCache = nullptr;
 CurrentCache = nullptr;
 }
 }
}

As you can see, we look up the cache page (the integer division will be optimized out to a single shift, since Mem::InstructionCacheBlockSizeBytes is 256 (256 bytes per cache block). We then look up in this table which indices are filled in that page. This will usually be at most 1 or 2 blocks, but will probably be 0 blocks in most cases. That’s a lot fewer loops than 128!

Then we also need to check if the current block is overwritten. You might recall that we check this in the new run methods. If the address is in the same page as pc, we destroy the current block, and show that there is no longer a block there by setting CurrentCache = nullptr.

The pros and the cons

I’ve tried it out on a few games, and got between 10-20% performance increase in most games, but some ROMs that run a lot of code from ROM (like AGS) I gained about 50% more frames per second. I said this before, but I also tried rewriting it to not dynamically allocate blocks with unique_ptr, but besides being a lot more annoying in having to deal with a bump allocator, it also didn’t gain me any more speed over dynamic allocation. And on top of that, this way with dynamic allocation uses a minimal amount of memory.

The reason that this dynamic allocation probably doesn’t matter performance wise is because a lot of blocks don’t have to be recreated. In the very large ROM region, blocks will always be the same, and only have to be allocated once. Only in the iWRAM region will blocks have to be recreated and allocated. The big read only code region is a major reason why this might not be the case for other systems. In a lot of systems, the entire memory range is writeable, and you would spend a lot more time allocating and deallocating memory for the blocks.

Bump allocator

I’ve mentioned this term before, and said that I did not use it, but I will briefly explain an approach to using a bump allocator instead. A bump allocator basically means you have a large array holding cached instructions, and every time you make a new block you push instructions to that instead of to an allocated InstructionCache. The caches themselves would look a bit different as well. When I wrote one to see performance differences, I did something like this:

struct InstructionCache {
 const bool ARM;
 u8 Length;
 CachedInstruction* const Pointer;
}

As you can see, the InstructionCache now no longer holds the instructions, but rather a pointer to the instructions in the pre-allocated memory, and a constant of how long the block is. We would no longer have an array of unique_ptr<InstructionCache>, but rather a straight up array of InstructionCaches. Our CurrentCache would be a InstructionCache*, and to check if the cache it points to is empty, we would check if the Length field is 0. You would then have something like a

std::array<CachedInstruction, 0x0100'0000> Cache;
u32 CacheEnd; // points to the first non-filled space in Cache

to hold the instructions. The length of this Cache is a bit arbitrary, and probably something you’ll have to play around with. You want it to be big enough so that you are not constantly invalidating your cache because it’s overflowing, but small enough that your memory usage is not through the roof. InstructionCache would be pointing into this Cache buffer of instructions. When making a new cache, I did:

InstructionCache ARM7TDMI::NewCache() {
 if (CacheEnd >= (Cache.size() - (Mem::InstructionCacheBlockSizeBytes >> 1)) {
 // invalidate entire cache, so go over the array(s) of `InstructionCache`s and set their length to 0
 CacheEnd = 0;
 }
 return InstructionCache(ARMMode, 0, &Cache[CacheEnd]);
}

The initial check is to see if we can fit an entire new cache in the unused space of the Cache array. If this is not the case, we will have to destroy all the cache blocks and just start again. And when adding new instructions, instead of pushing them back on the InstructionCache, I would do

void ARM7TDMI::BumpAlloc(CachedInstruction& instr) {
 Cache[CacheEnd++] = instr;
 CurrentCache->Length++;
}

This method can be very advantagious for systems where the entire address space is writeable, as you would have to invalidate blocks anyway. For a system like the GBA, where you have a lot of read-only memory, I found that it not as good of a fit, as we wouldn’t want to be destroying blocks that will not change anyway. There might be better ways to write a bump allocator, but this is just the idea I had for it, and how I wrote it to see the difference.

I could maybe have taken a hybrid approach, the dynamically allocated blocks for ROM/BIOS and a bump allocator for iWRAM, but it really won’t win much.

All in all, it was not only a quest for more performance, but also a fun learning experience for trying out the idea I had for the cached interpreter. Thank you for reading!

AGS Aging Cartridge Test Decompilation

Mon, 11 Jan 2021 00:00:00 +0000

AGS Aging Cartridge is a test used by Nintendo to test GBA systems. I have used it myself mostly to see how well my emulator performed in it while developing it, but I never knew exactly what the tests did. There wasn’t much documentation on the tests other than what Normmatt had already decompiled, but this led me mostly to the names and entry points (this was especially useful!) of the tests. Therefore, I decided to decompile the tests myself, and put them in a central place to help future emulator developers!

You might recognize it from the following screenshot:

AGS aging cartridge

The code was quite clean, and most of it wasn’t too hard to decompile. The hardest test to decompile was the DMA address control test, since it was so large, it used weird inlined functions it seemed and GHidra kind of messed up the stack on that one.

All tests in the suite return flags for which parts of the test the system failed, but these are never seen on the screen. In my decompilation, besides the actual code for the test, I also documented where in the ROM the test functions start, where they return, and what the return flags mean. I also added a python script to patch the ROM to write the test results to address 0004 (in the BIOS, so it won’t do anything, you can catch these writes and show print them to the console to see what the test returned).

After finishing decompiling the tests that are enabled in the AGS ROM with md5 sum 9f74b2ad1d33e08e8a570ffe4564cbc3, I noticed that there were some tests that were unused. I also wrote a script to patch the same ROM to use the unused tests instead of some of the default tests. Some of them will freeze up a system though (I think), so they might be disabled for this reason.

That patch script can also change it so all tests are disabled, or so that you can always get the “push start to aging” prompt, and the following animation, regardless of whether you pass all tests or not. Usually you would get a prompt saying “error occurred!!” if you failed any and it would freeze up.

I just want to stress that it is not a full decompilation of the ROM, they are only the tests, and it was meant mostly as info for people interested in them. AGS was always kind of a black box to me, but it doesn’t have to be anymore now!

The repository is here, feel free to check it out. All the info on the test functions is in the .h files, the .md files and of course the tests.c files for the tests in each category, corresponding to the respective folder.

GBA Emulator with Hardware Renderer

Sun, 29 Nov 2020 00:00:00 +0000

After having a lot of fun writing a GBA emulator in C# a while ago, I decided it was too slow for my liking. I took the time to rewrite it in C++ with an added challenge: I wanted to do all rendering on the GPU. I do some minimal parsing of video memory (mostly in OAM, but also some IO registers) to optimize GPU usage a bit, but other than that, the PPU is entirely emulated in shaders!

It runs pretty fast (on my system, i7 2600 and GTX 670 I have reached framerates of over 5 to 10k fps in some menus, generally in game it reaches framerates of 800fps). It can frameskip, although that causes some graphical glitches sometimes because I might not be syncing correctly. The frameskip feature is mainly there to save GPU usage (less data transfer). I noticed this more on lower end hardware (on my laptop with intel HD graphics for example).

As for accuracy: it’s fairly accurate, it passes all “sane” AGS tests (so not the prefetch buffer ones, or the ones that require cycle accuracy). It also passes a lot of endrift’s test in her mGBA suite.

I spent a lot of time profiling it, checking out the assembly it generated and optimizing based on that. Some major optimizations I made are:

Templated function handlers
Faking the pipeline (saving a lot of time spent reading memory on branches and such). I “reflush” the pipeline whenever a write close to PC happens.
Faster DMAs
A better scheduler (based on a heap, and it resets its own timer so that I could use a 32 bit int clock instead of a 64 bit one. This might not sound like a big improvement, but it really did save a lot of time)
Pagetables for memory reads (software fastmem)
Compiler intrinsics

And on the graphics side, to save resources (and time spent memcpy’ing to buffer video memory), I did some stuff as well:

Batching up scanlines to whether or not VRAM is “dirty” (writes happened), and only buffering the “dirty” regions of VRAM.
Object batching/palette batching (basically the same thing but for OAM or palette RAM)

To run the emulator, you need an opengl 3.3 compatible graphics card. It should be cross platform, though I haven’t tried that myself, but it works on windows for me.

Anyway, here is the link to the repository, feel free to check it out and let me know what you think!