Hardware Calibration

Hardware Calibration

File format #

The file consists out of a 0x200 big header (though the actual header is only 0x30 bytes, the rest is zerofilled), plus data whose size is indicated in the header.

OffsetSizeDescription
0x00x4Magic "CCAL"
0x40x4Version
0x80x4Data size, always 0x7D0
0xC0x1Model version (?)
0xD0x1CAL revision (incremented each time the CAL file is updated)
0xE0x2Bitmask of successful Aging tests
0x100x20Signature of the data section.

HMACSHA256 is used always except in the below cases where SHA256 is used:

 - devunits
 - PARTNER-DEBUGGER
 - PARTNER-CAPTURE
 - the SNAKE counterparts of the above
 - SNAKE-IS-DEBUGGER

0x300x1D0Zerofilled, padding for the 512byte block size

Aging masks #

BitDescription
0FCRAM
1LCD “flicker”/contrast (always successful)
2Camera
3Touch panel (always successful)
4Circle pad (analog stick)
5Codec
6Gyroscope
7RTC
8Accelerometer
9Surround
AAdaptive BackLight (ABL)
B3D screen (ULCD)
CBacklight PWM
DAnalog stick A (???)
ECamera extensions
FAdaptive BackLight (ABL) in legacy (DSi/GBA) mode

Data blocks #

OffsetSizeConfigInfoBlkSince versionDescription
0x000x100x00040000Touch panel
0x140x08???Circle pad
0x202*0x00050000Display panel contrast
0x241*0x00010000RTC
0x281*0x00030000DSPRAM related
0x300x8A???Camera position
0xBC0x120x00040002Gyroscope
0xD00xC0x00040003Accelerometer
0xE00x1340x00020000???Codec
0x2180x060x000500070x10Programmable Infrared Transmitter (PIT)
0x2200x2140x000700000x73D filters
0x4400x200x000500030x8Adaptive BackLight / Power saving mode
0x4700x200x000500050xA???
0x4A00x380x000500020xBBacklight PWM
0x4E00x18???Circle pad extra
0x5000xC??????
0x5100x200x000500040x8???
0x5400x080x001200000x7MCU
0x5500x020x000500060x93D screen (ULCD) delay
0x5600x080x000700020xDMicrophone echo cancellation
0x5700x10C0x000500080xFPower saving mode (ABL) extra
0x6800x080x000400040xFCStick (Right stick)
0x6900x180x001800010x12Quad Tracking Module (QTM)

Data block formats #

Touch #

Used for mapping touch ADC values to display pixel co-ordinates.

[4096, 4096] –> [320, 240]

[RawX, RawY] –> [PointX, PointY]

Usually [PointX0, PointY0] is placed around 25% from the top-left corner, and the same for [PointX1, PointY1] except 25% from the bottom-right corner. This offsetting is needed because the touch film starts to distort outside of that rectangle, which would skew the touch results near the center of the screen.

OffsetDeclaration
0x00s16 RawX0
0x02s16 RawY0
0x04s16 PointX0
0x06s16 PointY0
0x08s16 RawX1
0x0As16 RawY1
0x0Cs16 PointX1
0x0Es16 PointY1

Circle pad #

Contains the centering position of the circle pad. For other circle pad settings, see circle pad extra.

OffsetDeclarationDescription
0x00s16 CenterXRaw analog values corresponding to zero input position
0x02s16 CenterY

Screen flicker #

These values are written to MCU register 0x03 and 0x04 respectively. They both set the display contrast voltage.

OffsetDeclarationDescription
0u8 FlickerTopContrast voltage
1u8 FlickerBottom
2*Inline checksumChecksum low byte, NOT THIS[0]
3*Checksum high byte, THIS[1]

RTC #

OffsetDeclarationDescription
0u8 CompensationValue(???)
1*Checksum byte, NOT THIS[0]

Camera position #

OffsetDeclaration
0x00u32 flags
0x04float scale
0x08float RotationZ
0x0Cfloat TranslationX
0x10float TranslationY
0x14float RotationX
0x18float RotationY
0x1Cfloat ViewAngleRight
0x20float ViewAngleLeft
0x24float ChartDistance(???)
0x28float CameraDistance
0x2Cs16 ImageWidth
0x2Es16 ImageHeight
0x30u8 reserved[0x10]
0x40u8 ???[0x40]
0x80s16 aeBaseTarget(???)
0x82s16 kRL
0x84s16 kGL
0x86s16 kBL
0x88s16 ccmPosition

Gyro #

OffsetDeclaration
0x00s16 ZeroX
0x02s16 PlusX
0x04s16 MinusX
0x06s16 ZeroY
0x08s16 PlusY
0x0As16 MinusY
0x0Cs16 ZeroZ
0x0Es16 PlusZ
0x10s16 MinusZ

Accel #

OffsetDeclaration
0x00s16 OffsetX
0x02s16 ScaleX
0x04s16 OffsetY
0x06s16 ScaleY
0x08s16 OffsetZ
0x0As16 ScaleZ

CDC #

OffsetDeclarationDescription
0x00u8 DriverGainHPHeadphone gain
0x01u8 DriverGainSPSpeaker gain
0x02u8 AnalogVolumeHP
0x03u8 AnalogVolumeSP
0x04s8 ShutterVolume[2]
0x06u8 MicrophoneBiasCapacitive microphone bias voltage
0x07u8 QuickCharge(???)
0x08u8 PGA_GAIN??? (microphone gain)
0x09u8 reserved[3]
0x0Cs16 FilterHP32[3*5]Headphone filter for 32728.49Hz sampling rate
0x2As16 FilterHP47[3*5]Headphone filter for 47605Hz sampling rate
0x48s16 FilterSP32[3*5]Speaker filter for 32728.49Hz sampling rate
0x66s16 FilterSP47[3*5]Speaker filter for 47605Hz sampling rate
0x84s16 FilterMic32[(1+2)+((1+4)*5)]Microphone filter for 32728.49Hz sampling rate
0xBCs16 FilterMic47[(1+2)+((1+4)*5)]Microphone filter for 47605Hz sampling rate
0xF4s16 FilterFree[(1+2)+((1+4)*5)]Unknown
0x12Cu8 AnalogInterval
0x12Du8 AnalogStabilize
0x12Eu8 AnalogPrecharge
0x12Fu8 AnalogSense
0x130u8 AnalogDebounce
0x131u8 Analog_XP_Pullup
0x132u8 YM_Driver??? (circle-pad or touch panel related?)
0x133u8 reserved

PIT #

OffsetDeclaration
0x00u16 VisibleFactor
0x02u16 IRFactor

3D filters #

OffsetDeclaration
0x00u16 SpecialFilter[0x100]
0x200u32 IIRSurroundFilter[5]

ABL #

OffsetDeclaration
0x00u32 DitherPattern
0x04s16 StartX
0x06s16 StartY
0x08u16 SizeX
0x0Au16 SizeY
0x0Cs16 GTH_Ratio
0x0Eu8 DitherMode
0x0Fu8 MinRS
0x10u8 MaxRS
0x11u8 MinGTH
0x12u8 MinMax (???)
0x13u8 ExMax (???)
0x14u8 inertia
0x15u8 LutListRS[9]
0x1Eu8 reserved[2]

BLPWM #

OffsetDeclaration
0x00float coefficient[3][3]
0x24u8 NumLevels
0x25u8 padding
0x26u16 brightnesses[7];
0x34u16 BaseDivisor
0x36u16 MinimumBrightnessHw

Circle pad extra #

OffsetDeclaration
0x00float ScaleX
0x04float ScaleY
0x08s16 MaxX
0x0As16 MinX
0x0Cs16 MaxY
0x0Es16 MinY
0x10s16 type
0x12u8 unknown_padding[6]

MCU #

Somewhat misleading, these values are actually used for clamping the MCU’s raw slider readings to comprehensible values.

OffsetDeclarationDescription
0x00s16 SVR2_MinRaw 3D volume slider values <= this map to 3D slider value 0.0
0x02s16 SVR2_MaxRaw 3D volume slider values >= this map to 3D slider value 1.0
0x04s16 VolumeSliderMinWritten to MCU reg 0x58. Volume slider values <= this map to volume value 0x00
0x06s16 VolumeSliderMaxWritten to MCU reg 0x59. Volume slider values >= this map to volume value 0x3F

ULCD delay #

There is a delay between switching the parallax barrier, and adjusting the backlight. These delay values determine how many VBlank events to wait on before switching the backlight curves to the appropriate mode.

This is needed only to prevent epillepsy from analog jitter causing unwanted mode switches, and both values are usually always set to 1 or 2.

OffsetDeclaration
0x00u8 To2D
0x01u8 To3D

Microphone echo cancel #

OffsetDeclaration
0x00s8 params[8]

ABL extra #

OffsetDeclaration
0x00u8 MaxInertia
0x01u8 pad
0x02u16 PWM_CNT_EX
0x04u32 Histogram1
0x08u32 Histogram2
0x0Cu32 adjust[0x40]

CStick #

OffsetDeclaration
0x00u8 ThinningCountX(???)
0x01u8 ThinningCountY(???)
0x02u16 reserved[3]

QTM #

OffsetDeclaration
0x00float DivisorAtZero (???)
0x04float TranslationX
0x08float TranslationY
0x0Cfloat RotationZ
0x10float HorizontalAngle
0x14float OptimalDistance

Reading #

If 0x1FF81006 is 3 or 4 or 7 or 8 or 9 then the callibration data is read from the EEPROM using the i2c:EEP service command 0x001000C0, using offset 0x000 for HWCAL0, and offset 0x800 for HWCAL1. Otherwise attempt is made to read CTRNAND:/ro/sys/HWCAL(0|1).dat instead.