himddiskformat
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
himddiskformat [2009/04/28 22:07] – megadiscman | himddiskformat [2011/09/04 12:43] – fix end-of-code tag. megadiscman | ||
---|---|---|---|
Line 138: | Line 138: | ||
==== The file mclist0X.hma ==== | ==== The file mclist0X.hma ==== | ||
- | The filename " | + | The filename " |
+ | Furthermore, | ||
+ | The final authentication is done by combining checksums of the medium-specific and the track-specific parts into a master " | ||
- | The contents of " | + | The contents of " |
< | < | ||
Line 152: | Line 154: | ||
</ | </ | ||
- | It contains information | + | The MAC list file is divided into three parts. The first 16 bytes are the file header, the next 80 bytes are for disc authentication and the remaining part is for track authentication. Both authentication parts start with an encrypted 3DES key used for authentication, |
+ | |||
+ | < | ||
+ | 0000 BYTES magic signature " | ||
+ | 0004 DWORD unknown purpose | ||
+ | 0008 BYTES 8 unknown bytes, always zero, maybe padding. | ||
+ | |||
+ | 0010 BYTES 16 bytes encrypted 3DES key for authenticating | ||
+ | 0020 DWORD Generation number of the DRM info | ||
+ | 0024 BYTES 20 unknown bytes, always zero, maybe padding | ||
+ | 0038 DWORD | ||
+ | 003C DWORD 4 unknown bytes, always zero, maybe padding | ||
+ | 0040 BYTES 16 bytes disc ID. This is a copy of the real disc ID stored outside of the file system | ||
+ | 0050 BYTES 16 unknown bytes, always zero, maybe padding | ||
+ | |||
+ | 0060 BYTES 16 bytes encrypted 3DES key for authenticating | ||
+ | 0070 BYTES 4000 * 8 bytes MAC values of tracks. Intersting count, as Hi-MD only has 2048 tracks. | ||
+ | </ | ||
- | The fields at offset 10h and 60h are presumably encryption keys for 3DES. It's not yet confirmed, whether those are the keys in cleartext or those are the keys in an encrypted form themselves. | ||
==== The file 00010012.hma ==== | ==== The file 00010012.hma ==== | ||
- | This file has never been observed to change. It's md5sum always remains | + | This file contains |
- | ==== The Audio-Data File ==== | + | ==== The Audio Data File ==== |
+ | |||
+ | The audio data file contains the raw audio data. | ||
+ | |||
+ | It is divided into 16k blocks of the following layout | ||
+ | |||
+ | < | ||
+ | 0000 DWORD Block Type (" | ||
+ | 0004 WORD | ||
+ | 0006 WORD " | ||
+ | 0008 WORD Data size in bytes (MP3 blocks only) | ||
+ | 000A WORD | ||
+ | 000C DWORD Serial number of block in stream | ||
+ | 0010 BYTES PCM/Atrac: Encrypted DES key for the audio data | ||
+ | 0018 BYTES PCM/Atrac: DES CBC initialization vector for the audio data | ||
+ | |||
+ | 0020 BYTES up to 3FC0 of encrypted audio data (ATRAC never uses all 3FC0 bytes) | ||
+ | |||
+ | 3FE0 BYTES Backup of encrypted key (offset 0010) | ||
+ | 3FE8 BYTES 8 bytes reserved | ||
+ | 3FF0 DWORD backup of Block Type | ||
+ | 3FF4 WORD | ||
+ | 3FF6 WORD | ||
+ | 3FF8 DWORD low order 32 bits of the Content ID | ||
+ | 3FFC DWORD backup of serial number | ||
+ | </ | ||
- | The audio-data file contains | + | In case of MP3 audio, the data is XOR encrypted. The key for encryption is created from the DiscID which read/ |
+ | Each audio block contains only complete frames. The space in the audio block after the last complete frame is unused - which is especially wasteful for 352kbit/s ATRAC3+, as the frame size is exactly 2K, so 1984 bytes per frame are ignored, i.e. 12% of the block size. | ||
==== The _MDHIFI.HMA File ==== | ==== The _MDHIFI.HMA File ==== | ||
Line 246: | Line 290: | ||
Contains 2048 track descriptors of the following layout | Contains 2048 track descriptors of the following layout | ||
< | < | ||
- | 0000 DWORD date of recording (FAT format, 16 bit date, 16 bit time of day) | + | 0000 DWORD - date of recording (FAT format, 16 bit date, 16 bit time of day) |
- | 0004 DWORD " | + | 0004 DWORD + " |
- | 0008 WORD Title (string number) | + | 0008 WORD |
- | 000A WORD Artist (string number) | + | 000A WORD |
- | 000C WORD Album (string number) | + | 000C WORD |
- | 000E BYTE Track number (within Album, not on MD) | + | 000E BYTE |
- | 000F BYTE " | + | 000F BYTE |
- | 0010 8 BYTES MgrCK (upload requirement: | + | 0010 8 BYTES + MgrCK (upload requirement: |
- | 0018 8 BYTES CMac | + | 0018 8 BYTES |
- | 0020 BYTE CodecId (see below) | + | 0020 BYTE |
- | 0021 3 BYTES Codec specific info | + | 0021 3 BYTES ? Codec specific info |
- | 0024 WORD Part Number (index into Parts Info Table) | + | 0024 WORD |
- | 0026 WORD Track Number | + | 0026 WORD |
- | 0028 WORD Total time (units of seconds) | + | 0028 WORD |
- | 002A BYTE " | + | 002A BYTE |
- | 002B BYTE " | + | 002B BYTE |
- | 002C WORD More codec specific info | + | 002C WORD |
- | 002E WORD reserved | + | 002E WORD |
- | 0030 20 BYTE Content ID (for Mac Transferred Data: 02 03 00 00 + 16 random bytes) | + | 0030 20 BYTE + Content ID (for Mac Transferred Data: 02 03 00 00 + 16 random bytes) |
- | 0044 DWORD Start time (FAT format) | + | 0044 DWORD + Start time (FAT format) |
- | 0048 DWORD End time (FAT format) | + | 0048 DWORD + End time (FAT format) |
- | 004C BYTE " | + | 004C BYTE |
- | 004D BYTE " | + | 004D BYTE |
- | 004E BYTE " | + | 004E BYTE |
- | 004F BYTE " | + | 004F BYTE |
</ | </ | ||
+ | |||
+ | The +/-/? means: " | ||
" | " | ||
Line 281: | Line 327: | ||
== Codec specific info for ATRAC3 == | == Codec specific info for ATRAC3 == | ||
- | * 0023 Bit rate in bits 0..5 | + | * 0021 Bit 1: Set for joint stereo (LP4) |
- | ** 18: LP4 (66 kbit/s) | + | * 0022 Bit 7-5: Sample |
- | ** 26: 105 kbit/s | + | * 00: 32kHz |
- | ** 30: LP2 (132 kbit/s) | + | * 20: 44.1kHz |
+ | * 40: 48kHz | ||
+ | * 60: 88.2kHz | ||
+ | * 80: 96kHz | ||
+ | * 0023 Bit 5-0: Frame size in 8 bytes units | ||
+ | | ||
+ | * 26: (Frame size 304 bytes) | ||
+ | * 30: (Frame size 384 bytes) | ||
== Codec specific info for ATRAC3+ == | == Codec specific info for ATRAC3+ == | ||
- | * 0023 Bit rate | + | * 0022 Bit 7-5: Sample |
- | ** 17: 32 kbit/s | + | * 00: 32kHz |
- | ** 22: 48 kbit/s | + | * 20: 44.1kHz |
- | ** 2E: Hi-LP (64 kbit/s) | + | * 40: 48kHz |
- | ** 8B: 192 kbit/s | + | * 60: 88.2kHz |
- | ** B9: Hi-SP (256 kbit/s) | + | * 80: 96kHz |
- | ** FF: 352 kbit/s | + | * 0022 Bit 4-2: Number of Channels |
+ | * 04: Mono | ||
+ | * 08: Stereo | ||
+ | * 0023 Frame size in 8 bytes units (-1) | ||
+ | | ||
+ | * 22: (Frame size 280 bytes) | ||
+ | * 2E: (Frame size 376 bytes) | ||
+ | * 8B: (Frame size 1120 bytes) | ||
+ | * B9: (Frame size 1488 bytes) | ||
+ | * FF: (Frame size 2048 bytes) | ||
== Codec specific info for MPEG == | == Codec specific info for MPEG == | ||
Line 299: | Line 361: | ||
* 0022 unused, zero | * 0022 unused, zero | ||
* 0023 bit field | * 0023 bit field | ||
- | ** 80: Always set | + | |
- | ** 40: Variable MPEG version | + | * 40: Variable MPEG version |
- | ** 20: Variable MPEG Layer | + | * 20: Variable MPEG Layer |
- | ** 10: Variable bitrate | + | * 10: Variable bitrate |
- | ** 08: Variable sample rate | + | * 08: Variable sample rate |
- | ** 04: Variable channel mode (Joint stereo/ | + | * 04: Variable channel mode (Joint stereo/ |
- | ** 02: Variable preemphasis | + | * 02: Variable preemphasis |
- | ** 01: Always cleared | + | * 01: Always cleared |
* 002C: Bit field: | * 002C: Bit field: | ||
- | ** C0: MPEG version (C0 = MPEG1, 80 = MPEG2, 00 = MPEG2.5, as in MPEG spec) | + | |
- | ** 30: MPEG layer (encoded as in MPEG stream) | + | * 30: MPEG layer (encoded as in MPEG stream) |
- | ** 0F: Bitrate number (see MPEG spec) | + | * 0F: Bitrate number (see MPEG spec) |
* 002D: Bit field: | * 002D: Bit field: | ||
- | ** C0: Sample rate from header (see MPEG spec) | + | |
- | ** 30: Channel mode (00 = split stereo; 10 = joint stereo; 20 = 2*mono; 30 = mono) | + | * 30: Channel mode (00 = split stereo; 10 = joint stereo; 20 = 2*mono; 30 = mono) |
- | ** 0C: Preemphasis (see MPEG spec) | + | * 0C: Preemphasis (see MPEG spec) |
- | ** 02, 01: always cleared. | + | * 02, 01: always cleared. |
=== The Part Info Table === | === The Part Info Table === | ||
Line 516: | Line 578: | ||
See also [[http:// | See also [[http:// | ||
+ | |||
+ | ===== Fragmentation ===== | ||
+ | |||
+ | In order to implement download, we have to understand what happens when tracks are removed inbetween, e.g. have 3 tracks and delete #2. This creates a hole in the data-container. | ||
+ | |||
+ | The following screenshot illustrates a test with SonicStage and himdtest: | ||
+ | |||
+ | {{: | ||
+ | |||
+ | - download 3 MP3s onto HiMD, show tracks with himdtest | ||
+ | - delete MP3 #2, show tracks with himdtest | ||
+ | - add another, larger MP3, show tracks with himdtest | ||
+ | - download 3 MP3s onto HiMD, show tracks with himdtest | ||
+ | - delete MP3 #2, show tracks with himdtest | ||
+ | - add another, smaller MP3, show tracks with himdtest | ||
+ | |||
+ | ==== Fragmentation of the FAT-Filesystem ==== | ||
+ | |||
+ | After deleting track Nr. 2: | ||
+ | |||
+ | {{: | ||
+ | |||
+ | After downloading a new track after deletion of track 2: | ||
+ | |||
+ | {{: | ||
+ | |||
+ | During defrag: | ||
+ | |||
+ | {{: | ||
+ | |||
+ | ===== Encryption ===== | ||
+ | |||
+ | ==== General picture ==== | ||
+ | |||
+ | {{: | ||
+ | |||
+ | Red boxes indicate data stored on HiMD, Black boxes indicate "black boxes" - Operations we don't know how they work. For so called " | ||
+ | |||
+ | ==== Fragment keys ==== | ||
+ | |||
+ | Joining two tracks created on an RH10 (so not using zero-key-encryption) with an RH1 walkman (using zero-key-encryption) creates a result of two fragments employing fragment keys. Here are himdtest dumps before/ | ||
+ | |||
+ | === Before join === | ||
+ | < | ||
+ | 46: 0:07 AT3+ Unknown artist: | ||
+ | | ||
+ | | ||
+ | Key: 167960f39f024906 (EKB 00010012); MAC: 06f503a38982762b | ||
+ | 47: 0:07 AT3+ Unknown artist: | ||
+ | | ||
+ | | ||
+ | Key: 5966e351a690c9f4 (EKB 00010012); MAC: f3c0f999f13d48e0 | ||
+ | </ | ||
+ | |||
+ | === After join === | ||
+ | < | ||
+ | 47: 0:13 AT3+ Unknown artist: | ||
+ | | ||
+ | | ||
+ | | ||
+ | Key: 0000000000000000 (EKB 00010012); MAC: 448b34515a6df04d | ||
+ | </ | ||
===== Links ===== | ===== Links ===== | ||
Line 523: | Line 647: | ||
* [[http:// | * [[http:// | ||
* [[http:// | * [[http:// | ||
+ | * [[http:// |
himddiskformat.txt · Last modified: 2012/01/05 22:55 by megadiscman