====== ExtendingLibHiMD ====== This section covers the extension of the //libhimd// library and integrating these features into [[qhimdtransfer]]. ===== Background and motivation ===== //libhimd// is the part of linux-minidisc which abstracts access to HiMD hardware. Tasks include: * reading and decrypting audio data tracks (MP3/PCM/ATRAC3/ATRAC3+) from HiMD medium * writing and encrypting audio data tracks (MP3/PCM/ATRAC3/ATRAC3+) to HiMD medium * deleting, renaming, rearranging and grouping audio data tracks * sending special SCSI commands over USB to control and inquire the HiMD hardware (format, erase all, read discid etc) ===== Current status ===== Currently, //libhimd// implements only a limited set of the planned features it should have. Reading and decrypting audio tracks has been fully implemented. The official git repository source code supports reading and decrypting of MP3 audio data and PCM/ATRAC3/ATRAC3+ tracks written with a known constant encryption keys which have been recorded with the HiMD models **Sony MZ-RH1/MZ-M100/MZ-M200** (see: [[https://wiki.physik.fu-berlin.de/linux-minidisc/doku.php#status]]). With an additional secret root key not included in the official source code, PCM/ATRAC3/ATRAC3+ tracks written with different encryption keys can be read. The read support is therefore considered to be feature complete. As for writing to HiMD, initial support has been implemented with a large patch set in March 2011 (see: [[https://lists.fu-berlin.de/pipermail/linux-minidisc/2011-March/msg00002.html]]). This code implements solely writing MP3s as these use a rather simple encryption (the audio data is basically XOR'ed) and still very experimental, but it has been finally shown to work (see: [[https://lists.fu-berlin.de/pipermail/linux-minidisc/2012-January/msg00005.html]]). It currently supports MP3 tracks with 128kbps/44.1kHz only. Writing the track names is not properly working either. Support for writing PCM/ATRAC3/ATRAC3+ is currently missing completely as it requires additional support for 3DES encryption and generating valid MAC (mandatory access control) list entries. However, it is possible to add support for writing PCM/ATRAC3/ATRAC3+ on the MZ-RH1/MZ-M100/MZ-M200 models without the need of writing valid MAC list entries and by using the aforementioned constant encryption key. The main part of this project task would therefore be extending the write support and polishing it. Also, write support has not been integrated into the GUI, [[qhimdtransfer]], yet. Deleting, renaming, rearranging and grouping audio data tracks is currently not supported at all, but at least the capability of renaming a track has been implemented through the patch which added MP3 write support. Due to some remaining bugs, tracks names are not properly written to the track index file on the HiMD, however. Sending SCSI commands has not been implemented into //libhimd// yet. However, there is a fully working example code in the git repository in the subfolder //basictools// which implements the supported SCSI command set through //libscg//, the Linux SCSI layer and the SCSI abstraction layer which is part of the //dvd+rw-tools// (see: [[https://wiki.physik.fu-berlin.de/linux-minidisc/doku.php?id=scsisupport]]). The latter will be the **preferred implementation** as it works out of the box on the most important operating systems and does not require any additional dependencies. ===== Proposed project plan ===== Stemming from the aforementioned current state of //libhimd//, the following tasks arise as the next steps in development: * finalize and polish MP3 write support * add write support for PCM/ATRAC3/ATRAC3+ files (MZ-RH1 with //constant// key only) * implement and fix existing for functions for renaming and deleting tracks * integrate SCSI code proposed in [[https://lists.fu-berlin.de/pipermail/linux-minidisc/2010-December/msg00000.html]] into //libhimd//, (see [[http://users.physik.fu-berlin.de/~glaubitz/linux-minidisc/himdscsitest_dvd-tools.c]] for example source code using //transport.hxx//) ===== Detailed description of the tasks ===== Finally, the four proposed project tasks are to be explained in more detail here. ==== Finalizing MP3 write support ==== As mentioned before, write support for MP3s has largely been implemented already. However, the code is still buggy and also supports MP3s with //128kHz/44kbps// only. The task is therefore to properly parse MP3 input files with //libmad// and then set the parameters in the tracklist file accordingly for each MP3 frame so that the Walkman is able to decode and play back the MP3 files. The code should also check whether the input files are in a valid MP3 format which the Walkman can work with. ==== Adding prelimenary support for writing PCM/ATRAC3/ATRAC3+ ==== While MP3 audio data is encrypted with a relatively simple XOR compression and neither uses any MAC authentication, implementation is rather simple, writing PCM/ATRAC3/ATRAC3+ is rather difficult. The encryption employed is 3DES. ==== Fix renaming and deleting tracks ==== See [[https://wiki.physik.fu-berlin.de/linux-minidisc/doku.php?id=himddiskformat#the_track-index_file]]. ==== Integrate SCSI command code ==== The background of this task is that HiMD devices interpret SCSI commands through USB in order to perform tasks like formatting a medium, erasing all contents and certain DRM commands (see: [[https://wiki.physik.fu-berlin.de/linux-minidisc/doku.php?id=scsidrminfo]]). The fundamental part of this task is integrating the SCSI code using transport.hxx from the //dvd+rw-tools// into libhimd (see: [[http://users.physik.fu-berlin.de/~glaubitz/linux-minidisc/himdscsitest_dvd-tools.c]]) and providing a nice API which can be used from the application land to issue the various SCSI commands. A generic function to do SCSI I/O is enough, applications can send the specific SCSI commands themselves. It might useful to implement another abstraction layer providing the basic functions for //format//, //erase all// etc, however.