This section covers the porting of the NetMD Python code to C and integrating it into qhimdtransfer.

UPDATE: As of September, 03 2011, the porting of NetMD to C has been accomplished in the form of an updated libnetmd.

Python NetMD

• command line interface only
• difficult to integrate with QHiMDTransfer (graphical transfer application for HiMD)
• requires Python (version dependency)
• requires some basic knowledge on setting up Python
• not intuitively usable
• code is in some parts highly experimental

The current Python NetMD implementation allows both downloads (all NetMD models) as well digital uploads (MZ-RH1/MZ-RH200 models only) as well as titling, listing and editing of tracks as well as remote controlling NetMD units. Analog capture for upload is supported for models other than MZ-RH1/MZ-RH200. Download code contains a valid but unencumbered root key to allow necessary encryption during transfers. The download code itself is highly experimental, however.

There is already a recent attempt to port the Python code to C. The resulting tarball can be downloaded here: libnetmd. However, it is highly recommend to use the old, much more complete libnetmd (see: http://libnetmd.sourceforge.net/) as a basis and extend it with transfer capabilities.

The current NetMD implementation in Python is divided into several scripts

Utilities:

• lsmd.py - list the contents of any NetMD
• upload.py - transfer tracks from NetMD to the computer (works with MZ-RH1 Walkman only)
• downloadhack.py - allows transfer of tracks from the computer to NetMD (using unencumbered keys)
• dump_md.py - allows analogue transfers of tracks from NetMD to the computer (any NetMD Walkman)
• lsusb.py - sample implementation of lsusb command using usb1.py
• mdctl.py - pdb-based command line to test libnetmd.py methods

Libraries:

• libnetmd.py - implementation of a NetMD library in Python (core module)
• libusb1.py - Ctypes-based python wrapper around libusb1
• usb1.py - object definitions for libusb1.py functions

Since there is already a C library for NetMD available (libnetmd), it would be good idea to use this as a basis for a new implementation instead reinventing the wheel. This old implementation already allows renaming, moving, deleting tracks and so on (see: http://libnetmd.sourceforge.net/) but lacks the capabilities to transfer tracks to the device (in Sony terminology this is referred to as download) and to transfer tracks from the device (Sony calls this upload; upload is supported by the MZ-RH1/200 devices only).

Considering the old libnetmd is used as a basis, the first step would be to import the source code of this library into the git repository of linux-minidisc. Then, since the library has already been without maintenance since 2004, the second step would be compiling the library on a current Linux installation and fix any compiler issues that might arise. After the library has been verified to work in a current environment, it should be tested to work with Sony's MZ-RH1 MiniDisc Walkman (the latest and also last MiniDisc device available) and any problems that might arise should be patched.

After the previous rejuvenation steps, libnetmd is now ready to be extended with download and upload capabilities (note the terms mentioned above), with the help of the PythonNetMD code which serves as sample code for an implementation of both download and upload capabilities (see above summary of the Python scripts available). Since there is with the MZ-RH1 (the MZ-RH200 is a RH1 with different accessoires shipped) essentially only one MiniDisc model available which supports uploads, it's preferred to work on the implementation of the download code first.

While the upload code works unencrypted and thus without the need of any encryption keys, downloads are encrypted and require the data to be encrypted with a key accepted by the NetMD device. Since the keys are proprietary secrets of Sony Corp., we cannot use the original keys for the encryption. Luckily, the keys accepted by the NetMD hardware for an encrypted transfer can be calculated dynamically. In fact, the root key used by the downloadhack.py was encrypted with the help of a secret Sony master key. However, it is not necessary to know the secret Sony key, but it is enough to use the key that is already provided by downloadhack.py. Since we created this key ourselves, we do not have to fear any issues regarding copyright infringement of proprietary code and/or information from Sony Corp.

To summarize the proposed steps for the port:

1. import libnetmd into linux-minidisc, subfolder libnetmd (analogous to libhimd)
2. get libnetmd compile and work on a current release of any Linux distribution (might already work without patching)
3. get libnetmd to work properly with the Sony MZ-RH1 Walkman (see http://libnetmd.sourceforge.net/ for required functionality)
4. with the help of the PythonNetMD code (downloadhack.py), extend libnetmd with download capabilities
5. optional: implement upload capabilities for MZ-RH1 Walkman with the help of upload.py

For documentation, please refer to the the local wiki page netmdlinux with many extremely helpful links for further reading regarding NetMD (includes specifications and patents) and ask any questions on the list https://lists.fu-berlin.de/listinfo/linux-minidisc and/or the IRC channel (#linux-mindisc on FreeNode).

The following section describes the milestones planned for Google Summer of Code 2012 (GSoC 2012). It directly continues where the previous GSoC work left off, albeit there have been some minor bug fixes and improvements on the code which have been committed to the git master repository.

Currently, libnetmd has been integrated in the git master repository and has been updated to support NetMD downloads and USB uploads (supported on the MZ-RH1 only). Downloads are currently supported for LP2 audio files only. The planned changes therefore include the extension of the audio file format support.

As mentioned before, the current version of netmdctl provided as frontend for libnetmd allows for downloads of LP2 files only. The first task will therefore be to extend the code such that also PCM, SP and LP4 audio files are properly recognized. If the input files are provided in a different audio file format, ffmpeg and related utilities can be used to transcode these files prior to transfer. For creating LP2 and LP4 tracks, ATRAC3 encoding is needed at the computer. There is no free ATRAC3 encoder available, so this is only possible with material that is already ATRAC3 encoded. On the other hand, for creating SP tracks, raw PCM material is sent to the NetMD unit, an a transfer utility might decode other media formats on-the-fly.

After libnetmd has gained full functionality, the next step will be integration of the library into the GUI application qhimdtransfer. For this, qhimdtransfer needs to be extended such it will nicely work both with HiMD and NetMD devices. Currently, HiMD devices are supported only via simple file I/O (HiMD act like normal USB mass storage devices).

To summarize the proposed steps:

• redesign the transfer in libnetmd so that it is possible to stream the transfer (currently, the whole file needs to be in (virtual) memory before starting the transfer)
• support other input formats than LP2 encoded WAV in netmdctl
• integrate the upload function of libnetmd into qhimdtransfer
• add download infrastructure to qhimdtransfer
• add support for downloads to NetMD

(a bit apart from the previous list)

• port the analog NetMD copy script to libnetmd