Emulation Framework (EF): The Emulation Framework is software that enables the automatic recreation of computer environments in a user-friendly way. It does this by using (third-party) emulators which are carefully selected based on their reliability and accuracy of rendering a particular computer system. Although the emulators already exist, they are often difficult to use as their configuration is often complex for laypeople. The Emulation Framework takes away this problem by automatically selecting the best emulator for rendering a certain file or application. After that, it automatically prepares the emulator with the optimal settings and required software (operating system, applications, plug-ins, drivers, etc.). This happens in the background; the user only deals with a user-friendly interface for interacting with the original computer environment.
Transfer Tool Framework (TTF): A framework that facilitates the secure and accurate transfer of the original bit-stream of a wide range of digital media carrier. Its main functionality is to allow a user to retrieve the information from a digital media carrier and to create an image of it in such a way that a digital object will be created which fits in the landscape of the Emulation Access Platform. The TTF consists of a front-end GUI and back-end Core Engine which offer interfaces to support required transfer tools amongst others.
Questions and Answers
1) What is the purpose of KEEP?
KEEP is building a better and more stable base for the preservation of, and access to, our digital stored heritage. The KEEP approach is twofold: one part is the implementation of a virtual machine, whose purpose is to make existing and future emulators portable on future computer systems, without altering them (for further details please see question 5); secondly, an Emulation Framework will be developed, into which existing or future emulators can be plugged to enable them to be used in a wider context (for further details please see question 8). Both tasks will help address the interests and needs of emulator programmers and the growing demands of our society for a more systematic suite of effective preservation tools. In that respect KEEP will broaden the user base for emulators and make them more compatible for the institutional sector (see also question 16). To complete the picture KEEP is researching the technical and legal aspects of transferring digital artefacts from their original media carriers. A Transfer Tool Framework (TTF) will be designed and implemented as a prototype, which aims to enable a more or less automated transfer of digital artefacts.
2) Is KEEP creating yet another emulator?
No. KEEP is creating an emulator framework which provides additional services which will help to build a more solid ground for the emulation preservation strategy. KEEP is depending on existing and future emulators, and will not create an emulator itself. The KEEP partners are aware, that existing emulators play a crucial role in preserving digital objects. Instead of inventing the wheel twice KEEP offers the emulator programmers additional services, at the same time they of course will stay independent projects.
3) What is wrong with existing emulators?
Nothing. It is quite the opposite in fact: a great deal has been achieved mainly due to the initiative of many individuals and groups, who have their roots in the gamer community. On the other hand it is just the plurality of emulators with their different functionality, which makes it difficult to use them in a wider context. KEEP is just trying to add further value to existing emulators by easing their use and make them compatible with institutional needs.
4) Is KEEP creating yet another transfer tool?
No. The Transfer Tool Framework (TTF) delivers the basic architecture to build an environment in which various available and existing transfer tools can be executed and tested rather than being a Transfer Tool. If you try to imagine a mechanic in a car service working at a workbench offering several tools to him to do a specific car repair job then the TTF would not be the specific tool it would be the workbench offering various tools for a specific job giving more flexibility to the mechanic. The value which the TTF adds to existing transfer tools is its flexibility in providing a defined environment which can be thought of kind of a workbench or configurable tool chain which enables user to utilize various tools which are responsible for several tasks within the transfer process to securely and accurate transfer carrier images of a wide range of digital media carrier types.
5) Why is it important to use emulators for digital preservation?
Digital publications are stored on various types of carriers that are rapidly getting old. Moreover, machines used to read or run these publications are also getting old very fast and cannot be maintained in a working state in the medium or long term. This is the reason why National libraries need to consider carrier transfer, or carrier imaging, and hardware virtualization, or emulation, as the only durable way to preserve digital publications and the possibility to access, that is run, them. Emulation also provides very interesting advantages compared to migration, which is the preferred solution so far for static (i.e. text, audio, pictures…) electronic publications:
- digital artefacts are not altered: transfer/imaging them aims at storing all meaningful data contained on the original carrier, including physical characteristics such as carrier geometry, and so on, or anticopy mechanisms.
- emulators are cost efficient, because they dispense with the need for regular migration: each carrier has to be transferred to an image file only once. Future adaptations will only concern the emulator that has to be adapted to a future hardware and operating system.
- complex digital artefacts need their original environment: emulation allows to recreate this very environment and store it.
6) How will the Emulation Framework work?
The Emulation Framework can be seen as a largely automated workflow for emulation. This workflow can work on top of existing archiving solutions that contain digital collections (e.g. publications at libraries, scientific data sets in research, computer games, archival records, documents in organisations). When a user requests a digital file or application, the Emulation Framework can be used to recreate the original computer environment. For example, an old Wordperfect document can be shown via Wordperfect 5.1 on MS-DOS 5.0, or an old MS Access data base can be rendered via MS Access 95 running on Windows NT. If the user prefers to have this file or application rendered in its original computer environment, the Emulation Framework is started. From then on, the Emulation Framework analyses the file or application using external registries. After identification, the framework selects the best suitable emulator to mimic the original hardware and automatically prepares all required software (operating system, plug-ins, drivers, etc.). After that, the framework starts a new emulation session and offers the user an interface consisting of a virtual screen, keyboard, mouse and some additional services such as copy/paste text or creating a screenshot.
7) Will the Emulation Framework include old consoles such as Nintendo, SNES, N64, Neo Geo, Playstation, Gamecube, Sega, Atari, [insert your favorite console here], etc.?)
The Emulation Framework's goal is to include every existing emulator, including the above video game consoles. As with all emulators, there is the question of legality: KEEP will only have the capacity to integrate legal emulators. In summary, although this is technically possible, the present legal framework in Europe prevents it.
8) Will the Emulation Framework be easy to set up?
Setting up the Emulation Framework within your own business requires some IT skills. The framework will be available as a downloadable package that can be deployed on a dedicated computer. The framework will also supply the necessary documentation for installation and configuration. However, no service support will be available as the Emulation Framework is a prototype (i.e. alpha version) of this project. It is not (yet) an official product.
9) How can existing and future emulators be integrated into the KEEP system (VM and Emulation Framework)?
There will be a straight forward and well documented workflow to add emulators. This workflow will include steps to analyse the necessary modifications and configurations. The workflow could be conceived and performed by the community of KEEP users or by individual institutions based on their special needs.
10) Will it allow for plug-in controller play (Logitech Gamepads, etc)?
Controller play is a separate issue and one that will probably be looked at in the second phase of the project (remembering that KEEP is at root a virtual machine that can emulate O/S's, so it can be layered - so technically, you could emulated Windows XP with Logitech running as an application, and all that running an XBox emulator on top, etc, etc) - the next three years establishes KEEP as a concept and tackles the initial technical issues. It's not going to be complete in that period!
11) Will the software of the EF and the emulators be freely available?
The Emulation Framework will become available as open source software and is therefore freely downloadable. However, as the framework uses existing emulators, it depends on the license model of each particular emulator if it can be distributed together with the framework or not.
12) I am a transfer tool developer, how can I contribute?
The current strategy is that the TTF shall offer defined interfaces to take care on all aspects of a secure' and accurate transfer of carrier images. These interfaces and the strategy how they shall be able to be implemented to integrate tools such as Transfer Tools and others required to deal with all aspects are still under development.
One goal of the TTF design is to offer a modular and flexible framework which enables to be customized via a front-end for each of the modules the system has implemented. The TTF shall allow developers of an implementation for the TTF to decide thereself which modules (except for the mandatory ones) they would like to implement for there Transfer Platform based on the TTF. For instance a system configuration file could contain available modules which will be parsed by the TTF at startup. Anyway the TTF shall support a smart, lightweight and flexible configuration approach. To arm the TTF with powerful transfer tools for various digital media carrier the TTF depends on Transfer Tool Developers and the community to provide Transfer Tools in such a way that they will be able to be integrated within the TTF. The current integration strategy of the TTF is that Transfer Tools (and others too) shall be packed into a container format (e.g. .zip, .jar, etc.) including:
- the executable of the tool
- a descriptive metadata file (in the defined format as expected by the TTFs interface).
Whereas the metadata file will contain descriptive data which the TTFs interface requires to know about this specific tool to utilize it. For a Transfer Tool this could be for instance Name, Version, the way it may be able to communicate with the TTF, how it can be controlled, e.g. its supported Digital Media Carrier Types & Image File Formats, its supported handling of Copy-Protection Mechanism, , etc. to utilize it via the Transfer Tool Interface of the TTF. However, this strategy might change during development of the TTF therefore no responsibility can be taken for this information.
13) Where can I download the Emulation Framework and the KEEP VM?
During the project timeline (Feb. 2009 to Feb. 2012), the public deliverables are available in the download area of the official KEEP website. During the first year the project concentrated on state of the art analysis of current emulation preservation activities. The main result of this investigation is reports, which are currently available to download and read. Afterwards, the official website shall be redesigned to provide an easy access to all relevant outcomes produced during the project. But please be aware, that KEEP is a research project and it can not be expected that the software, which will be available at the end of KEEP will be more than a working prototype.
14) Who are the intended users of KEEP?
The short answer is that KEEP is aiming to reach as many users as possible. In particular we can identify two main groups. Firstly the KEEP partners are aware that the achievements of the emulation community are based mainly on private initiatives and driven by passion for the most part. To build on that momentum we are designing the KEEP system in as broad a way as possible so that individual users (particularly from the computer games / emulation communities) can benefit from it. The second user group comprises memory institutions with their growing demand for systematic usage of existing and future emulators to preserve our cultural heritage and keep it accessible for future generation.
15) How can I get the original software needed for emulation?
The independent user will need to use original copies of the software. These will then need to be converted to a disk image format usable by the appropriate emulator. KEEP will provide documentation pertaining to transfer tools capable of performing this task. The issues are essentially the same for institutions, who will need to have their own original copies of software etc.
16) I have this old game for an X console, can KEEP run it for me?
The same issues arise as in Q20: the user would need to create a disk image and KEEP would need to have an appropriate emulator. The first version of KEEP will include emulators for the C64 and the PC, but it is hoped that more will be added quickly.
17) What is the legal situation?
The emulation topic has various legal implications. The main areas are the transferring of the code, the recreating of the original hardware in software (which is actually the emulator) and the necessary software environments like obsolete operating systems and drivers. It is unfortunate that national copyright laws differ in that aspect, so it is not easy to provide a short answer here. Therefore KEEP has commissioned a legal study, in which these aspects are researched under legal terms in the legal framework of France, Germany and The Netherlands.