How the Inspector interacts with the Maxine VM

This page describes how the Maxine Inspector’s interaction with a running VM is implemented.

General goals for the Inspector all deal with making development and experimentation in the Maxine VM as productive and widely accessible as possible:

  • Support basic debugging of the Maxine VM, something not possible with standard tools.
  • Make visible as much internal VM state as possible, both in terms of design abstractions and in terms of low-level representations, even when the VM is broken.
  • Provide new developers with a fast path to understanding VM internals.

A few general strategies guide the Inspector’s implementation:

  • Run in a separate process (usually local, but see Guest VM), so that inspection depends neither on a correctly running VM process nor there being any process that all.
  • Require as little active support from the VM as possible, in particular require no active agent.
  • Reuse as much VM code as possible, especially reading/writing/understanding the low-level representations of data on the target platform (possibly different than the platform on which the Inspector runs).
  • Load VM classes into the Inspector for reflective use in understanding VM data.
  • Rely on platform-specific implementations for low-level interaction with a running VM: process control, threads, breakpoints, access to memory and registers.

Low-level VM Interaction

This section describes the Inspector’s access to the lowest level abstractions in the VM process, namely the resources provided by the underlying platform: memory, threads, and execution state.

Process control

One of the most difficult and frustrating parts of the Inspector’s implementation is the need to implement low-level process controls on the several supported platforms. These controls include reading and writing from the VM’s memory, finding and inspecting threads, setting breakpoints, setting watchpoints, and deciphering process state.

Generic controls are implemented in class com.sun.max.tele.debug.TeleProcess. Concrete subclasses using native methods implement the controls for specific platforms:

  • Solaris: platform support is best on Solaris, where libproc provides a programmatic interface to the Solaris /proc pseudo-filesystem. Watchpoints are supported with no limit on their number (see Native/tele/darwin/*.[ch]).
  • Linux: the Inspector uses a mixture of ptrace(2) and /proc (see Native/tele/linux/*.[ch]).
  • Mac OS X: on the Mac the Inspector uses a mixture of ptrace(2) and the Mach API (see Native/tele/darwin/*.[ch]).
  • Guest VM: the Guest VM variant of the VM runs in a Xen domain where such OS services are unavailable, so controls must be implemented using Xen inter-domain communication.

This code can be very subtle. It now seems to work fairly reliably, but at the cost of many hours deciphering non-documentation and gdb source code. In our experience, programming a debugger is a very niche activity.

Reading and writing VM memory

Low-level memory access is implemented using basic process control methods in class TeleProcess:

read(Address address, BtyeBuffer buffer, int offset, int length)
write(Address address, ByteBuffer buffer, int offset, int length)

However, interpreting the bits presents more of a challenge, since this must be done for a VM running on a potentially different platform. Fortunately, the Inspector is able to load the Java classes that describe the target platform and then reuse the VM’s own code for reading and writing bits representing the VM’s internal primitive data types. Methods for reading and writing those types appears in interface, and all but the lowest-level read methods are implemented by class

For performance reasons, especially for non-local debugging such as with the Guest VM, the Inspector caches pages of memory read since the most recent process execution (see class


The Inspector’s low-level interaction with the VM process can be observed. See Low-level logging for instructions on enabling all low-level VM logging. In order to observe only Inspector-related events, change log_TELE to 1 in Native/share/log.h, rather than log-ALL.

Passive VM support

Although the Inspector is designed to rely as little as possible on the internals of the VM, there are a number of ways in which the VM is constructed to make inspection as easy as possible. The mechanisms described in this section incur zero runtime overhead in the VM, and involve no writing into VM memory.

Locating critical VM resources

The Inspector leverages considerable knowledge of the VM’s internal data representations to build its model of VM state, but it must have somewhere to start when beginning to read from a memory image. The boot image generator stores in the boot image header a number of addresses and other data that help the Inspector (and VM) find things. These addresses get relocated, along with the contents of the heap, during Bootstrap. The Inspector leverages detailed knowledge of the header’s contents in order to locate, among others:

  • the VM’s schemes bindings, which are loaded into the Inspector
  • the boot heap
  • the boot code region
  • the class registry
  • the list of dynamically allocated heap segments
  • the list of thread local areas
  • the entry location of key methods

Field access

The Inspector uses a variety of mechanisms to locate instance or class fields in the heap. During the Inspector’s starting sequence (when little is yet known about VM state), fields are typically located by relying on specific knowledge of a few key object types, possibly using Java reflection on the VM classes (which are all loaded into the Inspector). This kind of access is relatively unsafe, since it bypasses the type system in the running VM. There are more abstract ways to access fields, but they rely on the Inspector’s model of VM’s class registry, which must first be created using the low-level mechanisms.

The simplest way to exploit higher-level field access mechanisms is to annotate (in VM code) fields of interest using @INSPECTED. The main method in com.sun.max.tele.field.TeleFields reads VM sources, generates field access methods, and writes them back into itself for use by the Inspector. These access method implementations hide all the indirections necessary to read or write field data (taking into account the hardware platform, the layout being used, the particular representation for the object, and the class layout) and return values of the desired types.

Method access

The Inspector uses a variety of mechanisms to locate methods and their compilations (either instance or class). Specific methods can be called out for enhanced access by the Inspector by annotating (in VM code) those methods using @INSPECTED. The offline program TeleMethods reads VM sources, generates method access methods, and writes them into class com.sun.max.tele.method.TeleMethods. These access method implementations hide all the indirection necessary to locate the annotated methods and their meta-information.

Method interpretation

VM methods annotated with @INSPECTED can be interpreted by the Inspector (for example, see TeleMethodAccess.interpret()). Interpretation takes place in the Inspector’s process, but in the execution context of the VM: object references are boxed locations in VM memory, reading/writing is redirected through VM data access, class ID lookup is redirected to the Inspector’s model of the VM’s class registry, and bytecodes are located using reflection on the VM’s code loaded in the Inspector.

The Inspector’s interpreter runs very slowly. It is used routinely by the Inspector in only a few situations, where VM data structures to be navigated are too complex (e.g. a hash table) to be navigated robustly using low-level techniques. For example, see the Inspector method TeleCodeCache.findCompiledCode(Address), which interprets remotely the VM method Code.codePointerToTargetMethod(Address).

Although the interpreter is in principle capable of writing into VM memory, it is not used in any situations where this happens.

Active VM support

Active VM support for inspection is kept to an absolute minimum, but in most cases either incur very little VM overhead or are enabled only when the VM is being inspected. There are several flavors of support mechanisms:

  • Distinguished fields, usually static, where the VM records information exclusively for the consumption by the inspector.
  • Distinguished methods, usually static and usually empty, called by VM code exclusively as potential breakpoint locations for the inspector; this is a weak kind of event mechanism.
  • Special VM memory locations into which the Inspector writes for consumption by specific VM mechanisms.

As a matter of organization, this kind of support is implemented mainly by VM classes in the package com.sun.max.vm.tele, but it often imposes some obligations on specific scheme implementations, for example to store a value or call a method. These obligations are increasingly specified and documented in scheme definitions.

The remainder of this section describes a few areas of active VM support for inspection.

Enabling inspection support

Many support mechanisms in the VM operate conditionally, depending on the value of static method com.sun.max.vm.tele.Inspectable.isVmInspected(). This predicate checks one of the bits in the static field Inspectable.flags in VM memory, which can be set in one of two ways:

  • When the VM is started by the Inspector, the Inspector sets that bit in VM memory early in its startup sequence (see Inspector method TeleVM.modifyInspectableFlags()).
  • When the VM is not started by the Inspector, but when it is anticipated that the Inspector might subsequently attach the VM process, a command line option to the VM makes it inspectable.

At present, the VM cannot be made inspectable unless this bit is set early during the VM startup sequence.

Inspector evolution

The Inspector’s life began long before the Maxine VM could run usefully, a period during which the novel meta-circular, highly modular architecture was refined and techniques for generating the Maxine boot image developed. The Inspector’s original role was static visualization and exploration of the binary boot image in terms of the higher level abstractions of the VM, something that could not be done by any existing tool.

As the VM became increasingly able to run through its startup (bootstrap sequence), basic debugging features were added: process controls and breakpoints, along with register and stack visualization. The Inspector remained monolithic (with no model/view separation) and single-threaded (the GUI froze during VM process execution).

As the VM began to execute application code, work on the Inspector proceeded incrementally along several fronts simultaneously:

  • features on demand: as the VM became more functional and the concerns of the development team evolved, many more features were added: additional views of internal state, more debugging controls, more user options, etc. These were, and continue to be, demand-driven according to the needs of the project.
  • UI functionality and consistency: the early window implementations were rewritten for code reuse and standardized around new conventions, the menu system was standardized and extended, Java Look & Feel compliance was added, and more.
  • re-architecting internals: model/view separation was added, direct interaction among views was replaced by a user event model, change propagation was refined, generalized notion of user selection defined, etc.

Once model/view separation became explicit in the previously monolithic code base, the Inspector sources were incrementally split into two “projects” with distinct concerns:

  • Tele: responsible for communicating with and managing the VM process, essentially being the keeper of the model of the VM’s state at any point during the session.
  • Inspector: responsible for user interaction, state visualization, and command handling.

Dependence between the two projects eventually became one-way, but remained complex: the Inspector project depends directly on many implementation classes from both the Tele and VM projects. A subsequent effort to further separate the two by re-engineering around new, well-documented interfaces is only partially complete.

As the Inspector evolved into a heavily used debugger, demand grew for multi-threaded management of the VM process so that the GUI would remain live and in particular so that a user could interrupt (“Pause”) a running VM. Concurrent operation is now supported, but the retrofit (over complex, distributed interactions in the reading and modeling of VM state) is incomplete Occasional concurrency problems appear as the VM and Inspector evolve.