How the Inspector interacts with the Maxine VM¶
This page describes how the Maxine Inspector’s interaction with a running VM is implemented.
General goals for the Inspector all deal with making development and experimentation in the Maxine VM as productive and widely accessible as possible:
- Support basic debugging of the Maxine VM, something not possible with standard tools.
- Make visible as much internal VM state as possible, both in terms of design abstractions and in terms of low-level representations, even when the VM is broken.
- Provide new developers with a fast path to understanding VM internals.
A few general strategies guide the Inspector’s implementation:
- Run in a separate process (usually local, but see Guest VM), so that inspection depends neither on a correctly running VM process nor there being any process that all.
- Require as little active support from the VM as possible, in particular require no active agent.
- Reuse as much VM code as possible, especially reading/writing/understanding the low-level representations of data on the target platform (possibly different than the platform on which the Inspector runs).
- Load VM classes into the Inspector for reflective use in understanding VM data.
- Rely on platform-specific implementations for low-level interaction with a running VM: process control, threads, breakpoints, access to memory and registers.
Low-level VM Interaction¶
This section describes the Inspector’s access to the lowest level abstractions in the VM process, namely the resources provided by the underlying platform: memory, threads, and execution state.
Process control¶
One of the most difficult and frustrating parts of the Inspector’s implementation is the need to implement low-level process controls on the several supported platforms. These controls include reading and writing from the VM’s memory, finding and inspecting threads, setting breakpoints, setting watchpoints, and deciphering process state.
Generic controls are implemented in class
com.sun.max.tele.debug.TeleProcess
.
Concrete subclasses using native methods implement the controls for
specific platforms:
- Solaris: platform support is best on Solaris, where libproc
provides a programmatic interface to the Solaris
/proc
pseudo-filesystem. Watchpoints are supported with no limit on their number (seeNative/tele/darwin/*.[ch]
). - Linux: the Inspector uses a mixture of
ptrace(2)
and/proc
(seeNative/tele/linux/*.[ch]
). - Mac OS X: on the Mac the Inspector uses a mixture of
ptrace(2)
and the Mach API (seeNative/tele/darwin/*.[ch]
). - Guest VM: the Guest VM variant of the VM runs in a Xen domain where such OS services are unavailable, so controls must be implemented using Xen inter-domain communication.
This code can be very subtle. It now seems to work fairly reliably, but at the cost of many hours deciphering non-documentation and gdb source code. In our experience, programming a debugger is a very niche activity.
Reading and writing VM memory¶
Low-level memory access is implemented using basic process control
methods in class TeleProcess
:
read(Address address, BtyeBuffer buffer, int offset, int length)
write(Address address, ByteBuffer buffer, int offset, int length)
However, interpreting the bits presents more of a challenge, since this
must be done for a VM running on a potentially different
platform.
Fortunately, the Inspector is able to load the Java classes that
describe the target platform and then reuse the VM’s own code for
reading and writing bits representing the VM’s internal primitive data
types.
Methods for reading and writing those types appears in interface
com.sun.max.tele.data.DataAccess
, and all but the lowest-level read
methods are implemented by class
com.sun.max.tele.data.DataAccessAdapter
.
For performance reasons, especially for non-local debugging such as with
the Guest VM, the Inspector caches pages of memory read since the most
recent process execution (see class
com.sun.max.tele.page.PageDataAccess
).
Logging¶
The Inspector’s low-level interaction with the VM process can be
observed.
See Low-level logging for instructions
on enabling all low-level VM logging.
In order to observe only Inspector-related events, change log_TELE
to 1
in Native/share/log.h
, rather than log-ALL
.
Passive VM support¶
Although the Inspector is designed to rely as little as possible on the internals of the VM, there are a number of ways in which the VM is constructed to make inspection as easy as possible. The mechanisms described in this section incur zero runtime overhead in the VM, and involve no writing into VM memory.
Locating critical VM resources¶
The Inspector leverages considerable knowledge of the VM’s internal data representations to build its model of VM state, but it must have somewhere to start when beginning to read from a memory image. The boot image generator stores in the boot image header a number of addresses and other data that help the Inspector (and VM) find things. These addresses get relocated, along with the contents of the heap, during Bootstrap. The Inspector leverages detailed knowledge of the header’s contents in order to locate, among others:
- the VM’s schemes bindings, which are loaded into the Inspector
- the boot heap
- the boot code region
- the class registry
- the list of dynamically allocated heap segments
- the list of thread local areas
- the entry location of key methods
Field access¶
The Inspector uses a variety of mechanisms to locate instance or class fields in the heap. During the Inspector’s starting sequence (when little is yet known about VM state), fields are typically located by relying on specific knowledge of a few key object types, possibly using Java reflection on the VM classes (which are all loaded into the Inspector). This kind of access is relatively unsafe, since it bypasses the type system in the running VM. There are more abstract ways to access fields, but they rely on the Inspector’s model of VM’s class registry, which must first be created using the low-level mechanisms.
The simplest way to exploit higher-level field access mechanisms is to
annotate (in VM code) fields of interest using @INSPECTED
.
The main method in com.sun.max.tele.field.TeleFields
reads VM sources,
generates field access methods, and writes them back into itself for use
by the Inspector.
These access method implementations hide all the indirections necessary
to read or write field data (taking into account the hardware platform,
the layout being used, the particular representation for the object, and
the class layout) and return values of the desired types.
Method access¶
The Inspector uses a variety of mechanisms to locate methods and their
compilations (either instance or class).
Specific methods can be called out for enhanced access by the Inspector
by annotating (in VM code) those methods using @INSPECTED
.
The offline program TeleMethods reads VM sources, generates method
access methods, and writes them into class
com.sun.max.tele.method.TeleMethods
.
These access method implementations hide all the indirection necessary
to locate the annotated methods and their meta-information.
Method interpretation¶
VM methods annotated with @INSPECTED
can be interpreted by the
Inspector (for example, see TeleMethodAccess.interpret()
).
Interpretation takes place in the Inspector’s process, but in the
execution context of the VM: object references are boxed locations in VM
memory, reading/writing is redirected through VM data access, class ID
lookup is redirected to the Inspector’s model of the VM’s class
registry, and bytecodes are located using reflection on the VM’s code
loaded in the Inspector.
The Inspector’s interpreter runs very slowly.
It is used routinely by the Inspector in only a few situations, where VM
data structures to be navigated are too complex (e.g. a hash table) to
be navigated robustly using low-level techniques.
For example, see the Inspector method
TeleCodeCache.findCompiledCode(Address)
, which interprets remotely the
VM method Code.codePointerToTargetMethod(Address)
.
Although the interpreter is in principle capable of writing into VM memory, it is not used in any situations where this happens.
Active VM support¶
Active VM support for inspection is kept to an absolute minimum, but in most cases either incur very little VM overhead or are enabled only when the VM is being inspected. There are several flavors of support mechanisms:
- Distinguished fields, usually static, where the VM records information exclusively for the consumption by the inspector.
- Distinguished methods, usually static and usually empty, called by VM code exclusively as potential breakpoint locations for the inspector; this is a weak kind of event mechanism.
- Special VM memory locations into which the Inspector writes for consumption by specific VM mechanisms.
As a matter of organization, this kind of support is implemented mainly
by VM classes in the package com.sun.max.vm.tele
, but it often imposes
some obligations on specific scheme implementations, for
example to store a value or call a method.
These obligations are increasingly specified and documented in scheme
definitions.
The remainder of this section describes a few areas of active VM support for inspection.
Enabling inspection support¶
Many support mechanisms in the VM operate conditionally, depending on
the value of static method
com.sun.max.vm.tele.Inspectable.isVmInspected()
.
This predicate checks one of the bits in the static field
Inspectable.flags
in VM memory, which can be set in one of two ways:
- When the VM is started by the Inspector, the Inspector sets that bit
in VM memory early in its startup sequence (see Inspector method
TeleVM.modifyInspectableFlags()
). - When the VM is not started by the Inspector, but when it is anticipated that the Inspector might subsequently attach the VM process, a command line option to the VM makes it inspectable.
At present, the VM cannot be made inspectable unless this bit is set early during the VM startup sequence.
Inspector evolution¶
The Inspector’s life began long before the Maxine VM could run usefully, a period during which the novel meta-circular, highly modular architecture was refined and techniques for generating the Maxine boot image developed. The Inspector’s original role was static visualization and exploration of the binary boot image in terms of the higher level abstractions of the VM, something that could not be done by any existing tool.
As the VM became increasingly able to run through its startup (bootstrap sequence), basic debugging features were added: process controls and breakpoints, along with register and stack visualization. The Inspector remained monolithic (with no model/view separation) and single-threaded (the GUI froze during VM process execution).
As the VM began to execute application code, work on the Inspector proceeded incrementally along several fronts simultaneously:
- features on demand: as the VM became more functional and the concerns of the development team evolved, many more features were added: additional views of internal state, more debugging controls, more user options, etc. These were, and continue to be, demand-driven according to the needs of the project.
- UI functionality and consistency: the early window implementations were rewritten for code reuse and standardized around new conventions, the menu system was standardized and extended, Java Look & Feel compliance was added, and more.
- re-architecting internals: model/view separation was added, direct interaction among views was replaced by a user event model, change propagation was refined, generalized notion of user selection defined, etc.
Once model/view separation became explicit in the previously monolithic code base, the Inspector sources were incrementally split into two “projects” with distinct concerns:
- Tele: responsible for communicating with and managing the VM process, essentially being the keeper of the model of the VM’s state at any point during the session.
- Inspector: responsible for user interaction, state visualization, and command handling.
Dependence between the two projects eventually became one-way, but
remained complex: the Inspector
project depends directly on many
implementation classes from both the Tele
and VM projects.
A subsequent effort to further separate the two by re-engineering around
new, well-documented interfaces is only partially complete.
As the Inspector evolved into a heavily used debugger, demand grew for multi-threaded management of the VM process so that the GUI would remain live and in particular so that a user could interrupt (“Pause”) a running VM. Concurrent operation is now supported, but the retrofit (over complex, distributed interactions in the reading and modeling of VM state) is incomplete Occasional concurrency problems appear as the VM and Inspector evolve.