AMC pool crashes with C++ virtual functions - need guidance

[tunerooster]

Problem Description

We're implementing a garbage collector for an interpreter written in C++ and have found that the AMC (Automatic Mostly-Copying) pool appears to be fundamentally incompatible with C++ virtual functions. The AMS (Automatic Mark-Sweep) pool works perfectly.

Setup

• MPS version: 1.118 (latest from master)
• Language: C++ with extensive use of virtual functions
• Platform: Linux x86_64 (Gentoo)
• Compiler: g++ 11.2.0

What We've Tried

1. Implemented all required format methods for AMC:

   • scan - scans object references
   • skip - returns next object address (reads size from object field, no virtual calls)
   • fwd - stores forwarding pointer in vtable slot with bit 0 set
   • isfwd - checks forwarding bit
   • pad - creates padding objects

2. Added object size field to base class to avoid virtual function calls during GC:

   class Instance {
   public:
     size_t mps_object_size;  // Set by Alloc()
     // ... vtable pointer is here implicitly

3. Fixed signal handler conflict - we were overriding SIGSEGV which MPS needs for write barriers

4. Changed to ambiguous roots - Changed from mps_rank_exact() to mps_rank_ambig() based on Configura's documentation

5. Registered thread stack - Using mps_root_create_thread() which should handle stack/register ambiguity

The Problem

Even with these fixes, AMC crashes when regular application code tries to access objects. The crash occurs when calling virtual functions:

#0  Instance::ClassPtr() - tries to call virtual ClassDesc()
#1  Instance::IsSpecies() - calls ClassPtr()
#2  Symbol::IsEqual() - virtual function
#3  Table::Lookup() - during normal operation

Our Understanding - The Fundamental Conflict

After extensive research, we've identified what appears to be a fundamental incompatibility:

1. MPS stores forwarding pointers in the first word of objects (as documented in your Scheme example)
2. C++ stores vtable pointers in the first word of objects with virtual functions
3. When AMC moves an object, it overwrites the vtable with a forwarding pointer
4. Result: Virtual function calls crash because the vtable pointer is destroyed

We found NO documentation addressing this conflict:

• The MPS guide only shows a C example with manual type dispatch
• No C++ examples with virtual functions exist in the MPS repository
• No open-source C++ projects using MPS with virtual functions found
• This vtable-forwarding conflict is not mentioned anywhere in MPS docs

Questions

1. Is AMC designed to work with C++ virtual functions at all? The Scheme example uses C with manual type dispatch via switch statements.

2. If AMC can work with C++, what's the correct approach? Do we need to:

   • Check for forwarding before every object access?
   • Use some MPS API to "fix" pointers before use?
   • Structure our code differently?

3. Is this a documentation gap? Should the AMC documentation explicitly state it's incompatible with C++ virtual functions?

Minimal Reproduction

// With MPS_POOL_TYPE=AMC environment variable
Instance* obj = Symbol::Intern("test");  // Allocates via MPS
// ... later, after GC might have run ...
obj->IsEqual(other);  // Virtual function call - CRASH

Working Alternative

AMS pool works perfectly with the same codebase:

• No write barriers
• No object relocation
• Virtual functions work normally

What We Need

1. Is it possible to use AMC with C++ virtual functions? Given the vtable-forwarding conflict, it seems fundamentally incompatible.

2. How does Configura handle this? They claim to use MPS with C++. Do they:

   • Avoid virtual functions entirely?
   • Use a modified MPS that doesn't overwrite the first word?
   • Have a different C++ object layout?
   • Use some undocumented workaround?

3. What are our options? Should we:

   • Give up on AMC and stick with AMS (which works)?
   • Redesign our C++ code to avoid virtual functions?
   • Modify MPS to handle vtables differently?
   • Use headers to offset the vtable from the forwarding location?

Request

Please clarify whether AMC can work with C++ virtual functions, given that:

• Forwarding pointers overwrite the first word (vtable pointer)
• This breaks virtual dispatch permanently

If it IS possible, please provide guidance on the correct approach. If it's NOT possible, this should be documented as a known limitation for C++ users.

Thank you for maintaining this excellent memory management system!

[tunerooster]

Anyone?

[tunerooster]

Is this package still supported?

[dlovemore]

If your code is seeing an overwritten vtable pointer, as you identify it is because the MPS has forwarded the object.
So you are seeing the old object, not the forwarded one.
This happens when not all the roots have been identified correctly.
It can also happen when not every pointer has been scanned.
There may be static roots that have not been registered for example.
Please check the documentation on roots.

[tunerooster]

Thank you for your reply! I must assume then, that MPS AMC is compatible with C++ virtual functions and that it is errors in our implementation that are causing the problem. If so, that gives me hope and we will continue to debug. Can you confirm this? Thanks again for your help and advice!

[dlovemore]

Yes, when I worked with the MPS we used to use virtual methods just fine.

It is nearly always the case that bugs caught by the MPS are not bugs in the MPS.

So checking your roots and scanning methods is the first thing I would think of.

The debuging guide in the documentation is almost certainly worth a read:

https://memory-pool-system.readthedocs.io/en/latest/guide/debug.html

[tunerooster]

I think we have AMC correctly implemented now, but we are having trouble stress testing it.

• Is it possible, with AMC, to force a manual collection? (Sample code to do that would be vary helpful)
• Is there a minimum arena size we can set so GC gets triggered frequently just by creating lots of objects. (We set it to 64M and it just segfaults)

Thanks for any tips!

[dlovemore]

This is covered in the reference section of the documentation under the Arena topic.

https://memory-pool-system.readthedocs.io/en/latest/topic/arena.html#running-garbage-collections

[tunerooster]

Thank you! I should have read all of it...