ArcGIS Pro Temporary Directory Errors Break Python Script Execution

[danmaps]

Summary

When executing via the arcgispro_cli terminal add-in, terminals crash with "Directory does not exist or cannot be accessed" errors referencing ephemeral ArcGISProTemp* directories. These directories are session-specific and get cleaned up when ArcGIS Pro closes or crashes, causing subsequent ops to fail.

Environment

• ArcGIS Pro Version: [Unknown - capture from environment]
• Python Environment: ArcGIS Pro Python (arcpy)
• Add-in: arcgispro_cli terminal launcher
• OS: Windows 11 (Windows_NT)

Problem Description

When using a terminal opened using the "Terminal" button, the following error occurs if ArcGIS Pro is closed:

ERROR  Directory does not exist or cannot be accessed: C:\Users\mcveydb\AppData\Local\Temp\ArcGISProTemp27980

Root Cause

1. ArcGIS Pro creates session-specific temp directories (e.g., ArcGISProTemp27980)
2. These directories are referenced by arcpy for scratch workspace
3. When ArcGIS Pro restarts or temp folders are cleaned, the directory reference becomes stale
4. Scripts executed via CLI inherit these invalid workspace settings
5. Operations fail because arcpy cannot access the non-existent temp directory

Impact

• Python scripts fail unexpectedly with cryptic errors
• User experience is degraded - users don't understand why scripts worked before but fail now
• Workarounds require manual intervention (restarting ArcGIS Pro, setting scratch workspace)
• Critical limitation: Users cannot restart ArcGIS Pro to fix the issue because closing ArcGIS Pro terminates the CLI session

Steps to Reproduce

1. Open ArcGIS Pro
2. Launch arcgispro_cli terminal add-in
3. Run a Python script that uses arcpy geoprocessing tools
4. Close ArcGIS Pro (or wait for temp directory cleanup)
5. Reopen ArcGIS Pro (new session = new temp directory)
6. Launch arcgispro_cli terminal add-in again
7. Run the same Python script
8. Observe error: Directory does not exist or cannot be accessed: C:\Users\...\ArcGISProTemp[old_PID]

Expected Behavior

Scripts executed through the CLI should:

• Use a valid, persistent workspace
• Automatically detect and use the current ArcGIS Pro session's temp directory
• Gracefully handle stale temp directory references
• Not require manual workspace configuration for basic operations

Actual Behavior

• Scripts fail with directory access errors
• User must manually configure scratch workspace in ArcGIS Pro settings
• No automatic recovery or detection of invalid workspace paths
• Error messages are not actionable (don't tell user how to fix)

Suggested Solutions

Option 1: Set Default Scratch Workspace on CLI Launch (Recommended)

When the arcgispro_cli add-in launches the terminal, automatically configure arcpy environment:

import arcpy
import os

# On CLI initialization, set reliable scratch workspace
project_dir = arcpy.mp.ArcGISProject("CURRENT").homeFolder
if project_dir:
    scratch_gdb = os.path.join(project_dir, "scratch.gdb")
    if arcpy.Exists(scratch_gdb):
        arcpy.env.scratchWorkspace = scratch_gdb
        arcpy.env.workspace = scratch_gdb

Pros:

• Transparent to user
• Uses project-relative path (portable)
• Scratch.gdb is standard ArcGIS Pro convention
• Persists across sessions

Cons:

• Requires add-in to inject environment setup

Option 2: Validate and Auto-Fix Temp Directory References

Before executing user scripts, validate that referenced temp directories exist:

import arcpy
import os

# Validate scratch workspace
scratch = arcpy.env.scratchWorkspace
if scratch and not os.path.exists(scratch):
    # Auto-fix: clear invalid reference
    arcpy.env.scratchWorkspace = None
    arcpy.ClearEnvironment("scratchWorkspace")
    # Let arcpy create new temp directory

Pros:

• Fixes stale references automatically
• Minimal intervention
• Fallback to arcpy defaults

Cons:

• Doesn't prevent initial error
• Relies on arcpy to create new temp directory

Option 3: Display Clear Error Message with Resolution Steps

If temp directory error detected, intercept and provide helpful message:

try:
    # User script execution
    exec(user_code)
except Exception as e:
    if "ArcGISProTemp" in str(e) and "does not exist" in str(e):
        print("ERROR: ArcGIS Pro temporary workspace is invalid.")
        print("SOLUTION: Set scratch workspace in ArcGIS Pro:")
        print("  Options → Geoprocessing → Scratch Workspace")
        print("  Set to: {project_dir}\\scratch.gdb")
        print("\nOr add to your script:")
        print("  arcpy.env.scratchWorkspace = r'{project_dir}\\scratch.gdb'")
    raise

Pros:

• Helps user understand and fix issue
• Educational
• Minimal code changes

Cons:

• Still requires user intervention
• Error still occurs

Option 4: Create and Use CLI-Specific Scratch Workspace

Create a dedicated scratch workspace for CLI operations:

import arcpy
import os

cli_scratch = r"C:\Users\{username}\.arcgispro\cli_scratch.gdb"
if not arcpy.Exists(cli_scratch):
    arcpy.management.CreateFileGDB(
        os.path.dirname(cli_scratch),
        "cli_scratch.gdb"
    )
arcpy.env.scratchWorkspace = cli_scratch

Pros:

• Completely independent from ArcGIS Pro session state
• Persistent across sessions
• Predictable location

Cons:

• Additional maintenance (cleanup, versioning)
• Not project-specific

Recommended Implementation

Combine Options 1 + 3:

1. On CLI Launch: Automatically set scratch workspace to {project_dir}\scratch.gdb
2. On Error: Detect temp directory errors and provide clear guidance
3. User Override: Allow users to override with their own scratch workspace if needed

Implementation in arcgispro_cli Add-in

# In CLI initialization code (when terminal launches)

import arcpy
import os
import sys

def initialize_arcpy_environment():
    """Set up reliable arcpy environment for CLI execution"""
    try:
        # Get current project
        aprx = arcpy.mp.ArcGISProject("CURRENT")
        project_dir = aprx.homeFolder
        
        if project_dir:
            # Use project's scratch.gdb
            scratch_gdb = os.path.join(project_dir, "scratch.gdb")
            
            # Create if doesn't exist
            if not arcpy.Exists(scratch_gdb):
                arcpy.management.CreateFileGDB(project_dir, "scratch.gdb")
            
            # Set as default workspace
            arcpy.env.scratchWorkspace = scratch_gdb
            arcpy.env.workspace = scratch_gdb
            
            print(f"✓ CLI workspace set to: {scratch_gdb}")
        else:
            print("⚠ Warning: Could not detect project directory. Scratch workspace not configured.")
            
    except Exception as e:
        print(f"⚠ Warning: Could not configure scratch workspace: {e}")
        print("  You may need to set arcpy.env.scratchWorkspace manually in your scripts.")

# Call on CLI startup
initialize_arcpy_environment()

Additional Considerations

Cleanup Strategy

If CLI-specific scratch workspace is used:

• Implement periodic cleanup (delete old temp data)
• Add user preference for scratch location
• Monitor disk usage

Documentation

Update CLI documentation to mention:

• Automatic scratch workspace configuration
• How to override if needed
• Best practices for workspace management

Testing

• Test with projects in various locations
• Test with missing/locked scratch.gdb
• Test with permissions issues
• Test error handling and user messaging

Related Issues

• Users cannot restart ArcGIS Pro to fix the issue because it terminates the CLI session
• Need to maintain CLI session across ArcGIS Pro restarts (separate issue?)
• Scratch workspace conflicts with ArcGIS Pro UI operations?