Packaged game : Drone camera freezing inplace

[geevargs]

[geevargs]

Ensure condition failed: TextureRHI [File:/home/airspace/UnrealEngine/Engine/Source/Runtime/OpenGLDrv/Private/OpenGLRenderTarget.cpp] [Line: 855]

[2018.03.23-21.29.16:602][592]LogStats: FPlatformStackWalk::StackWalkAndDump - 0.018 s
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: === Handled ensure: ===
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error:
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: Ensure condition failed: TextureRHI [File:/home/airspace/UnrealEngine/Engine/Source/Runtime/OpenGLDrv/Private/OpenGLRenderTarget.cpp] [Line: 855]
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error:
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: Stack:
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: [Callstack] 00 0x000000000259e0cf FLinuxPlatformStackWalk::CaptureStackBackTrace(unsigned long long*, unsigned int, void*)
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: [Callstack] 01 0x0000000002472d05 FGenericPlatformStackWalk::StackWalkAndDump(char*, unsigned long, int, void*)
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: [Callstack] 02 0x000000000259e089 FLinuxPlatformStackWalk::StackWalkAndDumpEx(char*, unsigned long, int, unsigned int, void*)
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: [Callstack] 03 0x00000000025e7148 FDebug::EnsureFailed(char const*, char const*, int, wchar_t const*)
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: [Callstack] 04 0x00000000025d344f FDebug::OptionallyLogFormattedEnsureMessageReturningFalse(bool, char const*, char const*, int, wchar_t const*, ...)
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: [Callstack] 05 0x0000000003942ba6 FOpenGLDynamicRHI::RHIReadSurfaceData(FRHITexture*, FIntRect, TArray<FColor, FDefaultAllocator>&, FReadSurfaceDataFlags)
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: [Callstack] 06 0x000000000501dffe PX4/Binaries/Linux/PX4() [0x501dffe]
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: [Callstack] 07 0x0000000005078543 PX4/Binaries/Linux/PX4() [0x5078543]
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: [Callstack] 08 0x0000000002498056 FNamedTaskThread::ProcessTasksNamedThread(int, bool)
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: [Callstack] 09 0x00000000024975ae FNamedTaskThread::ProcessTasksUntilQuit(int)
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: [Callstack] 10 0x00000000039cd32a RenderingThreadMain(FEvent*)
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: [Callstack] 11 0x00000000039db992 FRenderingThread::Run()
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: [Callstack] 12 0x00000000024cdbf7 FRunnableThreadPThread::Run()
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: [Callstack] 13 0x00000000024ab440 FRunnableThreadPThread::_ThreadProc(void*)
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: [Callstack] 14 0x00007ff02bf5d184 /lib/x86_64-linux-gnu/libpthread.so.0(+0x8184) [0x7ff02bf5d184]
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error: [Callstack] 15 0x00007ff02b0e7ffd /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff02b0e7ffd]
[2018.03.23-21.29.16:602][592]LogOutputDevice: Error:
[2018.03.23-21.29.16:602][592]LogStats: SubmitErrorReport - 0.000 s
[2018.03.23-21.29.16:639][592]LogStats: SendNewReport - 0.020 s
[2018.03.23-21.29.16:640][592]LogStats: FDebug::EnsureFailed - 0.038 s
Traceback (most recent call last):
File "", line 1, in
File "/home/airspace/DroneSimLab/UE4PyhtonBridge/src/Wrappers/pyinit.py", line 43, in PyTick
next(main_loop_iter)
File "../../DroneSimLab/demos/unreal_proxies/two_drones/unreal_proxy.py", line 117, in main_loop
img=ph.GetTextureData(drone_textures[drone_index])
File "/home/airspace/DroneSimLab/UE4PyhtonBridge/src/Wrappers/phandlers.py", line 102, in GetTextureData
tmp_capture_mem=np.zeros(req_mem_sz,'uint8')
MemoryError
ending main loop!!

[geevargs]

I ran a few test cases to diagnose the error:
It appears that the numpy array is full beyond its capacity and thus giving this error.
It only seems to be a problem in packaged games in my 2nd PC it crashes regardless of data being sent to the FDM but this problem is not seen in my dev platform.
after 1min of running.
tmp_capture_mem=np.zeros(req_mem_sz,'uint8')
MemoryError

I believe this error has occurred because I ran it for weeks continuously

I have linux 14,04 and have 12 gb free space on my PC.
Unreal 4.17.2

[orig74]

Try to add 2 lines between 101 and 102 in the file "/home/airspace/DroneSimLab/UE4PyhtonBridge/src/Wrappers/phandlers.py",
del tmp_capture_mem # free memory
print('requested size id:',req_mem_sz) #for debug

[geevargs]

I did the print('requested size id:',req_mem_sz) #for debug
requested size id: 4
requested size id: 8294400

[geevargs]

del tmp_capture_mem # free memory didnt work...should i redo set_path.py...
This occurs only in binary mode on one Pc but works fine on my dev platform.

[orig74]

Can you track the memory usage during the simulation on both systems (the dev platform and the other PC)? You can do it with tools like system monitor. It will help us understand the difference.

[geevargs]

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
13978 airspace 20 0 6954920 2.059g 195052 R 208.4 13.2 3:34.98 PX4 ----- production platform
19340 sujith 20 0 6126620 2.213g 205272 S 177.4 7.1 1:02.32 MyProject2+ ------dev platform

[geevargs]

The %memory usage is more than 200% and is 170+ in the dev platform.
Everything else is similar.

[geevargs]

Production platform:

[geevargs]

Dev platform

[orig74]

can you look if the memory usage is increasing?
show me the resource graphs from the moment you open the application until it fails on both stations.

[geevargs]

Production platform:

[geevargs]

Dev Platform:

[orig74]

I don't really understand why it fails since I can't see any memory issue. but let's try and bypass the problem for now.
try and change line 82 in "phandlers.py" to something like:
tmp_capture_mem=np.zeros(8294400,'uint8')
this hopefully will eliminate the need to reallocating the memory (line 102).