Unexpected viewport input behavior for empty UI areas

Clicking on empty areas that don't have a scrollview still rotate or pan the viewport. This doesn't apply to mouse wheel scrolling for some reason. This doesn't apply to actually interacting with the UI like with dragging sliders or clicking buttons.