INCLUDE_BACKGROUND is confusing

[gattia]

I think walking through the below I figured out what is going on - but this was confusing to me, and in the end, I don't think it allows the user to include a background class in their loss function during training (which they might want). Im also wondering if there is an implication for using the Softmax loss.

In config.py INCLUDE_BACKGROUND default is False and indicates if True should include a background mask, if False shouldn't:

MedSegPy/medsegpy/config.py

Lines 623 to 625 in 0c316ba

	def get_num_classes(self):
	if self.INCLUDE_BACKGROUND:
	return len(self.CATEGORIES) + 1

in losses.py INCLUDE_BACKGROUND is passed directly to get_training_loss for the parameter remove_background:

MedSegPy/medsegpy/losses.py

Lines 65 to 71 in 0c316ba

	loss = get_training_loss(
	loss,
	weights=cfg.CLASS_WEIGHTS,
	# Remove computation on the background class.
	remove_background=cfg.INCLUDE_BACKGROUND,
	reduce=reduction,
	)

This seems backwards based on the names alone, and is confirmed looking at the downstream class DiceLoss where remove_background removes the background label. So, if INCLUDE_BACKGROUND=True it will have remove_background=True which means we'll actually end up removing the background and not including it?

MedSegPy/medsegpy/loss/classification.py

Lines 78 to 79 in 0c316ba

	if self.remove_background:
	y_true, y_pred = y_true[..., 1:], y_pred[..., 1:]

But looking through other files INCLUDE_BACKGROUND seems to be used in different ways. In im_gens.py it seems to be used properly to add a background class to the array.

Ok, here's where I think I started to understand? sem_seg_evaluation.py & oai_test.py both use it to determine if they should remove the background:

MedSegPy/medsegpy/evaluation/sem_seg_evaluation.py

Lines 133 to 137 in 0c316ba

	# background is always excluded from analysis
	if includes_bg:
	y_true = output["y_true"][..., 1:]
	y_pred = output["y_pred"][..., 1:]
	labels = labels[..., 1:]

MedSegPy/projects/TechConsiderations/oai_test.py

Lines 208 to 211 in 0c316ba

	if config.INCLUDE_BACKGROUND:
	y_test = y_test[..., 1:]
	recon = recon[..., 1:]
	labels = labels[..., 1:]

They both also include a comment saying "background is always excluded from analysis". So, is INCLUDE_BACKGROUND being passed to remove_background in DiceLoss because we always want to skip/not consider the background class for the loss function, and we're just telling the loss function if it exists in the data so it should be removed?

Im guessing this might have something to do with using the softmax function, otherwise Im not clear on why we'd add a background to just exclude it from the loss. Whether this is the case or not, then I'm guessing that if the activation is a softmax we should be turning include_background to be True - this might be happening somewhere but I couldn't find it. If this is the case and it isn't being done somewhere, this could also explain some of the discrepancy between using sigmoid and softmax that I've experienced in training, because from config.py it looks like the default is to exclude the background:

MedSegPy/medsegpy/config.py

Line 116 in 0c316ba

INCLUDE_BACKGROUND = False

[gattia]

Another question - I'll look around tomorrow to see what I find. How is this all affected if you specify the background as a class/category, eg categories = [0, 1, [2, 3], 4, 5]. This is probably an important part that I've skipped over somewhere.