NewsRecommendation contex vectors generator

[DemirTonchev]

It seems that NewsRecommendation generates the same context vector for each arm.
Reading LinUCB(2011 Li et al.) paper I have the impression that the context vector should be different (at least at some extend depending the arms features) for each arm. I am mistaken? Or the the implementation is that just for convenience?

ts_tutorial/src/news_recommendation/env_recommendation.py

Lines 39 to 44 in 3e229f7

	context = []
	context_vector = np.random.binomial(1,max(0,1/(self.dim-1)),self.dim)
	context_vector[0] = 1
	for i in range(self.num_articles):
	context.append(context_vector)
	self.current_rewards[i] = 1/(1+np.exp(-self.thetas[i].dot(context_vector)))

[iosband]

Thanks for raising this!

Yes I think you are right, and this looks incorrect to me.
I am now checking if there is any disconnect between this public repo and the code we ran for the experiments.

[DemirTonchev]

Any update on the issue?

[iosband]

Abbas Kazerouni has been managing this section of the code/experiment and we've discussed this.
He's having a look at this now - apologies for the delay.

[abbas-kazerouni]

Hello @DemirTonchev. Thanks for raising the question and apologies for delayed response.

The variable 'context_vector' in the code here represents only the user's context vector (z_t in 7.1 of the TS tutorial). As such, it will be generated once a user arrives and the rewards associated to different arms (i.e., articles) is computed for the same user with his/her contexts being fixed.

In our formulation, the characteristics of different arms are captured by unknown \theta_x vectors which are different across x's.

[DemirTonchev]

@abbas-kazerouni thanks for clarifying that. I though it was the z_{t, x} vector in TS tutorial. Do you have any code/simulation examples with the later type of characterizing the contextual vector?
Thanks for the TS tutorial paper :)

[abbas-kazerouni]

In the paper and the codes, we have considered a version of the problem where the context z_t only changes over time and is the same for all arms. Although, generalizing the algorithms to the more complex setting of Li et al. (2010) should be straightforward.