About motion clustering

[anl13]

Hello! Thanks for your excellent work OpenMonkeyStudio.
After reading the paper, I have a question:

How did you determine the cluster labels in Fig.6A?

I know such 2D visualization spectrum is computed by UMAP. However, UMAP could not directly give you the information of per-point cluster label, right? And I didn't find a description on how to obtain the cluster label here. So I am curious about that, looking forward to your answer!

Thank you very much!

[thejanzimmermann]

Hi An,

thank you. Do you mean the semantic aspect of it ?

[anl13]

Hi An,

thank you. Do you mean the semantic aspect of it ?

Yes, the clustering result in the paper.

[christincha]

Hello I have the same question. How to get semantic labels of action sequences. Thanks!

[thejanzimmermann]

Hi there, sorry for the delay. You need an ethogram of behavior first and then you can compute the probability distribution from the cluster centers.

…

________________________________ From: Jingyuan Li <notifications@github.com> Sent: Tuesday, December 15, 2020 3:52:54 PM To: OpenMonkeyStudio/OMS_Data <OMS_Data@noreply.github.com> Cc: Jan Zimmermann <maloman@gmail.com>; Comment <comment@noreply.github.com> Subject: Re: [OpenMonkeyStudio/OMS_Data] About motion clustering (#2) Hello I have the same question. How to get semantic labels of action sequences. Thanks! — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#2 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACCBIISGIHDOJRMBHENHLC3SU7LDNANCNFSM4UULI2NA>.

[thejanzimmermann]

I think it’s also important to note that you will need large datasets to perform meaningful classifications.

…

________________________________ From: Jan Zimmermann <maloman@gmail.com> Sent: Tuesday, December 15, 2020 6:26:53 PM To: OpenMonkeyStudio/OMS_Data <reply@reply.github.com>; OpenMonkeyStudio/OMS_Data <OMS_Data@noreply.github.com> Cc: Comment <comment@noreply.github.com> Subject: Re: [OpenMonkeyStudio/OMS_Data] About motion clustering (#2) Hi there, sorry for the delay. You need an ethogram of behavior first and then you can compute the probability distribution from the cluster centers.

________________________________ From: Jingyuan Li <notifications@github.com> Sent: Tuesday, December 15, 2020 3:52:54 PM To: OpenMonkeyStudio/OMS_Data <OMS_Data@noreply.github.com> Cc: Jan Zimmermann <maloman@gmail.com>; Comment <comment@noreply.github.com> Subject: Re: [OpenMonkeyStudio/OMS_Data] About motion clustering (#2) Hello I have the same question. How to get semantic labels of action sequences. Thanks! — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#2 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACCBIISGIHDOJRMBHENHLC3SU7LDNANCNFSM4UULI2NA>.

[anl13]

Do you mean we should follow the procedure below: (1) we collect a large amount of data (2) we have clustered all data into some clusters using unsupervised method (like UMAP) ; (3) manually assign semantic label to each cluster by inspecting the cluster center; (4) for a new data, we compute the distance to cluster center to determine its own label?

Hi there, sorry for the delay. You need an ethogram of behavior first and then you can compute the probability distribution from the cluster centers.
…
________________________________ From: Jingyuan Li notifications@github.com Sent: Tuesday, December 15, 2020 3:52:54 PM To: OpenMonkeyStudio/OMS_Data OMS_Data@noreply.github.com Cc: Jan Zimmermann maloman@gmail.com; Comment comment@noreply.github.com Subject: Re: [OpenMonkeyStudio/OMS_Data] About motion clustering (#2) Hello I have the same question. How to get semantic labels of action sequences. Thanks! — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#2 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACCBIISGIHDOJRMBHENHLC3SU7LDNANCNFSM4UULI2NA.

[thejanzimmermann]

Exactly, and always best to do out of sample validation :)

[christincha]

Thanks for the reply.
I was wondering if there any ground truth label? That is labeled by some annotators.

Exactly, and always best to do out of sample validation :)

[anl13]

Thanks a lot! I still have a question.

What does "sample validation" mean?

Exactly, and always best to do out of sample validation :)

[christincha]

I still have another question. In the paper, you mentioned 3D representations are those transformed coordinated, and the y-axis is the gravity direction. Is there any information in the dataset that can provide what is the gravity direction? Or it can be some direction vertical to the z-axis.

[anl13]

I assume that, the coordinate system is: y-axis points towards the ground, which is gravity direction

I still have another question. In the paper, you mentioned 3D representations are those transformed coordinated, and the y-axis is the gravity direction. Is there any information in the dataset that can provide what is the gravity direction? Or it can be some direction vertical to the z-axis.

[thejanzimmermann]

The values here are not normalized so be careful. The data provided here is purely for model generation and inference. The actual experimental data is continuous for many hours and we re-reference the cameras intrinsic and extrinsic properties to the cages true coordinate system.

[christincha]

Thanks for replying. So is the data provided here is already in the cage coordinate system? Could you please share the clustering result?

Thanks!:)

[thejanzimmermann]

No the data is in an arbitrary reference frame that is related to one of the camera. I can check which one. We are not sharing behavioral data apart from the training data at this point as we are reading manuscripts on those datasets at the moment. Once those manuscripts are on a preprint server we are happy to share those full datasets as well but you will have to provide physical hard drives. One hour of data is 6 terabytes of image data.

…

________________________________ From: Jingyuan Li <notifications@github.com> Sent: Saturday, December 19, 2020 2:34:07 PM To: OpenMonkeyStudio/OMS_Data <OMS_Data@noreply.github.com> Cc: Jan Zimmermann <maloman@gmail.com>; Comment <comment@noreply.github.com> Subject: Re: [OpenMonkeyStudio/OMS_Data] About motion clustering (#2) Thanks for replying. So is the data provided here is already in the cage coordinate system? Could you please share the clustering result? Thanks!:) — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#2 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACCBIIWKXFOKHVIKFDHSFPTSVUE37ANCNFSM4UULI2NA>.

[christincha]

Thanks a lot for replying. Looking forward for your reply. It would be great if you can share how to use intrinsic and extrinsic camera information to register the frame.