[Question] Duplicate data?

[japrogramer]

Since all of the data is mirrored along the diagonal of the resulting image, could there be a better method of displaying the data without repetition?

[colinmorris]

Maybe! The obvious thing to do would be to show only half of the matrix.

But there's something nice about being able to fix a point on the diagonal and look forward and backward in time for matches by going right/left along the row (or down/right along the column). If it only showed the top-right half of the matrix, then you would look right to go forward in time, and up to look back, which is kind of counterintuitive.

I'm open to ideas though.

[waldyrious]

I wondered about this as well.

The most common method for visualizing these sort of structures seems to be arc diagrams, which were apparently first popularized by Martin Wattenberg (@wattenberg) in his 2001 project in Shape Of Song.

A similar approach, which can be thought of as a circular wrapping of the arc diagrams, are chord diagrams, which were brilliantly used by Paul Lamere (@plamere) in his 2012 The Infinite Jukebox, using repetitions in the actual sound, rather than the lyrics. (Actually, the experiment was so popular that it even sparked an entire subreddit, and is now hosted at "The Eternal Jukebox".)

I suppose these may be, if not easier to follow, at least prettier than the self-similarity matrices (which btw Wattenberg calls dotplots in his paper).

[waldyrious]

Also, Wikipedia points out that adjacency matrices of undirected graphs also produce such symmetric square representations. It might be of interest to explore some of the labeled graph representations for such matrices mentioned in the article, such as the Nauru graphs or the Cayley graphs. They certainly seem to be more compact.

[japrogramer]

Grate links, The adjacency matrices seem to be a good match, for visualization. However the chord diagrams are easier to process at a single glance. With the chord diagrams type representation if the same branching approach is used that was in 'The Eternal Jukebox' by @plamere than I imagine new lyrics could be generated.

[essenmitsosse]

I came up with an idea, that would remove the redundancy.

Here is an example for All you need is love

Classic Version

New Version

The way that is done is also the main advantage in my opinion: We have two clear axis. Left to right is the progression on the song. Top to bottom is present to future. So on the top from left to right we have the actual progression of the song. Then from each pixel down we see the future of that part (word). If it is repeated we see how far in the future it will be repeated.

If several words are repeated in order, they create lines. Diagonal lines only appear when words are repeated directly after each other (thats what creates squares in the classic version).

Also if you scan down an individual column its easier to see the repeating pattern for that individual word (or sentence), which was harder to spot in the classic version.

Now all the important information are either horizontal or vertical, instead of everything being diagonal, which has more resemblance to diagrams or charts most people are used to.

Basically its just a shifted version of the classic visualization, where we drop the 'past'. The disadvantage is it doesn't look so nice (lack of symmetry).

[waldyrious]

I like @essenmitsosse's version. I find it easier to wrap my head around the meaning of the image, but to be honest it still takes some effort to interpret the repeating structures. Probably with the interactive version this would be easier.

On a slightly related note: Have you though about a different coloring mode, with a rainbow progression from start to finish, so that it is easier to identify to what part of a song a given repetition came from? I suspect that may help.

[japrogramer]

If the chart needs to be interactive as the cursor passes over the image, the color that represents the word could become highlighted everywhere that it occurs. It could incorporate two cursors that move as the song progresses, one vertical and one horizontal. The vertical cursor can move from left too right and the horizontal cursor from top to bottom, both highlighting the current word.

[essenmitsosse]

Here is another version, that shows the future of a word, as well as the past.

[colinmorris]

@essenmitsosse those concepts are really cool! I agree that it is nicer to have repeated sequences appear as horizontal lines rather than diagonals. I do find it a little harder to conceptually relate these visualizations to the overall structure of the strong, but that might be because I've spent so much time with the original orientations.

If anyone feels ambitious enough to make a SongSim fork that uses something like these 'tilted' song matrices, they absolutely have my blessing. (I'd also be open to a P.R. to add this as an optional mode that can be toggled on)

[colinmorris]

On a slightly related note: Have you though about a different coloring mode, with a rainbow progression from start to finish, so that it is easier to identify to what part of a song a given repetition came from? I suspect that may help.

This is basically how the 'colorful' mode works right now. I use a color map and assign colors to words in order of their first appearance. So words that first appear in the same place will be assigned very similar colors. It's especially clear in a song like this for example.

Or did you have something else in mind?

You could assign a unique color to every point on the main diagonal (i.e. every word of the song), but then how do you decide how to color points off the diagonal?

[waldyrious]

I don't have a good solution to offer, unfortunately. The colors simply don't look like a sequence most of the time, to my eyes at least. It could be due to the black words breaking the progression.

Maybe a different color scale could help? Say, Viridis or some other color map aimed at displaying continuous data.