Use a 4x4 transformation matrix

We currently separately keep track of the rotations and the shifts. We probably should use a 4x4 transformation matrix:
```
[[ a11, a12, a13, dx],
[a21, a22, a23, dy],
[a31, a32, a33, dz],
[0,0,0,1],
]
```