Support for read your writes consistency

[bdionne]

On updates, an encoding of the shards that participate is returned to the client in a header X-Couch-RYW. If this header is then passed in to subsequent view queries, the encoded shards will be used.

BugzID: 11393

[kocolosk]

The "stale trumps RYW" logic would be relevant here as well, right?

[kocolosk]

All of the signature changes in fabric.erl are unfortunate. I really want to refactor that interface to accept Options as proplist and do away with the record crap. Then we could have just added this directly to Options. C'est la vie; at least we're only talking about changing the interface on a single node at a time (as opposed to modifying the RPC protocol between nodes).

We should think about how to deal with multi-valued X-Couch-RYW headers. Could certainly see a client saving two different documents and combining the X-Couch-RYW headers on a subsequent request. We ought to be able to merge those, provided we document the behavior in the case where the header values specify non-intersectin sets of copies for a shard.

[bdionne]

This last commit addresses the comment about including the current sequence number to help speed queries up. I don't like what I had to do in fabric_rpc:update_docs_seq but update_docs is also used by the internal replicator and with_db is used by several other functions.

I'll see if I can refactor this some more after dinner

[bdionne]

For the multi-valued RYW headers, we would assume the user would supply a list of values?

It seems the machinery to do this is now there with this last commit where we capture the current seq for each of the shards involved. For the multi-valued case we would then need to take the maximum of the sequences whenever we have an intersection. We also have to be careful in fabric_util:remove_stale_shards to not eliminate shards in cases where docs sharded to the same range but different nodes were preferred in the two requests. It seems a little tricky

[bdionne]

@kocolosk this has seen a lot of changes now, let me know if you'd like a rebase/squash before you have another read.

[bdionne]

@kocolosk - actually I see a better approach here, ignore this for now

[kocolosk]

@bdionne, I just realized there's a problem with your update_docs_seq function. You're returning the sequence of the database shard prior to the insertion of the documents. That doesn't actually provide RYW guarantees for a followup view query. You need to report the sequence number of the database after the documents have been written.

[bdionne]

@kocolosk - thanks for the feedback. The send_seq message handling was all motivated by the need to get the encoded seqs into the header on the first response, which made my first attempt of overloading the complete message fail.

I should be able to eliminate it and add to the total_and_offset as you suggest for the map, and something similar for the reduce.

Good catch on the update_docs_seq

[kocolosk]

Cool. I'm not so concerned with the extra message send (though it'll be nice to optimize that away) as I am with the possibility that a shard copy can "claim" some portion of the ring by sending that message even though that copy's view index is nowhere near built.

[bdionne]

@kocolosk - here's a new cut that takes your suggestions to heart. It's a little more complex. I didn't fix the update_doc_seq yet, will do next

I've only tested the easy paths and will likely refactor this more and do another squash

[kocolosk]

@bdionne can you shove the curr_seq into a proplist entry in #view_query_args.extra and avoid the modification to the record? It'll make upgrades more sane.

[kocolosk]

Aha, I missed this. You're suppressing the EncShards here, so your old code was actually correct. I'd rather keep parity with the ok case and report them to the client.