E2e xni #3
Open
E2e xni #3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adding local versions of the Python packages leveraged by XDD's xddmcp copy tool.
The interval option used here breaks with multiple threads. Using a large interval value was just a (possibly unnecessary) hack anyway.
Before this commit a test run could leave as many as 100 (!) iperf servers running.
TODO: make the sizes configurable.
This reverts commit a666d80.
Now XNI TCP will use the same window size as the legacy end-to-end TCP implementation.
Removes an extraneous parameter to xni_register_buffer(). Removes xni_unregister_buffer() because it was unused and unimplemented.
Previously the I/O buffers were allocated in the worker threads. This commit is a necessary first step for fixing end-to-end connections with XNI. Except that this commit breaks end-to-end transfers.
The whole e2e header will be moving to XNI soon. But before then I want to make the header as small as possible.
The wire protocol won't change (htonll(x) == ntohll(x)), but of course it's always better to use the correct function for documentation purposes.
The goal is to remove the xdd_e2e_header by sending the important information in the XNI message header.
Target buffers are now allocated by the client but registered by xni_accept_connection() or xni_connect(). Now that target buffers have moved from the xni_context to the xni_connection we should be able to easily open multiple connections. With multiple connections we should be able to support multiple NICs.
Uses XNI to store and retrieve the information that was formerly held in the E2E header. The old end-to-end code is thoroughly broken now. XNI should be used in its place via the "-xni" option.
Conflicts: src/common/end_to_end.h src/net/end_to_end.c Magic cookie support was removed. Heartbeat is probably broken.
Only one connection can be used currently, but multinic will require multiple connections.
This move is neccessary because soon the buffer won't be valid inside of worker init. This change will also cause the datapattern buffers to be initialized in parallel (whereas before it was serialized inside of worker init).
Connection establishment was moved to the worker threads from the target thread. This change will allow the workers to have different connections in the future (a requirement for multinic). Currently we still only support a single connection, i.e. one remote host. A mutex protects each connection. The first worker to hold the mutex will establish the connection. The other workers will use the connection that was already established.
A target thread should wait for all of its workers to finish connecting before returning from its init function. Synchronization is accomplished with a new barrier, td_target_worker_thread_connected_barrier.
Merge multinic feature into XNI branch
After cleaning up some templates it's now possible (and highly recommended) to use autoheader(1) to generate config.h.in instead of the old method of editing it by hand. The file config checks were modified to stop generating preprocessor symbols.
This bug was exposed thanks to the new build system actually defining HAVE_VALLOC.
This file is automatically generated by autoheader(1).
bwatkinson
pushed a commit
to bwatkinson/xdd
that referenced
this pull request
Jun 22, 2023
Change Build System to CMake
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.